MD5 speed test not working on Mac

[Roger-Shepherd]

Timing reporting mechanism fails to report timing. The log file looks like

Warning: Failed to find timing Args to subprocess: sh '-c' './md5sum; echo RET=$?' cbecbdb0fdd5cec1e242493b6008cc79 ... 147899 lines more of the same cbecbdb0fdd5cec1e242493b6008cc79 Real time: 693.355000 ms CPU time: 693.312000 ms RET=1

Under investigation; failure happens for x86 Mac and M1 running ARM or x86 code

[Roger-Shepherd]

The problem is that the benchmark is writing to std out in benchmark_body. The Mac scripts assumes the only info on standard out is timing info. I think the print statements should be removed, or at least put under a debugging option.

[hirooih]

Hi,

I took a look at the code. The following patch should fix this issue. I don't have Mac and cannot test it by myself. Could you try this?

--- a/pylib/run_mac.py
+++ b/pylib/run_mac.py
@@ -64,7 +64,7 @@ def decode_results(stdout_str, stderr_str):
         return 0.0

     # Match "Real time: dd.ddd"
-    time = re.search('^Real time: (\d+)[.](\d+)', stdout_str, re.S)
+    time = re.search('^Real time: (\d+)[.](\d+)', stdout_str, re.M)
     if time:
         ms_elapsed = float(time.group(1) + '.' + time.group(2))
         # Return value cannot be zero (will be interpreted as error)

I think the print statements should be removed, or at least put under a debugging option.

I agree with you. The current md5sum test is measuring the performance of printf() :-)

[Roger-Shepherd]

I’ll take a look

Roger

On 20 Oct 2021, at 13:56, hirooih @.***> wrote:

Hi,

I took a look at the code. The following patch should fix this issue. I don't have Mac and cannot test it by myself. Could you try this?

--- a/pylib/run_mac.py +++ b/pylib/run_mac.py @@ -64,7 +64,7 @@ def decode_results(stdout_str, stderr_str): return 0.0

 # Match "Real time: dd.ddd"

• time = re.search('^Real time: (\d+).', stdout_str, re.S)
• time = re.search('^Real time: (\d+).', stdout_str, re.M) if time: ms_elapsed = float(time.group(1) + '.' + time.group(2))

  Return value cannot be zero (will be interpreted as error)

  I think the print statements should be removed, or at least put under a debugging option.

I agree with you. The current md5sum test is measuring the performance of printf() :-)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/embench/embench-iot/issues/149#issuecomment-947638441, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC24LZO6I6WZBEI2JO2SQB3UH237PANCNFSM5GGO3MSA.

-- Roger Shepherd @.***

[jeremybennett]

I think the fix proposed by #152 addresses the use of external library functions.

[hirooih]

This is what Roger and I wrote.

I think the print statements should be removed, or at least put under a debugging option. I agree with you. The current md5sum test is measuring the performance of printf() :-)

I still recommend you to replace re.S with re.M in pylib/run_*.py.

https://docs.python.org/3/library/re.html#re.S https://docs.python.org/3/library/re.html#re.M

[Roger-Shepherd]

The change to pylon/run_*.y suggested does allow the benchmark to run.

We still need to remove the print statements though.