embench / embench-iot

The main Embench repository
https://www.embench.org/
GNU General Public License v3.0
248 stars 101 forks source link

json-comma and no-json-comma option cause error #156

Open fanghuaqi opened 2 years ago

fanghuaqi commented 2 years ago

Hi there,

When I tried to print json output of size information, and found this strange logic here https://github.com/embench/embench-iot/blob/master/benchmark_size.py#L158-L168

This will cause the following error when I run ./benchmark_size.py --json-output:

$ python3 --version
Python 3.7.3

./benchmark_size.py --json-output
  "size results" :
  { "detailed size results" :
    { "aha-mont64" : 1.56,
      "crc32" : 0.77,
      "cubic" : 6.09,
      "edn" : 1.26,
      "huffbench" : 1.86,
      "matmult-int" : 1.17,
      "md5sum" : 12.58,
      "minver" : 2.82,
      "nbody" : 2.79,
      "nettle-aes" : 1.45,
      "nettle-sha256" : 1.74,
      "nsichneu" : 1.43,
      "picojpeg" : 1.71,
      "primecount" : 0.79,
      "qrduino" : 1.56,
      "sglib-combined" : 1.41,
      "slre" : 1.49,
      "st" : 2.96,
      "statemate" : 1.04,
      "tarfind" : 1.87,
      "ud" : 2.49,
      "wikisort" : 1.65
    },
    "size geometric mean" : 1.85,
    "size geometric standard deviation" : 1.85
Traceback (most recent call last):
  File "./benchmark_size.py", line 440, in <module>
    sys.exit(main())
  File "./benchmark_size.py", line 429, in main
    embench_stats(benchmarks, raw_data, rel_data, 'size', opt_comma)
  File "/home/hqfang/workspace/software/embench-iot/pylib/embench_core.py", line 279, in embench_stats
    output_stats(geomean, geosd, georange, count, bm_type, opt_comma)
  File "/home/hqfang/workspace/software/embench-iot/pylib/embench_core.py", line 266, in output_stats
    log.info('  }{oc}'.format(oc=opt_comma))
ValueError: Single '}' encountered in format string

And I debug into it, found this json_comma is false, and but according to the code, it should be true. So I changed dest='json_comma' to dest='json_comma2', and found evaluate the args, and found there will be two values as below(json_comma=False, json_comma2=True), it seems the argparse package default convert the --json-comma argument to json_comma:

(Pdb) p args
Namespace(absolute=False, baselinedir='baseline-data', bss=[], builddir='bd', data=[], format='elf', json_comma=False, json_comma2=True, logdir='logs', metric=[], output_format=<output_format.JSON: 1>, rodata=[], text=[])
(Pdb) q

So could we just keep one argument --no-json-comma and remove the unused --json-comma

Thanks

fanghuaqi commented 2 years ago

Found this error below is caused by https://github.com/embench/embench-iot/blob/master/pylib/embench_core.py#L266, can be fixed by

  File "/home/hqfang/workspace/software/embench-iot/pylib/embench_core.py", line 266, in output_stats
    log.info('  }{oc}'.format(oc=opt_comma))
ValueError: Single '}' encountered in format string
-        log.info('  }{oc}'.format(oc=opt_comma))
+        log.info('  }}{oc}'.format(oc=opt_comma))