ltechkorea / mlperf-inference

Reference implementations of MLPerf™ inference benchmarks
Other
0 stars 0 forks source link

[BUG] 정확도 계산시 오류 #60

Open lyly403 opened 3 years ago

lyly403 commented 3 years ago

Describe the bug

A clear and concise description of what the bug is.

Screenshots or Logs

If applicable, add screenshots to help explain your problem.

Please cite the following paper when using nnUNet:
Fabian Isensee, Paul F. Jäger, Simon A. A. Kohl, Jens Petersen, Klaus H. Maier-Hein "Automated Design of Deep Learning Methods for Biomedical Image Segmentation" arXiv preprint arXiv:1904.08128 (2020).
If you have questions or suggestions, feel free to open an issue at https://github.com/MIC-DKFZ/nnUNet

Loading necessary metadata...
Loading loadgen accuracy log...
Traceback (most recent call last):
  File "accuracy-brats.py", line 156, in <module>
    main()
  File "accuracy-brats.py", line 129, in main
    predictions = load_loadgen_log(log_file, output_dtype, dictionaries)
  File "accuracy-brats.py", line 85, in load_loadgen_log
    assert len(predictions) == len(dictionaries), "Number of predictions does not match number of samples in validation set!"
AssertionError: Number of predictions does not match number of samples in validation set!

Expected behavior

A clear and concise description of what you expected to happen.

Possible Solution

  1. LoadGen build
  2. benchmark run 구동시 --accuracy option 활성화

Additional context

Add any other context about the problem here.

lyly403 commented 3 years ago

run.py option

  1. default = performance
  2. --accuracy = create mlperf_accuracy.json file performance 디렉토리, accuracy 디렉토리 총 2개 생성 필요.

**accuracy.py 옵션값 (txt파일 생성 << 제출시 필요) python3 accuracy-brats.py --log_file ./build/logs/mlperf_log_accuracy.json --preprocessed_data_dir ./build/preprocessed_data/ --postprocessed_dadir ./build/postprocessed_data/ --output_dtype float16 2>&1 | tee accucay.txt

jay-huh commented 3 years ago

@lyly403 수정되었으면 이슈 닫아 주세요.