mlcommons / inference_results_v3.0

This repository contains the results and code for the MLPerf™ Inference v3.0 benchmark.
https://mlcommons.org/en/inference-datacenter-30/
Apache License 2.0
18 stars 15 forks source link

Bert binary files are taking too much size in 3 of the submitter codes #2

Open arjunsuresh opened 1 year ago

arjunsuresh commented 1 year ago

Three of the submitter codes are taking more than 3G each and this makes it hard to clone the inference_results repository. All of these corresponds to bert binary files inside the code directory as shown below.

arjun@hp-envy:~/inference_results_v3.0/closed$ du -shB 1M * | sort -nr
4025    Nettrix
3623    H3C
3528    GIGABYTE
569 NVIDIA
395 Dell
184 xFusion
130 Supermicro
85  Qualcomm
68  Krai
62  HPE
61  Neuchips
58  Inspur
55  Intel
52  ASUSTeK
30  cTuning
29  Azure
28  Quanta_Cloud_Technology
22  Lenovo
8   SiMa
4   rebellions
arjun@hp-envy:~/inference_results_v3.0/closed/Nettrix/code$ du -shB 1M * | sort -nr
1162    bert-99.9
1162    bert-99
1162    bert
2   retinanet
2   harness
1   rnnt
1   resnet50
1   README.md
1   plugin
1   main.py
1   __init__.py
1   dlrm-99.9
1   dlrm-99
1   dlrm
1   common
1   actionhandler
1   3d-unet-99.9
1   3d-unet-99
1   3d-unet
psyhtest commented 1 year ago

Great point, @arjunsuresh. But even if the files are removed now, the repo size will remain the same. This should be done at the point of converting the results repo from private to public at the latest.

arjunsuresh commented 1 year ago

Yes @psyhtest . I don't see an automatic way to do this - I'll add this (total code size being reasonable) as a check to be done manually during the review period and an issue can be raised against the respective submitter. Do you have any better way to handle this?

Currently we are using a forked repository like this

git clone https://github.com/cknowledge/inference_results_v3.0.git --depth 1