Regarding 3D Unet Pre Process Data

[Sweeton-JK]

AssertionError: Some of the target inference cases were found: {'case_00111', 'case_00400', 'case_00185', 'case_00052', 'case_00065', 'case_00000', 'case_00084', 'case_00076', 'case_00157', 'case_00044', 'case_00005', 'case_00034', 'case_00056', 'case_00171', 'case_00041', 'case_00049', 'case_00078', 'case_00207', 'case_00112', 'case_00169', 'case_00189', 'case_00092', 'case_00187', 'case_00012', 'case_00206', 'case_00176', 'case_00138', 'case_00066', 'case_00003', 'case_00162', 'case_00203', 'case_00086', 'case_00006', 'case_00125', 'case_00024', 'case_00087', 'case_00128', 'case_00160', 'case_00061', 'case_00161', 'case_00198', 'case_00070', 'case_00080'} make: *** [Makefile:204: preprocess_data] Error 1

I ran the make setup after that when I ran make preprocess_data it shows the above error. How to rectify it?

[nv-jinhosuh]

https://github.com/mlcommons/inference/blob/a248595cf55d367a68a973bbe8616e4fb1f7f81f/vision/medical_imaging/3d-unet-kits19/Makefile#L198

What is your $RAW_DATA_DIR and what files do you have in there?

https://github.com/mlcommons/inference/blob/a248595cf55d367a68a973bbe8616e4fb1f7f81f/vision/medical_imaging/3d-unet-kits19/preprocess.py#L248

It looks like your dataset inside $RAW_DATA_DIR might not have all the files you need to have for inference task (https://github.com/mlcommons/inference/blob/master/vision/medical_imaging/3d-unet-kits19/meta/inference_cases.json).

[Sweeton-JK]

But I ran the make setup which have downloaded all the necessary files. I have just followed the steps on it and it throwed this error.

[Sweeton-JK]

Hi @nv-jinhosuh I have moved the raw data to the $RAW_DATA_DIR, the pre process worked fine. But it got stuck on certain pre process case.

Thanks !

[nv-jinhosuh]

It might be working on, so please give some minutes to see further progress is made. :)

[Sweeton-JK]

@nv-jinhosuh It was still for the past 6 hrs.

[nv-jinhosuh]

That sounds strange. I wouldn't expect it to take more than 10~30min total. Can you break and retry?

[Sweeton-JK]

@nv-jinhosuh

Yeah have tried that too.

• Actually at first doing the make setup the entire raw data were stored in /inference/vision/medical_imaging/3d-unet-kits19/kits19_raw_data_dir/kits19/data.
• So I have moved the entire data to /build/raw_data/ folder.
• Then ran the command make preprocess_data and got the above issue. Is it mandatory to run make build_docker and make launch docker commands

[nv-jinhosuh]

I think it's a good idea following the directions in the reference model step by step including launching docker as it's likely that the whole setup is supposed to work in docker environment.

[Sweeton-JK]

Sure then will do that

[Sweeton-JK]

Hi @nv-jinhosuh

It still got stuck there even after running the docker commands

[arjunsuresh]

Many of the content on the READMEs are outdated as it is very difficult to maintain them without automation. The best way to run the reference implementation is by following the commands given here which takes care of downloading the dataset, preprocessing, downloading the model and then doing the inference.