Error when submit result to semantic scene completion benchmark

[sj-li]

Hi:

I submit my result to semantic scene completion, but get following erroe:

Traceback (most recent call last):
  File "/worker/worker.py", line 313, in run
    bundles = get_bundle(root_dir, 'run', bundle_url)
  File "/worker/worker.py", line 173, in get_bundle
    metadata[k] = get_bundle(bundle_path, k, v)
  File "/worker/worker.py", line 173, in get_bundle
    metadata[k] = get_bundle(bundle_path, k, v)
  File "/worker/worker.py", line 133, in get_bundle
    z.extractall(bundle_path)
  File "/usr/local/lib/python2.7/zipfile.py", line 1040, in extractall
    self.extract(zipinfo, path, pwd)
  File "/usr/local/lib/python2.7/zipfile.py", line 1028, in extract
    return self._extract_member(member, path, pwd)
  File "/usr/local/lib/python2.7/zipfile.py", line 1084, in _extract_member
    shutil.copyfileobj(source, target)
  File "/usr/local/lib/python2.7/shutil.py", line 66, in copyfileobj
    fdst.write(buf)
IOError: [Errno 28] No space left on device

However, my zip file pass through the test by the script offered by you. What should I do?

[jbehley]

Hi @sj-li,

Thanks for reporting this issue and I looked into this.

Your submission file is unfortunately to large. You have to provide only the predictions for the scans in the testset. (A submission is something around 120-150 MB zipped and 16 GB unzipped.)

Please ensure that the file containes only labels for the files from the testset, i.e., 11/000000.bin, 11/000005.bin, 11/000010.bin, etc.

I will modify the verification script to check that a zip file contains only the necessary files.

[jbehley]

Please consult the official zip file: http://www.semantic-kitti.org/assets/data_odometry_voxels.zip

The test set contains every 5th scan. Therefore your submission.zip should also only contain this data.

I updated the validation script to handle these cases.

[jbehley]

I deleted your failed submissions on Codalab, since it was an issue from our side. Hope that helps.

[sj-li]

Got it! I will try it now.

And I have another question, I see in evaluation code, you use:

    # Map labels "pred_labels" and "gt_labels" from semantic-kitti ID's to [0 : n_classes -1]
    pred = remap_lut[pred]
    target = remap_lut[target]

But my predictions already only contains [0 : n_classes -1], So I need to adjust label by myself?

[jbehley]

The submission should contain our original labels (car = 10, etc.). How you map the data for training is an implementation issue that you have to figure out.

We expect to get the original label ids, therefore you have to remap the labels before submission.

[sj-li]

Thanks for your help, I submit successful ;)