Submission format question and no result

[Cattalyya]

I submitted a prediction 12 hrs ago to test out the format, but the status still only showed "submitted" without any result/stdout/stderr. May I ask what might go wrong and how to debug locally?

Also, from the submission page, the formats of 1) groupings predicted part-mat pairs and 2) groupings labels for each point are {n-shapes x max-groups x 2} and {n-shapes x n-points x 2} respectively which are not the same as sizes shown in HDF5 schema:

    "part_mat_pairs": array(shape=[n_shapes * max_groups * 2], type=int16)
    "point_grouping": array(shape=[n_shapes * n_points],       type=uint8)

nor those in example submission code on the same page:

train_hdf5.create_dataset('part_mat_pairs',
                            shape=(n_shapes * MAX_GROUPS, N_POINTS),
                            dtype='int16')
train_hdf5.create_dataset('point_grouping',
                            shape=(n_shapes, N_POINTS),
                            dtype='uint8')

May I ask what are the correct formats?

[HabibSlim]

Hello @Cattalyya, thanks for your interest in the challenge :)

1) Thanks for reporting the typos! These mistakes from the doc are now fixed. To clarify things a bit:

• The part_mat_pairs array is a list of part-material groupings (where each grouping is in the form of a vector with two entries: a part label and a material label)
• The point_grouping array associates each point with a group index, where the index maps to the part_mat_pairs array

2) I have reran the submission worker and your submission did exit, but with an output error. I have checked your submission and it appears that the format of the part_mat_pairs array is incorrect, it should be of dimensions n_shapes x max_groups x 2.

3) You can pad the part_mat_pairs matrix by -1 (not 0 as in your submissions) to get the evaluation script to automatically evaluate the part-material pairs by maximum voting.

4) There should be a total of 6770 entries in your submission (there are 677 shapes in the validation set, and the challenge is evaluated with 10 compositions per shape). Your submission has 6784 which causes a mismatch with our ground truth.

Here is what an example submission should look like for the validation set:

Best, Habib

[Cattalyya]

Thank you so much Habib for your quickly resolve my issue! Really appreciate it and thank you for correcting the submission page as well. :D

[HabibSlim]

No worries, and sorry again about these typos :smile: