search

microsoft / scene_graph_benchmark

image scene graph generation benchmark
MIT License
385 stars 86 forks source link

CUDA error: device-side assert triggered #61

Open DavidHuji opened 2 years ago

DavidHuji commented 2 years ago

Hi,

I get the following error when debugging with VG and RelDN:

IndexKernel.cu:93: operator(): block: [157,0,0], thread: [32,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.

The trigger is from scene_graph_benchmark/relation_head/reldn/reldn.py line number: 103 at: proposal_per_image.get_field("subj_box_features")[rel_ind_i[:, 0]] Actually the indexs at rel_ind_i are too big for proposal_per_image.get_field("subj_box_features").

It looks like something was not configured correctly in my env. I have downloaded the weights to the right path though.

Any idea?

sliu-github commented 2 years ago

@DavidHuji You can just set the batch size to 1 with "TEST.IMS_PER_BATCH 1". Line 98 "rel_ind_i += offset" causes the problem. If you read the for loop, you will see how setting the batch size to 1 solves the problem.