search

sovit-123 / vision_transformers

Vision Transformers for image classification, image segmentation, and object detection.
MIT License
41 stars 7 forks source link

RuntimeError: CUDA error: device-side assert triggered #21

Open kawaiiGTR opened 3 months ago

kawaiiGTR commented 3 months ago

Trying to run DETR on custom dataset. When executing the launch command:

The output is:

RuntimeError: CUDA error: device-side assert triggered

CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

After some digging the error either points to class mismatch or problematic activation function. Either way it doesn't seem to work!! :(

Any advise on how to get this working? Cheers!

sovit-123 commented 3 months ago

Hello @kawaiiGTR Can you please let me know how many classes you have and if you can provide the your custom dataset YAML file information here?

kawaiiGTR commented 3 months ago

I think there is a mismatch in Linear out between Input shape [2,100,92] and the output shape [2.100,114]

sovit-123 commented 3 months ago

Were you able to solve it?

kawaiiGTR commented 3 months ago

I don't know where e.g. what file I need to edit to change the size of the input layer to match the output layer. Could you kindly advise? I have 114 classes not 92.. Cheers!

sovit-123 commented 3 months ago

Is it possible for you to provide me link to the dataset? I will be able to debug it if I have the dataset.