Problems with Training

[Oceandeepzxr]

Hi, I got a problem when I start training, here is the error message:

Im using Python3.8, tensorflow2.5 with cuda 11.8(Changing the version doesnt seem possible since im using a public server). Could you tell me how this error message may come from?

[xychen2022]

It is difficult to identify the problem without you providing more information. Have you changed the default setting and/or the data?

Anyway, you can try on the provided data first and this will familiarize yourself with our algorithm.

[Oceandeepzxr]

Thanks for the reply, Im using provided data, here is full log for running training code 'python landmark_detection_group1_size24.py':

please tell me if something Im missing, thanks in advance.

[xychen2022]

The second screenshot has told you the problem. Since your environment is different than ours (Tensorflow 2.5, CUDA 11.2 and cuDNN 8.1), you need to compile the two CUDA C codes, 3D ROI-align and non-maximum suppression, before using the algorithm, which is already mentioned in ReadMe. To compile the sources codes, please follow the instructions given in the ReadMe under folders CropAndResize3D and non_max_suppression. Then, change the paths in landmark_detection_group1_size24.py or replace crop_and_resize_op_gpu.so and non_max_suppression_op.so in the current folder with the ones you got after recompiling.

[Oceandeepzxr]

Thanks! Im working on it, I'll let you know if theres any problem 👍 :D

[CarrieX6]

@xychen2022 Hi, I'm having the same issue. I'm still getting the same bug after recompiling two C files. Could you tell me how it was resolved? Thanks a lot!