how about val.py file!

[urbansound8K]

How to use val.py the same way you introduced train.py?

What I mean can that be used for val.py? or just the same way as the official yolov5 after training you use val.py for validation ? please help me out.

[sebastianvitterso]

We mostly followed the original yolov5-way, so if you've managed to train your model, then you can see in val.slurm on what command we ran (we ran this using slurm, which is a HPC tool for supercomputers/server farms).

In short:

python val.py --weights "runs/train/YOUR_MODEL/weights/best.pt" --img 1280 --save-txt --save-conf --data YOUR_YAML.yaml.

Is this correct, @ingin97 ?

[urbansound8K]

I am trying to train the model. I had a look at what you have made. but I am trying to train the ch4 with depth not IR!

do you have any idea how to make that? what I have is that RGB images a long with the estimated depth of each image.

any idea please?

[ingin97]

@urbansound8K It does not matter what type of information is stored in the fourth channel. Though it must be stored as a image, e.g. if the fourth channel is an image with a depth value for every pixel in the image, then you can put the depth image the seperate folder ir, and you should be fine. If it is just a number per image, then you probably don't have to go as far as we have done in this repo, though you still have to do some modifications.

[urbansound8K]

thanks a lot

i am working on it i got what you have said getting some error will let you know once I am done

[urbansound8K]

i would real appreciate if you could share more defiles with what you have done in the four-input branch

still going into it. managed to train for some epochs

[ingin97]

@urbansound8K If you managed to train for some epochs things seems to be working alright. You are sure you have enough memory for the size of your batch size and img size? What is the error you are getting?

[urbansound8K]

hi there! things seem to be fine i am training will keep you update once I get the results thanks a lot

[urbansound8K]

Hi @ingin97 I have done some training for datasets I will share some of the results I got firstly, I took the depth of the dataset I wanted to train. the extension was .png. Then, I trained for 300 epochs and the results was almost similar to the model which was trained with 3 channels!!! and I have looked at the val.py in your repo I noticed you have made some changes. then, I used the official val.py and checked the performance of the two models (3 n 4ch), found the same results too!! any idea please?

[sebastianvitterso]

Might I ask what kind of data of which we are speaking? If the depth data isn't helpful to solve the issue, then you might not see a large bump in effectiveness.

For example: When we worked on this, we had some issues with the network not improving too much when adding IR-data, because our IR images were so low scale (160px x 120px). Additionally, some of our IR-images basically held no relevant data, even if they were captured from the same area at the same time (simply because there were no registered heat signatures).

I believe that in your case, it could be the case that your depth data might be a bit flat, leading to the network not noticing the connection between the values in the depth images and the shapes in the RGB-images. I don't have a solution for you, but I just wanted to point out that if the data isn't good, then the network can't do too much more with RGB+depth than it could for RGB.

Could you possibly post some examples of RGB+depth image pairs here, so we can understand a bit more?

[urbansound8K]

sure man! I have been going around, looking for what you have pointed. I believe the way you load the fourth channel was correct. it is just the way I got the depth.

I would really appreciate if you could help me out

I have the seahsip7000 dataset. I used MiDaS to get the depth. Then, I have made the same thing in your code. It took almost 60 hours to train and I could notice that the the way you introduce could load the data correctly. here are some examples of the images https://drive.google.com/drive/folders/1O8WoK-06o3D1GMYuU2YamJQEbMq8FOjL

[sebastianvitterso]

Alright, so here's what I think is the issue: Your depth-data does not really add any new information, because it is generated using the information found in the RGB-images. This means that the total actual information does not increase, and the network doesn't really learn anything from this. I'm sure the MiDaS-network is cool and very helpful if you wish to directly use the depth-data (e.g. for adding depth-of-field to a video render), but for this use case, I don't think it will help the network learn.

I also see in the example images that the depth is pretty "vague" and non-focused. It's impossible to e.g. pick out the red buoy in the middle of the image, when looking at the depth data.

RGB:

Depth:

I am sure depth is a good data source for this network, but I sadly think you need more detailed images, for example data from a 2D lidar image.

[urbansound8K]

thanks a lot it is clear right now.

[sebastianvitterso]

Alright, sorry we couldn't be of more help, hopefully you find a solution that helps you!

Is there anything else we can help with? Otherwise I'll close the issue.

[urbansound8K]

sorry for bothering again.

I am trying again then I suddenly removed all the file and got the thing back, but got this error again

I could not pass it strange!!

I tried what you have mentioned in the closed issue! but got

any help please?

[urbansound8K]

thank you very much guys you have done great work i have been going through the code trying fix it again

[sebastianvitterso]

I'll have to assign @ingin97 to this - I'm not sure if I see an immediate issue.

[ingin97]

@urbansound8K You are sure you are still loading both RGB and depth images? Though if the depth images is just processed from the RGB images and not containing any actual extra information, I think you might be better off using the original yolov5 repository.

[sebastianvitterso]

Closing this for inactivity.