search

biomag-lab / hypocotyl-UNet

MIT License
11 stars 2 forks source link

how to prepare validation dataset for the training.py command? #3

Open amoudomola opened 4 years ago

amoudomola commented 4 years ago

Hi. Tivadar,

I finally came back with some results. After I realized that I need GUI (do I?), I somehow figured out the no-screen problem.

Then, I have used about 1000 seedlings (image size: 5184 x 3456) grown under red light to obtain masks for hypocotyl and non-hypocotyl.

train-horz2

The mask.py script gave me multiple lines of warnings as below which I ignored at the moment.

Blowing up the brightness/contrast didn't solve the issue. I might have to try lowering the "depth" or "bitrate" of my images to get more sharp image boundaries.

mask.py:22: UserWarning: /home/pdbl_lab_members/goh/unet/unet_train/train_export_highcontr/converted/masks/train002.png is a low contrast image
  io.imsave(export_path, mask)
mask.py:22: UserWarning: /home/pdbl_lab_members/goh/unet/unet_train/train_export_highcontr/converted/masks/train007.png is a low contrast image
  io.imsave(export_path, mask)

I have used two options --device cuda and --batch_size 5 for train.py command. The 1000 epoch training was done within a day with the minimum training loss about 0.038.

I ran measure.py command with following options --min_object_size 80 --device cuda --visualize True. It also gave me some warnings, but finished the measurements anyway. It might be a kind of outdated package issue. But, I have been procrastnating package update, worried that I might get the black screen back which I still don't know exactly how I fixed it. If it's not that serious issue, I'll leave it like that for a while.

Measuring _MG_4551.png
/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_gtk3.py:197: Warning: Source ID 7 was not found when attempting to remove it
  GLib.source_remove(self._idle_draw_id)
/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_gtk3.py:197: Warning: Source ID 8 was not found when attempting to remove it
  GLib.source_remove(self._idle_draw_id)
/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_gtk3.py:197: Warning: Source ID 9 was not found when attempting to remove it
  GLib.source_remove(self._idle_draw_id)

I compared the model provided with the model I trained. It looks like my set-up or model is not ready yet!

Upper was done with your model, lower with my model. measure-tile

Obviously, your model performance was better than mine especially in the sense of recognizing the whole part of the seedling. But, it mis-annotated some of the root-parts as hypocotyl segments. It is probably because the model looked at those as dark-grown seedlings, and painted blue all the way down.

My training set only has red-light grown seedlings, so the mis-annotation didn't happen to the measurements with my model. However, I got more "broken" seedlings. I'm not sure that can be fixed by increasing my training images.

I'm also thinking of using scanned images, hoping that it removes the bright spots and reflection near the seedlings with my current camera setting. If I use white box, I can remove the ghost images. But, not sure the script allows bright background. What do you say? I'll be appreciated any suggestions if you have some.

In addition, I also want to know how to prepare validation data to use the --val_dataset flag for the training.

Thanks in advance, Goh

cosmic-cortex commented 4 years ago

Hi!

Sorry for not reacting sooner, I was extremely busy. Can you contact me at niwcf52g4gv5@opayq.com ? I think it would be better to discuss these things via email, as I am quicker to respond there.

(Don't be surprised with the seemingly strange email, this is completely valid. It is just a masked email address, since I don't want to put out my personal email. Once you have messaged me, I will reply from my real email and this address won't be valid anymore.)