added imaging tutorial - Githubissues

shiltemann commented 4 years ago

I just ran through the tutorial, it was fun, thanks!

I would maybe add some of the png image outputs to the manual, they are cool, and if people don't have time to finish or if they mess up, then they can at least see the output they were supposed to get, what do you think?
The input dataset is large (500Mb) and we only use one image (2.6Mb) from it, could we just upload the single file to zenodo? or if unzipping is something you want to teach them, maybe just create a zip file with only that image and possibly one or two others? it will save some time on import and some disk space on the server
Please still add a workflow file btw (since I just performed the tutorial I could extract it from my history too if you prefer)

again, very cool, thanks a lot!

ThomasWollmann commented 4 years ago

I just ran through the tutorial, it was fun, thanks!

I would maybe add some of the png image outputs to the manual, they are cool, and if people don't have time to finish or if they mess up, then they can at least see the output they were supposed to get, what do you think?

Yes this would be nice! Since, you already have the result images you can maybe just upload them :)

The input dataset is large (500Mb) and we only use one image (2.6Mb) from it, could we just upload the single file to zenodo? or if unzipping is something you want to teach them, maybe just create a zip file with only that image and possibly one or two others? it will save some time on import and some disk space on the server

Soon, I will add a second tutorial, which is using the whole dataset and should be teached along with this tutorial. Therefore, I think we can just keep the large dataset. Also makes sense to teach people from the beginning that imaging datasets are often not just single files.

Please still add a workflow file btw (since I just performed the tutorial I could extract it from my history too if you prefer)

Would be nice if you can add it, since you have it already. Thank you in advance!

again, very cool, thanks a lot!

You're welcome! Also checkout https://www.denbi.de/training/73-microscopy-image-analysis-course were we will teach a whole course on microscopy imaging and hopefully will use this workflow.

shiltemann commented 4 years ago

@ThomasWollmann ok, no problem, I will add the images and workflow on Monday :)

and cool about the workshop! have a great weekend!

shiltemann commented 4 years ago

Hi @ThomasWollmann I added the images and workflow, could you please check if it looks ok to you?

In the final image, I get a yellow label of 65535 on the image, is that meant to happen or did I do something wrong somewhere? and if not, maybe explain what this label signifies?

ThomasWollmann commented 4 years ago

Hi @ThomasWollmann I added the images and workflow, could you please check if it looks ok to you?

In the final image, I get a yellow label of 65535 on the image, is that meant to happen or did I do something wrong somewhere? and if not, maybe explain what this label signifies?

Yes, I missed a step. I'll add it to the tutorial and then you can try again ;)

ThomasWollmann commented 4 years ago

@shiltemann I changed the description. Can you please try it again and update the results?

ThomasWollmann commented 4 years ago

@shiltemann I added the extra step and some questions regarding this issue to the tutorial. Do you think 30 minutes is too optimistic for this tutorial?

ThomasWollmann commented 4 years ago

@shiltemann Currently, I'm preparing the second tutorial. Should I push it to the same PR?

shiltemann commented 4 years ago

@ThomasWollmann whatever you prefer. If you're ok with this staying open a little longer then go ahead and add it here :)

ThomasWollmann commented 4 years ago

@ThomasWollmann whatever you prefer. If you're ok with this staying open a little longer then go ahead and add it here :)

Since you're already into the topic, we can just process both at once ;)

ThomasWollmann commented 4 years ago

@shiltemann Now, I added the second tutorial. It closer to a real use case. I know that it still lacks some biological motivation, which I'll add later if your are satisfied with by technical description.

ThomasWollmann commented 4 years ago

@shiltemann I talked to the guy who imaged the dataset. I think I can change the second tutorial to make it closer to a real use case and also reduce the dataset size. Just give me some time to adapt the tutorial ;)

ThomasWollmann commented 4 years ago

@shiltemann I changed the dataset and the description. I think the second tutorial is now more useful for practitioners. Please take a look!

ThomasWollmann commented 4 years ago

@bgruening Now, the splitted dataset is small and only two subsets are required for the tutorials. This should solve the issues for EU?

bgruening commented 4 years ago

@bgruening Now, the splitted dataset is small and only two subsets are required for the tutorials. This should solve the issues for EU?

Did you had issues? I could run the other completely. I would like to keep the old datasets and workflow for testing. I think its a cool example where we can benchmark the system.

ThomasWollmann commented 4 years ago

@bgruening Now, the splitted dataset is small and only two subsets are required for the tutorials. This should solve the issues for EU?

Did you had issues? I could run the other completely. I would like to keep the old datasets and workflow for testing. I think its a cool example where we can benchmark the system.

I just took time. The workflow is still the same. Now I splitted the dataset into the different treatments and in the course you can actually compare phenotypes visually or apply machine learning on the extracted features.

The full dataset is still on Zenodo. I think it is a good example for preparing Galaxy for large scale imaging datasets. Keep in mind that this dataset is still very small ;)

shiltemann commented 4 years ago

thanks a lot @ThomasWollmann :) The second tutorial looks good, I like that they also learn about workflows a bit at the same time :)

I reworked the last hands_on section of the first tutorial a bit, because using the normalized image resulted in higher IDs that overlapped in the final image and didnt look as nice

so please check that it all still looks ok to you :)

ThomasWollmann commented 4 years ago

thanks a lot @ThomasWollmann :) The second tutorial looks good, I like that they also learn about workflows a bit at the same time :)

I reworked the last hands_on section of the first tutorial a bit, because using the normalized image resulted in higher IDs that overlapped in the final image and didnt look as nice

so please check that it all still looks ok to you :)

Hmm since the image contains many objects, we probably should not display the ids. In the overlay tool you can set "Plot Labels" to "No". I think this looks much more useful. @shiltemann What do you think?

@shiltemann Thank you for putting so much effort in improving the quality of the tutorials!

shiltemann commented 4 years ago

oh, that was just how it looked before, to explain why I changed it back. ..without the histogram equalization step it looks quite ok now:

So I just reverted to your original steps, and made the histogram equalization step an exercise to help explain the completely black looking image they get, but use the original image for the segmentation mask overlay. What do you think?

ThomasWollmann commented 4 years ago

oh, that was just how it looked before, to explain why I changed it back. ..without the histogram equalization step it looks quite ok now:

So I just reverted to your original steps, and made the histogram equalization step an exercise to help explain the completely black looking image they get, but use the original image for the segmentation mask overlay. What do you think?

Sounds good! Now everything looks fine :)

ThomasWollmann commented 4 years ago

@shiltemann @bgruening still something missing for this tutorial?

shiltemann commented 4 years ago

@shiltemann @bgruening still something missing for this tutorial?

Hi @ThomasWollmann oh, you mentioned you were planning to add more biological motivation so I thought you were still working on this

Could you just add some key_points to the tutorial metadata? just the main conclusions/take home messages you want people to remember from the tutorials.

Also: do we need to cite the person who created the dataset somehow? was it part of a publication or project we can refer to?

ThomasWollmann commented 4 years ago

@shiltemann The data was created by Manuel Gunkel as part of de.NBI Systematic Phenotyping. I do not think that we need additional citations, since we have the zenodo record.

ThomasWollmann commented 4 years ago

@shiltemann Now, I added some more motivation and key_points ;)

shiltemann commented 4 years ago

@ThomasWollmann thanks! This looks great!

ThomasWollmann commented 4 years ago

Wow, @shiltemann awesome work. Thank you for the support!

shiltemann commented 4 years ago

:tada:

galaxyproject / training-material

added imaging tutorial #1534