search

Plant-Tracer / webapp

Client and Server for web-based JavaScript app
GNU Affero General Public License v3.0
0 stars 2 forks source link

Mao U-Net ML custom model training #158

Open sbarber2 opened 10 months ago

sbarber2 commented 10 months ago

The Mao U-Net ML model implementation in https://github.com/YixiangMao/PlantTracerML is the only method we currently know that can reliably track arabidopsis circumnutation.

However, the model provided is specific to the experimental setup on which the model was trained. Students and researchers who want to use this tracking method will likely want a way to train their own models using their own experimental setup.

While it is of course possible for people who are well versed in creating predictive models using Python tools to create such models on their own, we don't believe such people will be common in botany classes or research labs. Therefore, PlantTracer as a plant movement tracking platform can provide guidance and tools to create such models from a set of training and testing videos.

This is a very broad and big feature, by the way and needs a lot more definition and tool selection.

This Issue does not included deploying the custom models once trained and exported as model artifacts. That will be another issue #159

simsong commented 10 months ago

Can you provide a reference movie (and ideally 5-10) that demonstrate the problematic circumnutation?

sbarber2 commented 9 months ago

Here's one. Go to http://circumnutation.umcs.lublin.pl/en.html and download the video that is captioned "Film 1"

Then clone Plant-Tracer/klt. cd ml, follow the README instructions to set up dependencies, etc, and run: python3 PAT.py

Load the downloaded Film 1.mp4, Select Apex, Confirm, Track. Maybe try selecting multiple apex boxes on separate runs. Probably have to re-load the video into PAT for each run.

Note that the circumnutation sample videos PlantTracer has appear to be drawn from the same training set used to create the PAT model. I'm not aware of other circumnutation video sets, but Eric Brenner can probably help with that.

simsong commented 9 months ago

Our options are putting the URLs in the repo or putting in the movies.Apparently there are a lot of different kinds of movies to make and we probably want two cameras

On Tue, Nov 21, 2023 at 7:48 AM Steve Barber @.***> wrote:

Here's one. Go to http://circumnutation.umcs.lublin.pl/en.html and download the video that is captioned "Film 1"

Then clone Plant-Tracer/klt. cd ml, follow the README instructions to set up dependencies, etc, and run: python3 PAT.ml

Load the downloaded Film 1.mp4, Select Apex, Confirm, Track. Maybe try selecting multiple apex boxes on separate runs. Probably have to re-load the video into PAT for each run.

Note that the circumnutation sample videos PlantTracer has appear to be drawn from the same training set used to create the PAT model. I'm not aware of other circumnutation video sets, but Eric Brenner can probably help with that.

— Reply to this email directly, view it on GitHub https://github.com/Plant-Tracer/webapp/issues/158#issuecomment-1820863202, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMFHLBPVBZDMPZAKRPITHTYFSPKXAVCNFSM6AAAAAA7TTYA32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRQHA3DGMRQGI . You are receiving this because you commented.Message ID: @.***>

--

sbarber2 commented 9 months ago

I'm afraid I don't understand how "Our options are putting the URLs in the repo or putting in the movies" relates directly to add model training features to the platform. Is this a separate Issue?

"probably want two cameras" -- this I think refers to a 3-D tracking, which is a different kind, or at least aspect, of tracking, and probably deserves a separate Issue (or set of Issues).

simsong commented 9 months ago

The issue is that we need a corpus of images for model training so that when we compare models we are comparing apples-to-apples. That corpus is either a bunch of movies/images in the git repo, or it's the URLs for movies/images that are downloaded and cached as part of the training.

two-cameras — yes, it deserves another issue. But it was the contents of the web page that you shared, and it was most impressive.

sbarber2 commented 9 months ago

OK, thanks. I've added #161 and #162 in response to your comments here. As I've noted in the Issues themselves and similar indeed to the scope of this Issue, these two ideas can both initiate projects that require substantial work and experimentation, and are not mere well-defined features to be implemented and added to application per se. Indeed, one might posit that they are research programs in and of themselves.