search

weigertlab / trackastra

BSD 3-Clause "New" or "Revised" License
55 stars 12 forks source link

Command line interface? #4

Closed tinevez closed 5 months ago

tinevez commented 6 months ago

Would it be possible to have a command line interface for this?

I am asking egoistically: This would make it 'easy' to include this work in TrackMate... Totally understand if it is not a priority nor a goal.

bentaculum commented 5 months ago

Hi @tinevez, thanks for the suggestion. It would be quite cool to integrate into TrackMate! I drafted a tracking CLI trackastra track ... in #5, currently supporting tiff sequences. Would this be useful? Happy to have this evolve a bit before merging.

tinevez commented 5 months ago

Yes it would absolutely awesome! I am sitting on the train and won't be able to test. But basically this is the way to go.

Here is how I envision the integration into TrackMate for now, with a few questions for you.

Also:

  1. Does trackastra work in 3D? Can I resave a single IJ tiff per 3D stack?

I am so excited!

bentaculum commented 5 months ago

This looks totally doable and would be very cool!

Trackastra pretrained models are still WIP, so some glitches in tracking performance might happen. I'm very curious where we might still have trouble with linking, it would be great if you could give feedback for datasets where it's brittle.

Thanks for the detailed description, trying to answer below:

Yes it would absolutely awesome! I am sitting on the train and won't be able to test. But basically this is the way to go.

Here is how I envision the integration into TrackMate for now, with a few questions for you.

  • The movie (cropped by the user) is resaved by TrackMate as individual tiffs in a temp folder. What it will save are label images, typically uint16 with the objects labeled with a unique int > 0 (background is 0).
    1. Would that work for trackastra?
    2. There would be one tiff file per frame. Do you need it to be something else?
    3. Does the label needs to be unique over the whole movie?

Yes, a tiff file per frame is the current base input as well. We also need the corresponding raw images as input. The labels don't have to be unique.

  • Java makes a system call that runs trackastra cli on the folder.

  • When it completed, TrackMate loads the output of trackastra and uses it to create the links.

    1. What will be the output from trackastra?

Right now, the default output to disk is in the cell tracking challenge format, that should work right? To avoid going via writing relabelled dense masks to disk, one could use the trackastra.tracking.TrackGraph object we have, which corresponds to the input masks, and write that into a table that TrackMate can process.

Ideally, I would simply need some kind of edge tables, a list of ids to connect in the track graph. If the label image is resaved with unique IDs in the movie, I could process a list like:

ID_source, ID_target
12, 23
556, 7

I could make it so that the these IDs correspond to the internal TrackMate spot IDs, and directly retrieve them from a map.

Also:

  1. Does trackastra work in 3D? Can I resave a single IJ tiff per 3D stack?

We currently have one pretrained model ctc that works for both 2d and 3d.

I am so excited!

🎉

tinevez commented 5 months ago

I noticed I will have to make a major API change in TrackMate to include Trackastra (which I am totally willing to do).

The tracker interface of TrackMate only received the 'spots' (the detected / segmented objects) and some parameters: https://github.com/trackmate-sc/TrackMate/blob/master/src/main/java/fiji/plugin/trackmate/tracking/SpotTrackerFactory.java#L46

It does not receive the image. But I understand from the article that Trackastra additionally require the raw image as an input. Is that correct?

maweigert commented 5 months ago

From the raw image, we actually only need the mean intensity (after normalizing the whole image) per object, not the raw pixels. I guess trackmate has access to that feature? If not, we tried on a few timelapses to just set the intensity to 1 and it surprisingly still worked. In worst case, we could train a model that doesn't need intensity (as we found this feature might actually not be that necessary)

tinevez commented 5 months ago

Ok but it's still worthwhile to include this feature for now.

Just to be sure, to the compute this mean intensity:

maweigert commented 5 months ago

yes! (normalization is done with 1/99.8th percentile)

tinevez commented 5 months ago

ok this I can do no issue.

Shall I provide it in a csv file, mapping label -> mean stuff?

maweigert commented 5 months ago

Mhm, can one save the images in the temp folder too? Or if that information is not there anymore, one could construct from the mask image a "fake" intensity image where every object contains the constant pixel value of its objects mean?

tinevez commented 5 months ago

I think I should be good now, and that we can finish it when we have the CLI.

On the TrackMate side, the tracker will do the following:

A prototype yields the following: https://github.com/weigertlab/trackastra/assets/3583203/9e63c49d-5152-4d42-ad04-37519689612a

We 'just' need the CLI :)

bentaculum commented 5 months ago

This is really cool :) ! I just merged v0.1 of the CLI in #5. Is the CLI working for you as is, or should we make some more changes? I'd be happy to adapt, since TrackMate is the first CLI user.

tinevez commented 5 months ago

Yep it's all good but I would like to have an argument added to specify that the output is going to be the edge table. Something like --output-edge-table path/to/file.csv. Ideally, if the -o flag is not set and this one is set, only the csv would be outputted. After merging #8 of course.

tinevez commented 5 months ago

Oh and if this is christmas and I can ask for one more thing: It would be great to have a --version switch that just returns the version number. This is based on my experience with cellpose, where we could not find an easy way to guess whether we were running cellpose 1 or 2.

bentaculum commented 5 months ago

Oh and if this is christmas and I can ask for one more thing: It would be great to have a --version switch that just returns the version number. This is based on my experience with cellpose, where we could not find an easy way to guess whether we were running cellpose 1 or 2.

I would suggest using the built-in python way to do this, sort of a christmas gift without wrapping paper ;). Does that work?

python -c "import trackastra; print(trackastra.__version__)"

Prints 0.1.4 to the command line right now.

bentaculum commented 5 months ago

Yep it's all good but I would like to have an argument added to specify that the output is going to be the edge table. Something like --output-edge-table path/to/file.csv. Ideally, if the -o flag is not set and this one is set, only the csv would be outputted. After merging #8 of course.

Good catch, that was of course still missing. Just added --output-edge-table to #8, in addition to --output-ctc. Both formats are only written if the respective option is set.

bentaculum commented 5 months ago

Hi @tinevez, I just merged #8, curious to know whether this works. I'm curious, how can we install the environment needed for trackastra from within ImageJ?

tinevez commented 5 months ago

I am going to defer to the users to install trackastra via conda, simply following the instructions you put together. For this attempt, I will simply emulate what we have been doing with cellpose;

https://imagej.net/plugins/trackmate/detectors/trackmate-cellpose

So we should be set with this merge, and the ball is now in my hands. I have been in working on a generic way to create TrackMate modules from tools called with the command line, and will now put in practice. https://github.com/trackmate-sc/TrackMate/blob/cli-tools/src/main/java/fiji/plugin/trackmate/util/cli/cellpose/ExampleCellposeCLI.java

I just need some time... (and a release of trackastra with the PR merged).

tinevez commented 5 months ago

By the way how should I spell and write trackastra in the TrackMate integration? I see there is a logo I can use; I will simply blend it with TrackMate. And I will add a link to the preprint.

tinevez commented 5 months ago

Work ongoing in https://github.com/trackmate-sc/TrackMate-Trackastra

bentaculum commented 5 months ago

By the way how should I spell and write trackastra in the TrackMate integration? I see there is a logo I can use; I will simply blend it with TrackMate. And I will add a link to the preprint.

Sounds great! Trackastra with capitalized t is fine.

bentaculum commented 5 months ago

I just need some time... (and a release of trackastra with the PR merged).

v0.2.0 with the new CLI is being released at the moment.

tinevez commented 5 months ago

All good but I seem to be unable to use the CLI. Can you give me an example of a command line that would work?

bentaculum commented 5 months ago

Does trackastra track --help work, on an updated pip install?

tinevez commented 5 months ago

Yes this works!

(trackastra) C:\Users\tinevez>trackastra track --help
usage: trackastra track [-h] -i IMGS -m MASKS [--output-ctc OUTPUT_CTC] [--output-edge-table OUTPUT_EDGE_TABLE] [--model-pretrained MODEL_PRETRAINED]
                        [--model-custom MODEL_CUSTOM] [--mode {greedy_nodiv,greedy,ilp}] [--device {cuda,cpu}]

options:
  -h, --help            show this help message and exit
  -i IMGS, --imgs IMGS  Directory with series of .tif files.
  -m MASKS, --masks MASKS
                        Directory with series of .tif files.
  --output-ctc OUTPUT_CTC
                        If set, write results in CTC format to this directory.
  --output-edge-table OUTPUT_EDGE_TABLE
                        If set, write results as an edge table in CSV format to the given file.
  --model-pretrained MODEL_PRETRAINED
                        Name of pretrained Trackastra model.
  --model-custom MODEL_CUSTOM
                        Local folder with custom model.
  --mode {greedy_nodiv,greedy,ilp}
  --device {cuda,cpu}

BUT! I would need to call the command with python -m prefix. Could I do that? Right now it gives me this:

(trackastra) C:\Users\tinevez>\Users\tinevez\miniforge3\envs\trackastra\python.exe -m trackastra track --help
C:\Users\tinevez\miniforge3\envs\trackastra\python.exe: No module named trackastra.__main__; 'trackastra' is a package and cannot be directly executed
bentaculum commented 5 months ago

Now you can https://github.com/weigertlab/trackastra/commits/main/, from within the repo with python -m trackastra.cli track --help. But wouldn't it be preferable to use directly the CLI entrypoint with trackastra track --help? This would avoid calling the cli module in whatever hidden folder pip installs it.

tinevez commented 5 months ago

Great! I will test this now. This trick allows us calling a Python program from Java without having to call conda first. It works great with cellpose

However Minh-Son in the team has found that on Windows you MUST do it anyway. I need to test if Minh-Son trick also work on MacOS.

tinevez commented 5 months ago

Forgot to say: the v0.2.0 TrackMate integration works now on windows at least I am still fighting with reimporting the edges in TrackMate, but the issues are now on TrackMate side. Getting there!

tinevez commented 5 months ago

Ca marche ! image

https://github.com/weigertlab/trackastra/assets/3583203/0f556d96-b9fd-4c77-8db1-c5a12789c707