Global Moth Model - Githubissues

RolnickLab / ami-ml

Software, algorithms and documentation related to the Automated Monitoring of Insects using deep learning and other machine learning methods.

MIT License

1 stars 1 forks source link

Global Moth Model #18

Open adityajain07 opened 3 months ago

adityajain07 commented 3 months ago

A global moth species classifier

joseepoirier commented 2 months ago

@adityajain07 Will you break up this issue into more issues, or have multiple tasks within this issue? I would recommend the former.

adityajain07 commented 2 months ago

Sure! Here it is:

Preliminaries

[x] Get the DwC-A file
[x] Understand, prepare, curate and the GBIF moth checklist
[x] Understand Fagner's codebase for curating the data

Dataset Preparation

[x] Download data
[x] Image verification
[x] Life stage model prediction
[x] Final clean raw dataset
[x] Copy the final raw dataset to object store
[x] Dataset splits
[x] Convert whole dataset to webdataset files with splits
[x] Make a subset of the dataset to test the training pipeline
[x] Copy all dataset files to the object store

Model training

[ ] Understand and modify the existing training code
[ ] Make sure square pad the GBIF images
[ ] Train a standard ResNet-50 model
[ ] Decide on the evaluation metrics, if more needed
[ ] Implement #23
[ ] Implement #32

joseepoirier commented 1 month ago

Following a conversation between AJ and DR, this model is the top priority from 23 Jul.

adityajain07 commented 3 weeks ago

Code profilings to be done:

[ ] Uploading category map to W&B
[ ] Make the training code into a cli, like the ami-dataset module
[ ] Documentation
[ ] Check for preemption; resume training from last saved point
[ ] Test the code (with breakpoints) with a smaller dataset
[ ] Add argument and return types in function definition

joseepoirier commented 3 weeks ago

Thanks for adding the code-profiling tasks above! How are you prioritising them with respect to the data prep/model training above?