Model library

[bw4sz]

@ethanwhite, @henrykironde and I were discussing that as the number of trained models increase, we probably want some more refined way of versioning them and calling them. Right now they are saved alongside the release tags. Each new model would get a "use_release" method, with redundant code, etc.

An example release looks like:

https://github.com/weecology/DeepForest/releases/tag/1.0.0

Which is fine. When use_release is called, the model checks github for the latest release. A couple problems.

1. It doesn't ask the user if they want to use the new release, so any update to the model should be considered breaking. That seems like a big problem and we should atleast ask if we should update.
2. The metadata doesn't live alongside the model in python. There is a link, but not alot of info.

from deepforest import main
m = main.deepforest()
Reading config file: deepforest_config.yml
m.use_release()
Model from DeepForest release https://github.com/weecology/DeepForest/releases/tag/1.0.0 was already downloaded. Loading model from file.
Loading pre-built model: https://github.com/weecology/DeepForest/releases/tag/1.0.0
m.__release_version__
'https://github.com/weecology/DeepForest/releases/tag/1.0.0'

3. Torchvision recently released some API improvements. I think they are largely redundant with pytorch lightning. https://pytorch.org/blog/introducing-torchvision-new-multi-weight-support-api/ I have not read deeply enough to know if they can be used in parallel.

[ayeankit]

Hey @bw4sz , I would like to work on this issue, Can you please guide me?

[henrykironde]

What have you done so far?

[ethanwhite]

It seems like moving to Hugging Face for model distribution is the most common solution here. If we use the pytorch integration:

https://huggingface.co/docs/hub/models-uploading#upload-a-pytorch-model-using-huggingfacehub

then we can use the from_pretrained() method and pass in any of our models (and potentially others models as well; we can wrap from_pretained in use_release as well for backward compatibility):

model.from_pretrained("weecology/deepforest-trees")

I setup an org account https://huggingface.co/weecology and can add @bw4sz & @henrykironde if this sounds like the general way to go.

[bw4sz]

@henrykironde this will be an issue i'll connect to alive/dead model.

[bw4sz]

What we want

• [ ] a single function that loads models from hugging face that replaces https://github.com/weecology/DeepForest/blob/290062c47a98edc329ca252558d5a4d69198b5b1/deepforest/utilities.py#L55 and https://github.com/weecology/DeepForest/blob/290062c47a98edc329ca252558d5a4d69198b5b1/deepforest/utilities.py#L122
• [ ] versioning of the models that takes in a url