inaturalist / iNaturalistMLWork

0 stars 0 forks source link

Implement a representation of model releases in the Rails codebase #36

Open pleary opened 4 months ago

pleary commented 4 months ago

This isn't clearly defined yet. There would be value in having model releases be represented in the iNaturalist database, and since DB tables are created via rails models and migrations, this will likely be implemented in that codebase. If the DB had knowledge of all our model releases, when the started, when they finished, their version name, and paths to their release directories that contain their taxonomy.csv files, then we could use that information to automate some post-release processes.

We're currently generating a blog post with each release that includes a static image of a chart showing each release and how long training took. That chart could be generated dynamically, and be interactive instead of static, from a database table that contained that information. We also include in that blog post links to observations of taxa included in a release that were not included in previous releases. This information could also be derived programmatically. We plan to eventually include model evaluation metrics in model release posts. That data could be stored in a related database table, or somewhere in the model release directory that is fetched to dynamically render metrics charts.

loarie commented 4 months ago

With the Observation Accuracy Experiments, we're settling into a monthly blog post that announces the new Experiment page. We could do an analogous approach for new models