New Extension: Deep Learning Model Extension Specification

[fmigneault]

In a previous effort in collaboration with Natural Resources Canada (NRCan) and the Computer Research Institute of Montréal (CRIM), an extension has been developed in order to better describe runtime capabilities and descriptive metadata of Deep Learning Models (DLM).

The repository if hosted here: https://github.com/crim-ca/dlm-extension

We would like to increase visibility of this extension (and improve it based on user feedback as well), since we are planing to continue developing STAC-based services that will make use of such definitions.

Would there be a way to display this proposed extension within https://stac-extensions.github.io/ ?

[HamedAlemo]

Hi @fmigneault, our team worked with both NRCan and CRIM teams to merge the dlm-extension with what we at Radiant were working on more than a year ago (geo-ml-model) and it resulted in the STAC ml-model extension. It would be great to keep improving the ml-model one (which addresses the requirements of deep learning models too).

We currently using ml-model extension for cataloging models on Radiant MLHub, and happy to connect and see how we can improve the spec.

[fmigneault]

Hi @HamedAlemo I remember briefly participating in this effort. I would love to see the extension evolve further.

I believe the major lacking point still remains that inputs/outputs are not properly described in https://github.com/stac-extensions/ml-model. The directories (INPUT_DATA and OUTPUT_DATA) for volume mount are not sufficient to understand what the model operates on (pixel resolution, sensor type, input layer dimensions, produced classes, etc.) to reproduce experiments and results.

Are there plans to integrate those? Some of these concerns are addressed in the proposal of https://github.com/crim-ca/dlm-extension