lfai / model_openness_tool

Model Openness Tool
MIT License
5 stars 5 forks source link

RESTful API #39

Open lehors opened 2 weeks ago

lehors commented 2 weeks ago

While the models data can now be accessed through the repo this does not include the classification data. To make it easier for external tools to get to this info we should add a RESTful API to the MOT allowing public, read-only access to the list of models, with all their associated data and classification information, and document it.

gregcube commented 2 weeks ago

I have an endpoint setup -- /api/v1/model/{model} -- that returns a JSON-formatted model. How do you feel about adding the classification information as another top-level element? Something like this for example:

{
    "framework": {
        "name": "Model Openness Framework",
        "version": "1.0",
        "date": "2024-12-15"
    },
    "release": {
        "name": "Baichuan-1-13B",
        "version": "13B",
        "date": "2024-10-03",
        "type": "language",
        "architecture": "transformer decoder",
        "origin": "Baichuan-1",
        "producer": "Baichuan AI",
        "contact": "",
        "components": [
            {
                "name": "Model architecture",
                "description": "Well commented code for the model's architecture",
                "location": "",
                "license_name": "Apache-2.0",
                "license_path": ""
            },
            ...
        ]
    },
    "classification": {
        "class": 3,
        "label": "Class III - Open Model",
        "total_progress": 56.19047619047618,
        "class_progress": {
            "1": 28.57142857142857,
            "2": 40,
            "3": 100
        }
    }
}
lehors commented 2 weeks ago

LGTM but I would drop total_progress since it can easily be calculated from the rest. Where is that used in the MOT btw? Is it used to sort the models on the model list page? Thanks.

gregcube commented 1 week ago

Yup. The Classification column is sorted by total progress. Will drop it from the API, leaving only class progress.