search

BharatSahAIyak / autotune

A comprehensive toolkit for seamless data generation and fine-tuning of NLP models, all conveniently packed into a single block.
MIT License
9 stars 5 forks source link

Modify the train API to support versioning #142

Open Gautam-Rajeev opened 2 months ago

Gautam-Rajeev commented 2 months ago

Constructing the minio save path :

as a part of train API, we receive : 

"BaseModel": Base Model Branch on HF (from model registry)
    "epochs":  (from model registry)
      "task_type":  (from model registry)
    "dataset":  (from dataset registry)
        "versioning": {
         "owner": botid   
        "environment": bot environment 
        “ModelName ': (from model registry) 
        "tag":   optional parameter to start a new tag 
    },

HF repo name :

repo : BaseModel_owner_ModelName branch : env tag : last tag + 0.1 unless provided by the user

minio naming structure :

'model'/ BaseModel/ owner_ModelName/ tag/

Next steps :