search

huggingface / blog

Public repo for HF blog posts
https://hf.co/blog
2.3k stars 711 forks source link

Host the Huggingface Transformer models in Azure without uploading to Huggingface #322

Open dreji18 opened 2 years ago

dreji18 commented 2 years ago

Recently Huggingface endpoints have been released https://huggingface.co/blog/hugging-face-endpoints-on-azure It would be great if we can host these Huggingface fine-tuned models directly hosted in Azure.

osanseviero commented 2 years ago

cc @philschmid

philschmid commented 2 years ago

Hey @dreji18,

Thank you for this request! I hope we can provide something in the near future. Maybe you can help us better understand how this integration should look.

Where are you currently storing the models inside Azure? Are you using the structure of an HF repository? Is there a particular reason why you store them in Azure rather on the Hub either as private/public?

dreji18 commented 2 years ago

Hi @philschmid,

Currently, this newly created feature is a 2 step process if we have to

  1. We fine-tune the model using any Huggingface models with custom data/architecture and push it to Huggingface hub
  2. This model we deploy with Huggingface endpoint in Azure providing the link to Huggingface hub

Here the problem is every time there is a process of pushing the model to huggingface hub and calling it back to Azure.

Proposed Solution: After fine-tuning the huggingface models with our data/architecture if there is a way directly to push these models to Azure using a huggingface endpoint gateway and not following the typical azure ml deployment pipeline (which is complex and time consuming) it would have been great.

philschmid commented 2 years ago

Proposed Solution: After fine-tuning the huggingface models with our data/architecture if there is a way directly to push these models to Azure using a huggingface endpoint gateway and not following the typical azure ml deployment pipeline (which is complex and time consuming) it would have been great.

Were are you saving those artifacts?

You can also "update" the Azure endpoint programmatically. Meaning you would use the hub as a model backend/repository with version control etc. and can programmatically deploy to azure then.

dreji18 commented 2 years ago

@philschmid so currently we save the model file and tokenizer and load the entire directory for prediction in the runtime. Deploying this model as an azure endpoint requires prerequisites like preparing a scoring file, environment details etc.

dreji18 commented 2 years ago

Sorry, I didn't understand the "update" option with the hub. If we directly deploy the model as an azure endpoint there is no role for huggingface hub endpoint.