search

deepjavalibrary / djl-serving

A universal scalable machine learning model deployment solution
Apache License 2.0
182 stars 59 forks source link

fix bug with duplicate models when HF_MODEL_ID points to model store #2054

Closed siddvenk closed 3 weeks ago

siddvenk commented 3 weeks ago

Description

If customer has mounted model artifacts to /opt/ml/model, and also specified HF_MODEL_ID=/opt/ml/model, then the container fails to startup with

ERROR ModelServer Invalid configuration
ai.djl.serving.http.BadRequestException: Workflow model is already registered.
    at ai.djl.serving.models.ModelManager.registerWorkflow(ModelManager.java:92) ~[serving-0.28.0.jar:?]
    at ai.djl.serving.ModelServer.initModelStore(ModelServer.java:444) ~[serving-0.28.0.jar:?]
    at ai.djl.serving.ModelServer.start(ModelServer.java:208) ~[serving-0.28.0.jar:?]
    at ai.djl.serving.ModelServer.startAndWait(ModelServer.java:174) ~[serving-0.28.0.jar:?]
    at ai.djl.serving.ModelServer.main(ModelServer.java:143) [serving-0.28.0.jar:?]

Issue is that when adding to the list or urls, we returned the local path which is different than the url so it looks like 2 different models, but at workflow register level they are the same