allegroai / clearml-serving

ClearML - Model-Serving Orchestration and Repository Solution
https://clear.ml
Apache License 2.0
138 stars 40 forks source link

Issue with pytorch preprocess code #38

Open okyspace opened 1 year ago

okyspace commented 1 year ago

Hi, I have encountered error stating that model was expecting input [1 28 28] but given [1 784] when trying out the pytorch example. I think it is due to the flatten() of the array before return by the preprocess method.

Can I also ask

  1. How do we update the preprocess code to the same created endpoint using command line / codes?
  2. When we create the endpoint with the preprocess code, the code preprocess.py is stored in the clearml server. Does the inference container periodically pull from clearml server or the clearml server will push to the inference container upon any update? May I know where to access this codes that manage this behavior to better understand what's going behind this?

Thanks.

thepycoder commented 1 year ago

Hi! This issue should be fixed as of: https://github.com/allegroai/clearml-serving/pull/50/files

  1. You can just redeploy the endpoint, pointing to a different preprocessing script
  2. Yes, the inference container should be able to update its preprocessing script dynamically (which means that when doing what is described in 1. you would not have downtime) If I'm not mistaken this is the code governing that. It will get all endpoints that are synced every interval (non of them are synced at first) and redeploy them if needed. Triton will do the rest :)