allegroai / clearml-server

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Other
381 stars 131 forks source link

missing user_id prevents running pipeline tasks from GUI #169

Open doronser opened 1 year ago

doronser commented 1 year ago

Since version 1.9.0 release, I can't use the webapp to generate new pipeline runs. I get the following error: _Error 0 : enqueue_task() missing 1 required positional argument: 'userid'

I did some digging, and it looks like ChangeStatusRequest added a user_id argument that has no default, but I'm not sure what's the correct fix here.

Can you advise on how to fix this?

ainoam commented 1 year ago

Thanks for reporting @doronser! Looking into it...

doronser commented 1 year ago

I added a default value of None to the function apiserver.bll.task.task_operations enqueue_task(), but now I get a new error that is rather ambigous: Error 12 : Validation error (error for field 'args': ("error for field 'name'.", validationerror('field is required!',)).)

ainoam commented 1 year ago

@doronser Working on getting a patch release out. If you need to patch your code before-hand, the fix should look like:

--- a/apiserver/services/pipelines.py
+++ b/apiserver/services/pipelines.py
@@ -60,6 +60,7 @@ def start_pipeline(call: APICall, company_id: str, request: StartPipelineRequest
     queued, res = enqueue_task(
         task_id=task.id,
         company_id=company_id,
+        user_id=call.identity.user,
         queue_id=request.queue,
         status_message="Starting pipeline",
         status_reason="",
-- 
doronser commented 1 year ago

how can I apply this change to an already running clearml-server? If it matters, I'm running self-hosted on a Lunix machine. I tried editing the files from inside the docker but that didn't change much: docker exec -it clearml-apiserver /bin/bash vi apiserver/services/pipelines.py

any other ideas?

jkhenning commented 1 year ago

Hi @doronser, Unfortunately unless you build your own server docker image, this is a bit problematic, unless you want to try saving the patched file in some host-machine folder and mounting it directly into the apiserver docker image - the change to the docker-compose file should be something like:

services:
  ...
  apiserver:
    ...
    volumes:
      ...
      /path/to/local/pipelines.py:/opt/clearml/apiserver/services/pipelines.py
markbarna commented 1 year ago

Curious if there is an ETA on a patch release to fix this bug? Thanks!

jkhenning commented 1 year ago

@markbarna A new server version v1.9.1 was just released solving this 🙂

markbarna commented 1 year ago

Thank you @jkhenning ! I will update my server deployment.