search

allegroai / clearml-server

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Other
364 stars 132 forks source link

API issues with GET request when installing in GCP #193

Open alexpusch opened 1 year ago

alexpusch commented 1 year ago

I've installed clearml server using the official Helm chart under the k8s service (GKE) in Google cloud platform

The app itself shows up fine, but when using some apis, for example clearml-data create --name=test --project=project I receive the following error: 

Error: Bad Request
Your client has issued a malformed or illegal request.

After some digging it seems that clearml SDK and CLI send GET requests with http body data - which is bad practice and rejected by the GCP load balancer - link link

example to an http request with GET body:

GET 'http://api.clearml.../v2.23/projects.get_all' \
'{"name": "^project/\\.datasets/test$", "only_fields": ["id"], "shallow_search": false, "search_hidden": true}'

This will make installing clearml server under GCP pretty hard

alexpusch commented 1 year ago

ha, just found out about CLEARML_API_DEFAULT_REQ_METHOD. seems that settings this to post is a workaround

jkhenning commented 1 year ago

Hi @alexpusch, sorry for replying late, I was just about to say you can use the configuration setting api.http.default_method: PUT and/or using the CLEARML_API_DEFAULT_REQ_METHOD=PUT environment variable :)

PUT is better, I think, but POST should work as well 🙂

alexpusch commented 1 year ago

@jkhenning Thanks for the reply. This works, but I think it would be nice if all http requests with body will be non GET by default

jkhenning commented 1 year ago

Well, it's a loaded discussion 🙂 - conceptually, most of these requests are not to send stuff to the server (i.e. PUT), but to get resources, with additional parameters (that can't be put into the query string)