ocean-data-factory-sweden / kso

Notebooks to upload/download marine footage, connect to a citizen science project, train machine learning models and publish marine biological observations.
GNU General Public License v3.0
4 stars 12 forks source link

MLflowError: 404, Issue with finding model when choosing model in notebooks #415

Closed Bergylta closed 1 week ago

Bergylta commented 1 week ago

🐛 Bug

Getting a 404 error when running the code block to choose which model to either upload, or use for evaluation/run

To Reproduce (REQUIRED)

Input:

model = mlp.choose_model()

Output:

Downloading ref-gu-4sp.zip from https://zenodo.org/api/records/10932673/files/ref-gu-4sp.zip/content...
ref-gu-4sp.zip downloaded successfully.
---------------------------------------------------------------------------
MlflowException                           Traceback (most recent call last)
Cell In[7], line 1
----> 1 model = mlp.choose_model()

File /cache/album/cache/kso-user/kso/kso_utils/project.py:1817, in MLProjectProcessor.choose_model(self, custom_project)
   1813 if self.registry == "mlflow":
   1814     # Fetch model artifact list
   1815     from mlflow import MlflowClient
-> 1817     experiment = mlflow.get_experiment_by_name(self.project_name)
   1818     client = MlflowClient()
   1820     if experiment is not None:

File [~/.local/lib/python3.10/site-packages/mlflow/tracking/fluent.py:1273](https://album.cloudina.org/user/kso-user/lab/tree/kso/notebooks/analyse/~/.local/lib/python3.10/site-packages/mlflow/tracking/fluent.py#line=1272), in get_experiment_by_name(name)
   1243 def get_experiment_by_name(name: str) -> Optional[Experiment]:
   1244     """
   1245     Retrieve an experiment by experiment name from the backend store
   1246 
   (...)
   1271         Creation timestamp: 1662004217511
   1272     """
-> 1273     return MlflowClient().get_experiment_by_name(name)

File [~/.local/lib/python3.10/site-packages/mlflow/tracking/client.py:508](https://album.cloudina.org/user/kso-user/lab/tree/kso/notebooks/analyse/~/.local/lib/python3.10/site-packages/mlflow/tracking/client.py#line=507), in MlflowClient.get_experiment_by_name(self, name)
    477 def get_experiment_by_name(self, name: str) -> Optional[Experiment]:
    478     """
    479     Retrieve an experiment by experiment name from the backend store
    480 
   (...)
    506         Lifecycle_stage: active
    507     """
--> 508     return self._tracking_client.get_experiment_by_name(name)

File [~/.local/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py:220](https://album.cloudina.org/user/kso-user/lab/tree/kso/notebooks/analyse/~/.local/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py#line=219), in TrackingServiceClient.get_experiment_by_name(self, name)
    215 def get_experiment_by_name(self, name):
    216     """
    217     :param name: The experiment name.
    218     :return: :py:class:`mlflow.entities.Experiment`
    219     """
--> 220     return self.store.get_experiment_by_name(name)

File [~/.local/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py:306](https://album.cloudina.org/user/kso-user/lab/tree/kso/notebooks/analyse/~/.local/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py#line=305), in RestStore.get_experiment_by_name(self, experiment_name)
    304 try:
    305     req_body = message_to_json(GetExperimentByName(experiment_name=experiment_name))
--> 306     response_proto = self._call_endpoint(GetExperimentByName, req_body)
    307     return Experiment.from_proto(response_proto.experiment)
    308 except MlflowException as e:

File [~/.local/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py:59](https://album.cloudina.org/user/kso-user/lab/tree/kso/notebooks/analyse/~/.local/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py#line=58), in RestStore._call_endpoint(self, api, json_body)
     57 endpoint, method = _METHOD_TO_INFO[api]
     58 response_proto = api.Response()
---> 59 return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)

File [~/.local/lib/python3.10/site-packages/mlflow/utils/rest_utils.py:202](https://album.cloudina.org/user/kso-user/lab/tree/kso/notebooks/analyse/~/.local/lib/python3.10/site-packages/mlflow/utils/rest_utils.py#line=201), in call_endpoint(host_creds, endpoint, method, json_body, response_proto, extra_headers)
    200     call_kwargs["json"] = json_body
    201     response = http_request(**call_kwargs)
--> 202 response = verify_rest_response(response, endpoint)
    203 js_dict = json.loads(response.text)
    204 parse_dict(js_dict=js_dict, message=response_proto)

File [~/.local/lib/python3.10/site-packages/mlflow/utils/rest_utils.py:140](https://album.cloudina.org/user/kso-user/lab/tree/kso/notebooks/analyse/~/.local/lib/python3.10/site-packages/mlflow/utils/rest_utils.py#line=139), in verify_rest_response(response, endpoint)
    135     else:
    136         base_msg = (
    137             f"API request to endpoint {endpoint} "
    138             f"failed with error code {response.status_code} != 200"
    139         )
--> 140         raise MlflowException(
    141             f"{base_msg}. Response body: '{response.text}'",
    142             error_code=get_error_code(response.status_code),
    143         )
    145 # Skip validation for endpoints (e.g. DBFS file-download API) which may return a non-JSON
    146 # response
    147 if endpoint.startswith(_REST_API_PATH_PREFIX) and not _can_parse_as_json_object(response.text):

MlflowException: API request to endpoint /api/2.0/mlflow/experiments/get-by-name failed with error code 404 != 200. Response body: '404 page not found
'

Additional context

Add any other context about the problem here.

jannesgg commented 1 week ago

Status: Vendor is currently down.