HumanSignal / label-studio-ml-backend

Configs and boilerplates for Label Studio's Machine Learning backend
Apache License 2.0
584 stars 261 forks source link

HTTP 403 Forbidden Error During Model Prediction Due to File Access Denial when using with Cloud Storage Sync #660

Open SabolSocare opened 2 weeks ago

SabolSocare commented 2 weeks ago

I am unable to use the Yolo ML Backend model to annotate image data from Cloud Storage.

/label-studio/media/upload, upload_dir=/Users/admin/Library/Application Support/label-studio/media/upload
[2024-11-04 16:16:50,467] [INFO] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::get_local_path::144] Resolving url using hostname [http://localhost:8089/]: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg
[2024-11-04 16:16:50,468] [INFO] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::download_and_cache::205] Download http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg to /Users/admin/Library/Caches/label-studio/24791863__1f742291bf201cfc4c95d5ab7ce3bfc5.jpg
[2024-11-04 16:16:50,468] [DEBUG] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::download_and_cache::218] Authorization token is used for download_and_cache
[2024-11-04 16:16:50,586] [ERROR] [label_studio_ml.exceptions::exception_f::53] Traceback (most recent call last):
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/api.py", line 69, in _predict
    response = model.predict(tasks, context=context, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/model.py", line 120, in predict
    path = model.get_path(task)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/control_models/base.py", line 187, in get_path
    else get_local_path(task_path, task_id=task.get("id"))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 166, in get_local_path
    filepath = download_and_cache(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 221, in download_and_cache
    r.raise_for_status()
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg

Traceback (most recent call last):
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/api.py", line 69, in _predict
    response = model.predict(tasks, context=context, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/model.py", line 120, in predict
    path = model.get_path(task)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/control_models/base.py", line 187, in get_path
    else get_local_path(task_path, task_id=task.get("id"))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 166, in get_local_path
    filepath = download_and_cache(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 221, in download_and_cache
    r.raise_for_status()
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg

[2024-11-04 16:16:50,589] [DEBUG] [label_studio_ml.api::log_response_info::191] Response status: 500 INTERNAL SERVER ERROR
[2024-11-04 16:16:50,589] [DEBUG] [label_studio_ml.api::log_response_info::192] Response headers: Content-Type: application/json
Content-Length: 2389
[2024-11-04 16:16:50,589] [DEBUG] [label_studio_ml.api::log_response_info::193] Response body: b'{\n  "detail": "HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg",\n  "request": {},\n  "result": {\n    "traceback": "Traceback (most recent call last):\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/exceptions.py\\", line 39, in exception_f\\n    return f(*args, **kwargs)\\n           ^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/api.py\\", line 69, in _predict\\n    response = model.predict(tasks, context=context, **params)\\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/yolo/model.py\\", line 120, in predict\\n    path = model.get_path(task)\\n           ^^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/yolo/control_models/base.py\\", line 187, in get_path\\n    else get_local_path(task_path, task_id=task.get(\\"id\\"))\\n         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py\\", line 166, in get_local_path\\n    filepath = download_and_cache(\\n               ^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py\\", line 221, in download_and_cache\\n    r.raise_for_status()\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/requests/models.py\\", line 1024, in raise_for_status\\n    raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg\\n"\n  },\n  "status": 500\n}\n'
[2024-11-04 16:16:50,589] [INFO] [werkzeug::_log::97] 127.0.0.1 - - [04/Nov/2024 16:16:50] "POST /predict HTTP/1.1" 500 -
makseq commented 2 weeks ago

How does your docker-compose.yml look? Did you follow step #1 from https://github.com/HumanSignal/label-studio-ml-backend/blob/master/label_studio_ml/examples/yolo/README.md#quick-start ?

SabolSocare commented 2 weeks ago

Here's my Docker Compose configuration for the YOLO service:

It works when the image is uploaded from the browser, but it does not work with images synced from cloud storage.

version: "3.8"

services:
  yolo:
    container_name: yolo
    image: humansignal/yolo:v0
    build:
      context: .
      args:
        TEST_ENV: ${TEST_ENV}
    environment:
      # Optional basic auth for the model server
      - BASIC_AUTH_USER=
      - BASIC_AUTH_PASS=
      # Logging and server configuration
      - LOG_LEVEL=DEBUG
      - WORKERS=1
      - THREADS=8
      - MODEL_DIR=/data/models
      - PYTHONPATH=/app

      # Label Studio integration
      - LABEL_STUDIO_URL=http://127.0.0.1:8089
      - LABEL_STUDIO_API_KEY=9e06b3eca8f7edfc0f9b9XXXXXXXXXX

      # YOLO-specific configuration
      - ALLOW_CUSTOM_MODEL_PATH=true
      - DEBUG_PLOT=false
      - MODEL_SCORE_THRESHOLD=0.5
      - MODEL_ROOT=/app/models
    ports:
      - "9090:9090"
    volumes:
      - "./data/server:/data"
      - "./models:/app/models"
      - "./cache_dir:/app/cache_dir"
makseq commented 1 week ago

LABEL_STUDIO_URL=http://127.0.0.1:8089

This line is the same as the use of localhost. When you are inside of the docker, it won't work. In the example docker compose configuration it was a comment about label studio url, please read it carefully. https://github.com/HumanSignal/label-studio-ml-backend/blob/master/label_studio_ml/examples/yolo/docker-compose.yml#L26

SabolSocare commented 1 week ago

Thank you for the guidance. The URL http://127.0.0.1:8089 was just an example; I’m actually using my own domain, which works (auto-annotation) as expected with data under /data/upload. However, I'm encountering issues specifically with data paths prefixed by /data/local-files/? (Cloud Storage). It seems that these paths aren't accessible in the same way