Open plannaAlain opened 1 year ago
Do you ever get this working? I'm having the same issue, but with Azure Blob Storage.
I am having the same problem with Google Cloud Storage. Any solution so far?
Using the same format of checking "s3://" for AWS storage (eg. easyocr_labeling.py), I converted the code to GCP google cloud storage:
from label_studio_ml.utils import DATA_UNDEFINED_NAME
from google.cloud import storage
from datetime import timedelta
import logging
logger = logging.getLogger(__name__)
class YourModel(LabelStudioMLBase):
def _get_image_url(self, task, value='image'):
image_url = task['data'].get(self.value) or task['data'].get(DATA_UNDEFINED_NAME)
if image_url.startswith('gs://'):
# Generate signed URL for GCS
bucket_name, object_name = image_url.replace('gs://', '').split('/', 1)
storage_client = storage.Client.from_service_account_json('/secrets/service_account_key.json')
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(object_name)
try:
image_url = blob.generate_signed_url(
version="v4",
expiration=timedelta(hours=1), # Adjust expiration time as needed
method="GET",
)
except Exception as exc:
logger.warning(f'Can\'t generate signed URL for {image_url}. Reason: {exc}')
return image_url
def predict(.self, tasks, ......):
image_path = self._get_image_url(tasks[0])
I am running LS 1.8.1dev and have created a ML backend that works well with local images BUT when trying to analyse a cloud-based image, the following error is returned:
Traceback (most recent call last): File "/Users/me/dev/miniconda3/envs/macai5/lib/python3.9/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f return f(*args, **kwargs) File "/Users/me/dev/miniconda3/envs/macai5/lib/python3.9/site-packages/label_studio_ml/api.py", line 32, in _predict predictions, model = _manager.predict(tasks, project, label_config, force_reload, try_fetch, **params) File "/Users/me/dev/miniconda3/envs/macai5/lib/python3.9/site-packages/label_studio_ml/model.py", line 582, in predict predictions = m.model.predict(tasks, **kwargs) File "/Users/me/dev/project/planna/labs/label-studio-ml-od-external-id/od-model-external-id/external_id/external_id_detection.py", line 258, in predict prediction = self.predict_one_task(task) File "/Users/me/dev/project/planna/labs/label-studio-ml-od-external-id/od-model-external-id/external_id/external_id_detection.py", line 266, in predict_one_task image_path = self.get_local_path(image_url) File "/Users/me/dev/miniconda3/envs/macai5/lib/python3.9/site-packages/label_studio_ml/model.py", line 322, in get_local_path return get_local_path(url, project_dir=project_dir, hostname=self.hostname, access_token=self.access_token) File "/Users/me/dev/miniconda3/envs/macai5/lib/python3.9/site-packages/label_studio_tools/core/utils/io.py", line 91, in get_local_path r = requests.get(url, stream=True, headers=headers) File "/Users/me/dev/miniconda3/envs/macai5/lib/python3.9/site-packages/requests/api.py", line 73, in get return request("get", url, params=params, **kwargs) File "/Users/me/dev/miniconda3/envs/macai5/lib/python3.9/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) File "/Users/me/dev/miniconda3/envs/macai5/lib/python3.9/site-packages/requests/sessions.py", line 587, in request resp = self.send(prep, **send_kwargs) File "/Users/me/dev/miniconda3/envs/macai5/lib/python3.9/site-packages/requests/sessions.py", line 695, in send adapter = self.get_adapter(url=request.url) File "/Users/me/dev/miniconda3/envs/macai5/lib/python3.9/site-packages/requests/sessions.py", line 792, in get_adapter raise InvalidSchema(f"No connection adapters were found for {url!r}") requests.exceptions.InvalidSchema: No connection adapters were found for 'gs://bucket-name-xxx/IMG_2447.jpeg'
Should LS pass a local path for the cached image? Not sure if this is a LS or ML issue, probably an LS issue and could be related to https://github.com/heartexlabs/label-studio/issues/3790