Empty context in kwargs (SAM)

[RobinFrcd]

Hi, I'm trying to run the SAM backend provided in https://github.com/heartexlabs/label-studio-ml-backend/tree/master/label_studio_ml/examples/segment_anything_model

I'm using the Docker version which builds fine, but when I try to run a prediction it fails with:

the kwargs are {'login': None, 'password': None, 'context': None}
the tasks are [{'id': 451102, 'data': {'image': 's3://....jpg', 'ref_id': 3602}, 'meta': {}, 'created_at': '2023-06-20T16:29:12.201416Z', 'updated_at': '2023-06-20T16:29:12.201442Z', 'is_labeled': False, 'overlap': 1, 'inner_id': 2, 'total_annotations': 0, 'cancelled_annotations': 0, 'total_predictions': 0, 'comment_count': 0, 'unresolved_comment_count': 0, 'last_comment_updated_at': None, 'project': 218, 'updated_by': None, 'file_upload': None, 'comment_authors': [], 'annotations': [], 'predictions': []}]
[2023-06-27 14:13:42,314] [ERROR] [label_studio_ml.exceptions::exception_f::53] Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f
return f(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/label_studio_ml/api.py", line 51, in _predict
predictions, model = _manager.predict(tasks, project, label_config, force_reload, try_fetch, **params)
File "/usr/local/lib/python3.8/site-packages/label_studio_ml/model.py", line 617, in predict
predictions = cls._current_model.model.predict(tasks, **kwargs)
File "/app/segment_anything_model.py", line 70, in predict
height = kwargs['context']['result'][0]['original_height']
TypeError: 'NoneType' object is not subscriptable
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f
return f(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/label_studio_ml/api.py", line 51, in _predict
predictions, model = _manager.predict(tasks, project, label_config, force_reload, try_fetch, **params)
File "/usr/local/lib/python3.8/site-packages/label_studio_ml/model.py", line 617, in predict
predictions = cls._current_model.model.predict(tasks, **kwargs)
File "/app/segment_anything_model.py", line 70, in predict
height = kwargs['context']['result'][0]['original_height']
TypeError: 'NoneType' object is not subscriptable

Seems like kwargs are empty, how may I fix that ?

Thanks, Robin

[jackmead515]

Bump same issue. This seems to be a persistent issue with label-studio 1.8.0

[sriramsk1999]

I'm facing the same issue with 1.7.3 as well, the context object in kwargs is None

[feicccccccc]

Encounter the same problem for label studio versions 1.6.0, 1.7.3, 1.8.0 It seems it's sever side (label studio) problem: Request JSON:

{
 'tasks': [{'id': 128, 'data': {'image': '/data/local-files/?d=Documents/label_studio/dataset/coco128/000000000650.jpg'}, 'meta': {}, 'created_at': '2023-07-08T11:15:13.167604Z', 'updated_at': '2023-07-08T11:15:13.167616Z', 'is_labeled': False, 'overlap': 1, 'inner_id': 128, 'total_annotations': 0, 'cancelled_annotations': 0, 'total_predictions': 0, 'comment_count': 0, 'unresolved_comment_count': 0, 'last_comment_updated_at': None, 'project': 1, 'updated_by': None, 'file_upload': None, 'comment_authors': [], 'annotations': [], 'predictions': []}],
 'model_version': 'INITIAL',
 'project': '1.1688814855',
 'label_config': '<View>\n  <Image name="image" value="$image" zoom="true"/>\n  <BrushLabels name="tag" toName="image">\n  \t<Label value="Banana" background="#FF0000"/>\n  \t<Label value="Orange" background="#0d14d3"/>\n  </BrushLabels>\n  <KeyPointLabels name="tag2" toName="image">\n    <Label value="Banana" smart="true" background="#000000" showInline="true"/>\n    <Label value="Orange" smart="true" background="#000000" showInline="true"/>\n    <Label value="Orange Eraser" smart="true" background="#000000" showInline="true"/>\n  </KeyPointLabels>\n</View>', 
 'params': {'login': None, 'password': None, 'context': None}
}

[DaMuBo]

had the same issue.

Also tested it without docker. Getting the same error:

    height = kwargs['context']['result'][0]['original_height']
             ~~~~~~~~~~~~~~~~~^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

used label studio 1.7.1

context object printed from the predict: {'login': None, 'password': None, 'context': None}

[erinmikailstaples]

Hey all (@DaMuBo, @feicccccccc @sriramsk1999, @jackmead515, @RobinFrcd) 👋🏻

We discussed this issue during our issue triage today and are flagging this for further inspection and review.

Thank you all for your patience on this — I'll keep y'all posted on what we find.

[ see also https://github.com/heartexlabs/label-studio/issues/4493 ]

[DimIsaev]

I have the same problem!!

There seems to be a small problem why the context is not being passed ?

[hogepodge]

@shondle as the developer of the SAM feature, have you seen this issue in your own testing, and do you have a workaround?

[shondle]

I only had to deal with this before when I had a setting incorrect or was using the wrong tools to gather predictions.

I am currently working on a video tutorial with a new commit to show how to use SAM correctly. I have not have had to deal with this issue throughout my testing -> none in using multiple linux machines and have even gotten SAM to work on my windows laptop recently to test.

I can come back to this issue and drop the video, it should be completed very soon. Because I have been able to replicate with success in many environments, I believe the issue is I wasn't specific enough in my instructions in the README and a visual guide may be more helpful.

I think it would also be helpful if Human Signal could update the documentation to provide a bit more visual help to show how to use ML Backends with smart prediction tools. As far as I can tell, this is just a miscommunication in how people setup and use backends instead of being an issue in the code of the SAM backend itself.

[shondle]

Hi everyone, the video tutorial is included in the pull request mentioned above. Again, I have been able to replicate using SAM without issue in varying environments, so please follow the video after the installation instructions to see if this works for you.

If it doesn't, please send a video in this issue showing your settings in the Machine Learning part of Label Studio, how you are getting a prediction with the tool in Label Studio, and the output in your ML backend terminal.

I realize this video won't address all issues, but I hope it helps a good amount of you.

[DaMuBo]

@shondle thanks a lot. I tried it with your fork and with the mobileSAM it works like a charm now.

Even without my GPU attached. So great work thanks 👍🏻

[botcs]

Hi all!

Thanks @shondle for the great adaptation and @erinmikailstaples for maintaining the framework! I think there's gonna be a lot more people discovering label-studio through SAM's adaptation, so I believe fixing this thread bug is really important.

At the moment I was trying to change a few things in the framework to get the ML backend started, but I believe there might be a version mismatch all over the examples?

The one that I bumped into first was the init_app signature in the api.py vs the ones in used in the examples/*/_wsgi.py files.

Checking now #304 to see if that fixes the issue :) I'm sure I might be just missing something very basic stuff, let me know if I looked over something.

Thnks, Csabi

[sriramsk1999]

I was also able to get it working without problems. Was most likely facing some environment issues. I reinstalled everything in a fresh env and it worked like a charm. Thanks for the nice work on this!

[Jerry2990]

i also has this problem. Is there any new solution?

[Jerry2990]

@shondle thanks a lot. I tried it with your fork and with the mobileSAM it works like a charm now.

Even without my GPU attached. So great work thanks 👍🏻

did you run label-studio without EMPTY CONTETX? @DaMuBo

[DimIsaev]

no solution