search

HumanSignal / label-studio-ml-backend

Configs and boilerplates for Label Studio's Machine Learning backend
Apache License 2.0
533 stars 239 forks source link

ImportError: cannot import name 'LabelStudioMLBase' from 'label_studio.ml' #8

Closed serjrd closed 2 years ago

serjrd commented 3 years ago

Getting this error when trying to follow the guide: https://labelstud.io/guide/ml.html at label-studio-ml init my_ml_backend --script label_studio-ml/examples/simple_text_classifier.py step

p.s.: not sure if it matters, but I didn't run the pip install -r requirements.txt command, since I already have newer versions of the listed packages

rm-data-viz commented 3 years ago

Hi @serjrd,

I had the same issue this morning. Did you download LS docker?

The guide link you shared, I think are missing some steps because, before running "label-studio-ml init", you should change import inside cloned ml backend repository. For example

Go to the "model.py" file in the new ml backend directory that you cloned and change import statement by pointing to this model.py file.

from label_studio.ml import LabelStudioMLBase ------> from label_studio._new_ml_path_.model import LabelStudioMLBase.

Hope this helps.

serjrd commented 3 years ago

@rm-data-viz

Thanks for the hint! Changing path did help to run init without errors.

But I couldn't get it to produce any predicted bound boxes. Tried following the guide at https://labelstud.io/tutorials/object-detector.html but it neither shows any errors, nor produces anything :)

rm-data-viz commented 3 years ago

Hi @serjrd,

I believe you need to install requirements packages, as you mentioned earlier that you haven't. Looks like every ml backend has it's own requirements, so if you're just testing things go ahead and install requirements.

Go to mmdetection directory and

pip install -r requirements.txt

Once requirements are installed, to verify that mmdetection file is error free, you can use:

python3 mmdetection.py

It should run without errors. If it doesn't please install the missing packages. I guess, earlier I had to install mmcv and all.

I think it would be good to init coco detector again after all package are installed.

Then you can start the coco detector by using:

label-studio-ml start _coco_detector_name_

Hope this helps.

serjrd commented 3 years ago

@rm-data-viz

As I said, it doesn't show any errors. I don't know if something got broken at 1.0.0, but it just doesn't work the way I thought it should.

I tried deleting both label-studio and label-studio-ml-backend and installing them from scratch.

So here's what I did:

pip3 uninstall label-studio && pip3 uninstall label-studio-ml && rm -r label-studio-ml-backend 
pip3 install label-studio
git clone https://github.com/heartexlabs/label-studio-ml-backend
cd label-studio-ml-backend
pip install -e .

sed -i s/label_studio.ml/label_studio_ml/ label_studio_ml/examples/mmdetection/mmdetection.py
sed -i s/label_studio.utils.io/label_studio.core.utils.io/ label_studio_ml/examples/mmdetection/mmdetection.py

label-studio-ml init coco-detector --from label_studio_ml/examples/mmdetection/mmdetection.py

label-studio-ml start coco-detector --with config_file=vfnet.py checkpoint_file=vfnet.pth score_threshold=0.5  device=cuda:0
label-studio start

I set up labeling interface like this:

<View>
  <Image name="image" value="$image" zoom="true"/>
  <RectangleLabels name="label" toName="image">
    <Label value="object1(s)" background="#135d75" predicted_values="object1(s)"/>
    <Label value="object2(s)" background="blue" predicted_values="object2(s)/>
  </RectangleLabels>
</View>

I add machine learning backend and it says 'connected'. The Display ML-predicted annotations when labeling toggle is active.

But when I go ahead and import some images and open them for labeling objects - nothing happens. No errors in console. No predicted bounding boxes. Nothing :(

rm-data-viz commented 3 years ago

This looks fine.

If you're able to add the ml backend then it should show in the labeling window in the predicted section.

Does it show like a new project is available in prediction window?

Because, if ml is connected it should be visible in prediction window and when you click that prediction it will show the bounding box predictions.

Also, it would be good to replace with whole config files path rather than just name.

serjrd commented 3 years ago

@rm-data-viz

No, it says 'No predictions' in the Prediction section of the labeling window.

And yeah, I do use full paths when I run label-studio-ml start .... I just cropped it when pasting here so the line isn't too long.

rm-data-viz commented 3 years ago

I think you need to label an example first. Then start prediction training and it should show the predictions.

Just label few examples before running predictions.

serjrd commented 3 years ago

I'm using a pretrained model. Isn't it supposed to give some predictions?

rm-data-viz commented 3 years ago

I guess, you need to train some examples from your data to get predictions from pre trained model.

Predictions without training is not a good practice I think.

serjrd commented 3 years ago

I mean this model has already been trained on my data that I previously labeled by hand

serjrd commented 3 years ago

From the console output it seems that label-studio never really tries to get predictions from the label-studio-ml. I don't see any requests other than:

...
[2021-03-28 18:26:08,601] [INFO] [werkzeug::_log::122] 127.0.0.1 - - [28/Mar/2021 18:26:08] "POST /setup HTTP/1.1" 200 -
[2021-03-28 18:26:08,628] [INFO] [werkzeug::_log::122] 127.0.0.1 - - [28/Mar/2021 18:26:08] "GET /health HTTP/1.1" 200 -
...
rm-data-viz commented 3 years ago

I'll check it soon and update you if I find a solution. I worked on coco detector, but with older version of LS and it's working fine. I still need to check on new version.

There's been a lot of changes in last few days and the docs are not fully updated I think. Let's hope a reply from community soon.

Cheers!

serjrd commented 3 years ago

Ok, looks like label studio actually does try to access ml backend only when you select 'label' option from the projects screen.

But even then I get an error:

[2021-03-28 17:09:38,572] [django.request::log_response::224] [ERROR] Internal Server Error: /data/upload/5__2021.03.23_15-27_W48FJ8N.jpg
Traceback (most recent call last):
  File "/home/serjrd/.local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/serjrd/.local/lib/python3.8/site-packages/django/core/handlers/base.py", line 179, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/serjrd/.local/lib/python3.8/site-packages/django/views/decorators/http.py", line 40, in inner
    return func(request, *args, **kwargs)
  File "/home/serjrd/.local/lib/python3.8/site-packages/label_studio/core/permissions.py", line 190, in wrapper
    return raise_auth_denied('Authentication credentials were not provided', request, redirect_path)
  File "/home/serjrd/.local/lib/python3.8/site-packages/label_studio/core/permissions.py", line 233, in raise_auth_denied
    raise DRFPermissionDenied(msg)
rest_framework.exceptions.PermissionDenied: Authentication credentials were not provided
serjrd commented 3 years ago

Alright, so after a while it seems that I can sum up all the issues I faced on my path to set up ml-backend:

Thank for the help @rm-data-viz

There's clearly lots to be fixed!

rm-data-viz commented 3 years ago

Hi @serjrd,

You can skip the last two steps by just replacing:

self.project_dir=None

with

self.project_dir="/label-studio/data/media/"

in mmdetection.py

serjrd commented 3 years ago

Hi @rm-data-viz

Thanks.

I guess it might be a good solution when you host both label-studio and ml-backend on the same host. But I'm trying to host label-studio at a server and have ml-backend run on a local machine.

niklub commented 3 years ago

Hi @rm-data-viz

Thanks.

I guess it might be a good solution when you host both label-studio and ml-backend on the same host. But I'm trying to host label-studio at a server and have ml-backend run on a local machine.

Hi, @serjrd , In that case you need to go with the full image URLs like http://domain.com/image.jpg

serjrd commented 3 years ago

@niklub

For now I ended up adjusting the image URL in my mmdetection.py file like this: 

task['data']['image'] = 'http://10.0.0.1:8081' + task['data']['image'].split('data')[1]

Is there a better way to do it? Couldn't find any settings regarding this in label-studio.

And is there a way to either make label-studio serve images without authentication or make ml-backend provide credentials with every image request?

silverlining21 commented 3 years ago

I'm using LS 1.0.1 with ml-backend and got the same issue. data source: import through web settings: image

no predicitons in the section image

any updates? thanks!

flavienbwk commented 3 years ago

@niklub

For now I ended up adjusting the image URL in my mmdetection.py file like this:

task['data']['image'] = 'http://10.0.0.1:8081' + task['data']['image'].split('data')[1]

Is there a better way to do it? Couldn't find any settings regarding this in label-studio.

And is there a way to either make label-studio serve images without authentication or make ml-backend provide credentials with every image request?

I encounter the same problem. How to retrieve image data from LabelStudio with ML Backend as it is unauthenticated ?

serjrd commented 3 years ago

@flavienbwk

I start an independent http server like this:

Install a server:

npm i -g http-server

Run the server for your ml-backend:

(cd /home/serjrd/label-studio/media && http-server -p 8081 -a 10.0.0.1)
niklub commented 3 years ago

Hey, @serjrd @silverlining21 @flavienbwk !

I've added some simplifications on using images uploaded to a running Label Studio instance. Please check this PR and let me know if you'll get any problems with running it.

Please note that: