search

HumanSignal / label-studio-ml-backend

Configs and boilerplates for Label Studio's Machine Learning backend
Apache License 2.0
587 stars 261 forks source link

Connect Model Validation error #636

Closed ws-zhongm closed 2 months ago

ws-zhongm commented 2 months ago

I met the problem when setting my own ML model, can you help me ?

Validation error Successfully connected to http://localhost:9090 but it doesn't look like a valid ML backend. Reason: HTTPConnectionPool(host='localhost', port=9090): Read timed out. (read timeout=3.0). Check the ML backend server console logs to check the status.There might be something wrong with your model or it might be incompatible with the current labeling configuration.

below is my predict part of model.py

    def predict(self, tasks: List[Dict], context: Optional[Dict] = None, **kwargs) -> ModelResponse:
        """ Write your inference logic here
            :param tasks: [Label Studio tasks in JSON format](https://labelstud.io/guide/task_format.html)
            :param context: [Label Studio context in JSON format](https://labelstud.io/guide/ml_create#Implement-prediction-logic)
            :return model_response
                ModelResponse(predictions=predictions) with
                predictions: [Predictions array in JSON format](https://labelstud.io/guide/export.html#Label-Studio-JSON-format-of-annotated-tasks)
        """
        predictions = []
        for task in tasks:
            text = task['data']['text']
            token = self.tokenizer(text, add_special_tokens=True, padding='max_length', truncation=True, max_length=512)
            input_ids = token['input_ids']
            attention_mask = token['attention_mask']
            token_type_ids = token['token_type_ids']

            input_ids = torch.tensor([input_ids], dtype=torch.long)
            attention_mask = torch.tensor([attention_mask], dtype=torch.long)
            token_type_ids = torch.tensor([token_type_ids], dtype=torch.long)

            with torch.no_grad():
                outputs = self.model(
                    input_ids,
                    attention_mask,
                    token_type_ids,
                )
            probabilities = F.softmax(outputs, dim=1)
            pred_label_index = torch.argmax(probabilities, dim=1).item()
            confidence = probabilities[0][pred_label_index].item()
            pred_label = self.labels[pred_label_index]

            prediction = {
                "model_version": self.get("model_version"),
                "score": confidence,
                "result": [{
                    "from_name": "sentiment",
                    "to_name": "text",
                    "type": "choices",
                    "value": {
                        "choices": [pred_label]
                    }
                }]
            }
            predictions.append(prediction)

        return ModelResponse(predictions=predictions)

below is label-studio-ml log

(label_studio) [root@localhost label-studio-ml-backend]# LOG_LEVEL=DEBUG label-studio-ml start my_ml_backend --debug There are 2 GPU(s) available. We will use the GPU: NVIDIA GeForce GTX 1080 Ti

[2024-09-25 15:30:54,547] [DEBUG] [label_studio_ml.api::log_request_info::186] Request body: b'' [2024-09-25 15:30:54,548] [DEBUG] [label_studio_ml.api::log_response_info::191] Response status: 200 OK [2024-09-25 15:30:54,548] [DEBUG] [label_studio_ml.api::log_response_info::192] Response headers: Content-Type: application/json Content-Length: 50

[2024-09-25 15:30:54,548] [DEBUG] [label_studio_ml.api::log_response_info::193] Response body: b'{\n "model_class": "NewModel",\n "status": "UP"\n}\n' [2024-09-25 15:30:54,549] [INFO] [werkzeug::_log::97] 127.0.0.1 - - [25/Sep/2024 15:30:54] "GET /health HTTP/1.1" 200 - [2024-09-25 15:30:54,554] [DEBUG] [label_studio_ml.api::log_request_info::185] Request headers: Host: localhost:9090 Sentry-Trace: 1b921b038c8542e8a5be3350b16a43f1-917d1085e6837a6e-0 Baggage: sentry-trace_id=1b921b038c8542e8a5be3350b16a43f1,sentry-environment=opensource,sentry-release=label-studio%401.13.1,sentry-public_key=68b045ab408a4d32a910d339be8591a4,sentry-transaction=/api/ml/,sentry-sample_rate=0.02,sentry-sampled=false User-Agent: heartex/1.13.1+0.gd9b816a.dirty Accept-Encoding: gzip, deflate Accept: / Connection: keep-alive Content-Length: 1003 Content-Type: application/json

[2024-09-25 15:30:54,555] [DEBUG] [label_studio_ml.api::log_request_info::186] Request body: b'{"project": "3.1727088695", "schema": "\n <Text name=\"text\" value=\"$text\"/>\n <View style=\"box-shadow: 2px 2px 5px #999; padding: 20px; margin-top: 2em; border-radius: 5px;\">\n <Header value=\"Choose text sentiment\"/>\n <Choices name=\"sentiment\" toName=\"text\" choice=\"single\" showInLine=\"true\">\n \n \n \n <Choice value=\"\u5176\u5b83\u6587\u672c\"/><Choice value=\"\u6deb\u79fd\u8272\u60c5\"/><Choice value=\"\u6b3a\u8bc8\u6587\u672c\"/><Choice value=\"\u66b4\u6050\u8fdd\u6cd5\"/><Choice value=\"\u653f\u6cbb\u654f\u611f\"/><Choice value=\"\u4fae\u8fb1\u8c29\u9a82\"/><Choice value=\"\u8d4c\u535a\u5f69\u7968\"/><Choice value=\"\u5e7f\u544a\u63a8\u9500\"/>\n \n", "hostname": "http://localhost:8080", "access_token": "f9fc6984680f507112daeed288bedbb6081911c6", "extra_params": ""}' [2024-09-25 15:30:59,948] [DEBUG] [label_studio_ml.api::log_response_info::191] Response status: 200 OK [2024-09-25 15:30:59,949] [DEBUG] [label_studio_ml.api::log_response_info::192] Response headers: Content-Type: application/json Content-Length: 41

[2024-09-25 15:30:59,949] [DEBUG] [label_studio_ml.api::log_response_info::193] Response body: b'{\n "model_version": "NewModel-v0.0.1"\n}\n' [2024-09-25 15:30:59,949] [INFO] [werkzeug::_log::97] 127.0.0.1 - - [25/Sep/2024 15:30:59] "POST /setup HTTP/1.1" 200 -

ws-zhongm commented 2 months ago

Closed, just set the ML_TIMEOUT_SETUP and restart label studio.