Groundingdino error

[KTXKIKI]

) Listening {"listenAddress": ":8082"} 2023-10-12 14:23:52 23.10.12 06:23:52.658 processor.http (D) Creating worker pool {"num": 1} 2023-10-12 14:23:52 23.10.12 06:23:52.658 sor.http.w0.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-ckjp1u200s0kejd449fg.sock"} 2023-10-12 14:23:52 23.10.12 06:23:52.659 sor.http.w0.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-ckjp1u200s0kejd449g0.sock"} 2023-10-12 14:23:52 23.10.12 06:23:52.659 sor.http.w0.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"} 2023-10-12 14:23:52 23.10.12 06:23:52.665 sor.http.w0.python.logger (D) Using Python handler {"handler": "main:handler"} 2023-10-12 14:23:52 23.10.12 06:23:52.666 sor.http.w0.python.logger (D) Using Python executable {"path": "/usr/bin/python3"} 2023-10-12 14:23:52 23.10.12 06:23:52.666 sor.http.w0.python.logger (D) Setting PYTHONPATH {"value": "PYTHONPATH=/opt/nuclio"} 2023-10-12 14:23:52 23.10.12 06:23:52.666 sor.http.w0.python.logger (D) Running wrapper {"command": "/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --event-socket-path /tmp/nuclio-rpc-ckjp1u200s0kejd449fg.sock --control-socket-path /tmp/nuclio-rpc-ckjp1u200s0kejd449g0.sock --platform-kind local --namespace nuclio --worker-id 0 --trigger-kind http --trigger-name myHttpTrigger --decode-event-strings"} 2023-10-12 14:23:38 /usr/local/lib/python3.10/dist-packages/groundingdino/models/GroundingDINO/ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only! 2023-10-12 14:23:38 warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!") 2023-10-12 14:23:38 Traceback (most recent call last): 2023-10-12 14:23:38 File "/opt/nuclio/model_handler.py", line 23, in load_network 2023-10-12 14:23:38 self.model = load_model(model_config_path=model_config_path, model_checkpoint_path=model_checkpoint_path, 2023-10-12 14:23:38 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/inference.py", line 30, in load_model 2023-10-12 14:23:38 args = SLConfig.fromfile(model_config_path) 2023-10-12 14:23:38 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/slconfig.py", line 185, in fromfile 2023-10-12 14:23:38 cfg_dict, cfg_text = SLConfig._file2dict(filename) 2023-10-12 14:23:38 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/slconfig.py", line 90, in _file2dict 2023-10-12 14:23:38 mod = import_module(temp_module_name) 2023-10-12 14:23:38 File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module 2023-10-12 14:23:38 return _bootstrap._gcd_import(name[level:], package, level) 2023-10-12 14:23:38 File "", line 1050, in _gcd_import 2023-10-12 14:23:38 File "", line 1027, in _find_and_load 2023-10-12 14:23:38 File "", line 1006, in _find_and_load_unlocked 2023-10-12 14:23:38 File "", line 688, in _load_unlocked 2023-10-12 14:23:38 File "", line 883, in exec_module 2023-10-12 14:23:38 File "", line 241, in _call_with_frames_removed 2023-10-12 14:23:38 File "/tmp/tmp8eh_wv58/tmpjrzsznt8.py", line 1, in 2023-10-12 14:23:38 NameError: name 'false' is not defined. Did you mean: 'False'? 2023-10-12 14:23:38 2023-10-12 14:23:38 During handling of the above exception, another exception occurred: 2023-10-12 14:23:38 2023-10-12 14:23:38 Traceback (most recent call last): 2023-10-12 14:23:38 File "/opt/nuclio/_nuclio_wrapper.py", line 480, in 2023-10-12 14:23:38 run_wrapper() 2023-10-12 14:23:38 File "/opt/nuclio/_nuclio_wrapper.py", line 468, in run_wrapper 2023-10-12 14:23:38 loop.run_until_complete(wrapper_instance.initialize()) 2023-10-12 14:23:38 File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete 2023-10-12 14:23:38 return future.result() 2023-10-12 14:23:38 File "/opt/nuclio/_nuclio_wrapper.py", line 165, in initialize 2023-10-12 14:23:38 await self._initialize_context() 2023-10-12 14:23:38 File "/opt/nuclio/_nuclio_wrapper.py", line 188, in _initialize_context 2023-10-12 14:23:38 init_context_result = getattr(self._entrypoint_module, 'init_context')(self._context) 2023-10-12 14:23:38 File "/opt/nuclio/main.py", line 21, in init_context 2023-10-12 14:23:38 model = ModelHandler(labels) 2023-10-12 14:23:38 File "/opt/nuclio/model_handler.py", line 14, in init 2023-10-12 14:23:38 self.load_network(model_config_path="GroundingDINO_SwinT_OGC.py", model_checkpoint_path="groundingdino_swint_ogc.pth") 2023-10-12 14:23:38 File "/opt/nuclio/model_handler.py", line 28, in load_network 2023-10-12 14:23:38 raise Exception(f"Cannot load model {model_checkpoint_path}: {e}") 2023-10-12 14:23:38 Exception: Cannot load model groundingdino_swint_ogc.pth: name 'false' is not defined 2023-10-12 14:23:38 Exception ignored in: <function _TemporaryFileCloser.del at 0x7f1ddd299750> 2023-10-12 14:23:38 Traceback (most recent call last): 2023-10-12 14:23:38 File "/usr/lib/python3.10/tempfile.py", line 589, in del 2023-10-12 14:23:38 File "/usr/lib/python3.10/tempfile.py", line 585, in close 2023-10-12 14:23:38 FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp8eh_wv58/tmpjrzsznt8.py' 2023-10-12 14:23:39 panic: Wrapper process for worker 0 exited unexpectedly with: exit status 1 2023-10-12 14:23:39 2023-10-12 14:23:39 goroutine 41 [running]: 2023-10-12 14:23:39 github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(AbstractRuntime).watchWrapperProcess(0xc00039c0e0) 2023-10-12 14:23:39 /home/runner/work/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:639 +0x454 2023-10-12 14:23:39 created by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(AbstractRuntime).startWrapper 2023-10-12 14:23:39 /home/runner/work/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:296 +0x1fd 2023-10-12 14:23:48 /usr/local/lib/python3.10/dist-packages/groundingdino/models/GroundingDINO/ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only! 2023-10-12 14:23:48 warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!") 2023-10-12 14:23:48 Traceback (most recent call last): 2023-10-12 14:23:48 File "/opt/nuclio/model_handler.py", line 23, in load_network 2023-10-12 14:23:48 self.model = load_model(model_config_path=model_config_path, model_checkpoint_path=model_checkpoint_path, 2023-10-12 14:23:48 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/inference.py", line 30, in load_model 2023-10-12 14:23:48 args = SLConfig.fromfile(model_config_path) 2023-10-12 14:23:48 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/slconfig.py", line 185, in fromfile 2023-10-12 14:23:48 cfg_dict, cfg_text = SLConfig._file2dict(filename) 2023-10-12 14:23:48 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/slconfig.py", line 90, in _file2dict 2023-10-12 14:23:48 mod = import_module(temp_module_name) 2023-10-12 14:23:48 File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module 2023-10-12 14:23:48 return _bootstrap._gcd_import(name[level:], package, level) 2023-10-12 14:23:48 File "", line 1050, in _gcd_import 2023-10-12 14:23:48 File "", line 1027, in _find_and_load 2023-10-12 14:23:48 File "", line 1006, in _find_and_load_unlocked 2023-10-12 14:23:48 File "", line 688, in _load_unlocked 2023-10-12 14:23:48 File "", line 883, in exec_module 2023-10-12 14:23:48 File "", line 241, in _call_with_frames_removed 2023-10-12 14:23:48 File "/tmp/tmpuhmaw98f/tmpa5mm56vl.py", line 1, in 2023-10-12 14:23:48 NameError: name 'false' is not defined. Did you mean: 'False'? 2023-10-12 14:23:48 2023-10-12 14:23:48 During handling of the above exception, another exception occurred: 2023-10-12 14:23:48 2023-10-12 14:23:48 Traceback (most recent call last): 2023-10-12 14:23:48 File "/opt/nuclio/_nuclio_wrapper.py", line 480, in 2023-10-12 14:23:48 run_wrapper() 2023-10-12 14:23:48 File "/opt/nuclio/_nuclio_wrapper.py", line 468, in run_wrapper 2023-10-12 14:23:48 loop.run_until_complete(wrapper_instance.initialize()) 2023-10-12 14:23:48 File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete 2023-10-12 14:23:48 return future.result() 2023-10-12 14:23:48 File "/opt/nuclio/_nuclio_wrapper.py", line 165, in initialize 2023-10-12 14:23:48 await self._initialize_context() 2023-10-12 14:23:48 File "/opt/nuclio/_nuclio_wrapper.py", line 188, in _initialize_context 2023-10-12 14:23:48 init_context_result = getattr(self._entrypoint_module, 'init_context')(self._context) 2023-10-12 14:23:48 File "/opt/nuclio/main.py", line 21, in init_context 2023-10-12 14:23:48 model = ModelHandler(labels) 2023-10-12 14:23:48 File "/opt/nuclio/model_handler.py", line 14, in init 2023-10-12 14:23:48 self.load_network(model_config_path="GroundingDINO_SwinT_OGC.py", model_checkpoint_path="groundingdino_swint_ogc.pth") 2023-10-12 14:23:48 File "/opt/nuclio/model_handler.py", line 28, in load_network 2023-10-12 14:23:48 raise Exception(f"Cannot load model {model_checkpoint_path}: {e}") 2023-10-12 14:23:48 Exception: Cannot load model groundingdino_swint_ogc.pth: name 'false' is not defined 2023-10-12 14:23:48 Exception ignored in: <function _TemporaryFileCloser.del at 0x7f58d9e49750> 2023-10-12 14:23:48 Traceback (most recent call last): 2023-10-12 14:23:48 File "/usr/lib/python3.10/tempfile.py", line 589, in del 2023-10-12 14:23:48 File "/usr/lib/python3.10/tempfile.py", line 585, in close 2023-10-12 14:23:48 FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpuhmaw98f/tmpa5mm56vl.py' 2023-10-12 14:23:49 panic: Wrapper process for worker 0 exited unexpectedly with: exit status 1 2023-10-12 14:23:49 2023-10-12 14:23:49 goroutine 48 [running]: 2023-10-12 14:23:49 github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(AbstractRuntime).watchWrapperProcess(0xc00039c0e0) 2023-10-12 14:23:49 /home/runner/work/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:639 +0x454 2023-10-12 14:23:49 created by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(AbstractRuntime).startWrapper 2023-10-12 14:23:49 /home/runner/work/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:296 +0x1fd

[KTXKIKI]

metadata: name: pth-idearesearch-dino-vit-h-cpu namespace: cvat annotations: name: Grounding-Dino-CPU # 自定义万物目标检测名称 type: detector framework: pytorch spec: | [ { "id": 0, "name": "person" }, { "id": 1, "name": "bicycle" }, { "id": 2, "name": "car" }, { "id": 3, "name": "motorbike" }, { "id": 4, "name": "aeroplane" }, { "id": 5, "name": "bus" }, { "id": 6, "name": "train" }, { "id": 7, "name": "truck" }, { "id": 8, "name": "boat" }, { "id": 9, "name": "traffic light" }, { "id": 10, "name": "fire hydrant" }, { "id": 11, "name": "stop sign" }, { "id": 12, "name": "parking meter" }, { "id": 13, "name": "bench" }, { "id": 14, "name": "bird" }, { "id": 15, "name": "cat" }, { "id": 16, "name": "dog" }, { "id": 17, "name": "horse" }, { "id": 18, "name": "sheep" }, { "id": 19, "name": "cow" }, { "id": 20, "name": "elephant" }, { "id": 21, "name": "bear" }, { "id": 22, "name": "zebra" }, { "id": 23, "name": "giraffe" }, { "id": 24, "name": "backpack" }, { "id": 25, "name": "umbrella" }, { "id": 26, "name": "handbag" }, { "id": 27, "name": "tie" }, { "id": 28, "name": "suitcase" }, { "id": 29, "name": "frisbee" }, { "id": 30, "name": "skis" }, { "id": 31, "name": "snowboard" }, { "id": 32, "name": "sports ball" }, { "id": 33, "name": "kite" }, { "id": 34, "name": "baseball bat" }, { "id": 35, "name": "baseball glove" }, { "id": 36, "name": "skateboard" }, { "id": 37, "name": "surfboard" }, { "id": 38, "name": "tennis racket" }, { "id": 39, "name": "bottle" }, { "id": 40, "name": "wine glass" }, { "id": 41, "name": "cup" }, { "id": 42, "name": "fork" }, { "id": 43, "name": "knife" }, { "id": 44, "name": "spoon" }, { "id": 45, "name": "bowl" }, { "id": 46, "name": "banana" }, { "id": 47, "name": "apple" }, { "id": 48, "name": "sandwich" }, { "id": 49, "name": "orange" }, { "id": 50, "name": "broccoli" }, { "id": 51, "name": "carrot" }, { "id": 52, "name": "hot dog" }, { "id": 53, "name": "pizza" }, { "id": 54, "name": "donut" }, { "id": 55, "name": "cake" }, { "id": 56, "name": "chair" }, { "id": 57, "name": "sofa" }, { "id": 58, "name": "pottedplant" }, { "id": 59, "name": "bed" }, { "id": 60, "name": "diningtable" }, { "id": 61, "name": "toilet" }, { "id": 62, "name": "tvmonitor" }, { "id": 63, "name": "laptop" }, { "id": 64, "name": "mouse" }, { "id": 65, "name": "remote" }, { "id": 66, "name": "keyboard" }, { "id": 67, "name": "cell phone" }, { "id": 68, "name": "microwave" }, { "id": 69, "name": "oven" }, { "id": 70, "name": "toaster" }, { "id": 71, "name": "sink" }, { "id": 72, "name": "refrigerator" }, { "id": 73, "name": "book" }, { "id": 74, "name": "clock" }, { "id": 75, "name": "vase" }, { "id": 76, "name": "scissors" }, { "id": 77, "name": "teddy bear" }, { "id": 78, "name": "hair drier" }, { "id": 79, "name": "toothbrush" } ]

spec: description: Grounding dino via pytorch runtime: 'python:3.8' handler: main:handler eventTimeout: 10000s build: image: cvat.pth.idearesearch.grounding_dino.vit_h-cpu baseImage: ubuntu:22.04

directives:
  preCopy:
  # disable interactive frontend
    - kind: ENV
      value: DEBIAN_FRONTEND=noninteractive
  # set workdir
    - kind: WORKDIR
      value: /opt/nuclio/grounding_dino
  # install basic deps
    - kind: RUN
      value: apt-get update && apt-get -y install curl git python3 python3-pip ffmpeg libsm6 libxext6
  # install grounding-dino deps
    - kind: RUN
      value: pip3 install torch torchvision torchaudio opencv-python pycocotools matplotlib
  # install grounding-dino code
    - kind: RUN
      value: pip3 install git+https://github.com/IDEA-Research/GroundingDINO.git
  # download grounding-dino weights
    - kind: RUN
      value: curl -O https://github.com/IDEA-Research/GroundingDINO/blob/main/groundingdino/config/GroundingDINO_SwinT_OGC.py
    - kind: RUN
      value: curl -O https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
  # map pip3 and python3 to pip and python
    - kind: RUN
      value: ln -s /usr/bin/pip3 /usr/local/bin/pip && ln -s /usr/bin/python3 /usr/bin/python

triggers: myHttpTrigger: maxWorkers: 1 kind: 'http' workerAvailabilityTimeoutMilliseconds: 100000000 attributes: maxRequestBodySize: 53687091200 # 50GB

platform: attributes: restartPolicy: name: always maximumRetryCount: 3 mountMode: volume network: cvat-develop_cvat

[KTXKIKI]

import base64 import io import json

import yaml from model_handler import ModelHandler from PIL import Image

def init_context(context): context.logger.info("Init context... 0%")

# Read labels
with open("/opt/nuclio/function.yaml", 'rb') as function_file:
    functionconfig = yaml.safe_load(function_file)

labels_spec = functionconfig['metadata']['annotations']['spec']
labels = {item['id']: item['name'] for item in json.loads(labels_spec)}

# Read the DL model
model = ModelHandler(labels)
context.user_data.model = model

context.logger.info("Init context...100%")

def handler(context, event): context.logger.info("Run grounding dino model") data = event.body buf = io.BytesIO(base64.b64decode(data["image"])) threshold = float(data.get("threshold", 0.35)) image = Image.open(buf)

results = context.user_data.model.infer(image, threshold)

return context.Response(body=json.dumps(results), headers={},
    content_type='application/json', status_code=200)

[KTXKIKI]

mogul grounding dino There is a problem with the deployment of the CPU and GPU. Is there a version update？