Open mvpel opened 6 months ago
It occurred to me to try importing sys to check sys.path, and I found:
-------> sys.path is: ['', '/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python38.zip',
'/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python3.8',
'/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python3.8/lib-dynload',
'/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python3.8/site-packages']`
The only references to numpy in these paths was a collection of .h files:
Apptainer> find . -name 'numpy*'
./lib/python3.8/site-packages/nvidia/dali/include/dali/operators/reader/loader/numpy_loader.h
./lib/python3.8/site-packages/nvidia/dali/include/dali/operators/reader/loader/numpy_loader_gpu.h
./lib/python3.8/site-packages/nvidia/dali/include/dali/operators/reader/numpy_reader_gpu_op.h
./lib/python3.8/site-packages/nvidia/dali/include/dali/operators/reader/numpy_reader_op.h
Apptainer>
I threw in a sys.path.append() to add the /usr/local Python installation's path to sys.path, and that seems to have enabled Numpy to load. It threw an error but it appears to have loaded succesfully:
I0125 15:23:32.870758 1421448 dali_backend.cc:119] TRITONBACKEND_ModelInitialize: image_one255_494x648x3 (version 1)
I0125 15:23:32.870836 1421448 dali_backend.cc:131] Repository location: /triton.repos.d/image_one255_494x648x3
I0125 15:23:32.870843 1421448 dali_backend.cc:142] backend state is 'backend state'
Traceback (most recent call last):
File "<string>", line 5, in <module>
File "<frozen importlib._bootstrap>", line 553, in module_from_spec
AttributeError: 'NoneType' object has no attribute 'loader'
I0125 15:23:34.544454 1421448 dali_model.h:175] DALI pipeline from file /triton.repos.d/image_one255_494x648x3/1/dali.py
loaded successfully.
Hopefully it will work as intended in spite of the error. Any idea what might be going on? The message is pretty ambiguous.
With respect to Numpy, did I miss a step somewhere? Maybe I need to add Numpy to the DALI backend virtualenv?
Hey @mvpel
The easiest solution for this case would be not to use numpy at all. You can use the Constant
type like that:
images = images / types.Constant(255.0, dtype=types.FLOAT)
# or even better
images = images / 255.
Hey @mvpel The easiest solution for this case would be not to use numpy at all. You can use the
Constant
type like that:images = images / types.Constant(255.0, dtype=types.FLOAT) # or even better images = images / 255.
Nice, thanks! I'm puzzled that the example in the DALI math user guide didn't take that approach.
I'm using a very close approximation of the https://docs.nvidia.com/deeplearning/dali/user-guide/docs/math.html example to try to scale 0-255 RGB image values to 0.0-1.0 floating point numbers, due to the way our inference models were trained.
I tried running it without an "import numpy as np" line at first, which threw a NameError, but when I added that line, I got a "no module named numpy" error as Triton was working to load the model:
Here's my full pipeline in the dali.py:
I'm testing this under Triton v22.08, due to some program software approval requirements here, using the NGC Triton container.
Thanks for any suggestions you can offer!