client error

[ontheway16]

I have managed to run perf. test with the following:

docker run -it --ipc=host --net=host nvcr.io/nvidia/tritonserver:20.08-py3-clientsdk /bin/bash
cd install/bin
./perf_client -m yolov4 -u 127.0.0.1:8001 -i grpc --shared-memory system --concurrency-range 4

above is giving 95 inferences, with 2 x 1080Ti

But when I switch to client;

qgs@qgs-MS-7A74:~/yolov4-triton-tensorrt/clients/python$ python client.py -h
  File "client.py", line 142
    print(f"Received result buffer of size {result.shape}")
                                                         ^
SyntaxError: invalid syntax

its not accepting any parameters etc, directly giving the error.

[philipp-schmidt]

make sure you are running the client with a recent version of python, especially python3

you might have to call python3 explicitly if the default is python2

[philipp-schmidt]

You can btw execute the client on the host and you do not have to execute it inside the tensorrt container. I will add more instructions when the client is done with image and video mode.

[ontheway16]

Thanks for the comments. As I remember (also seen from the command prompt, above) I run the client from the host, not from the container. I am not sure about python version, though.. Probably v2.7.

One more thing, about readme, copying YOLOv4.wts "after" running the container, it could not see the newly copied .wts file within container, may be i did something wrong, but, I stopped the docker, copied file to where it belongs, then rerun docker stuff etc, mixed my mind and struggled a bit. It may help to check order of this operation in readme.

Also, as far as I understand, Triton server does not support inference via curl command, isn't it ?

(Edit. If client.py is under development, how about adding ability to infer images in a whole folder, some clients already have this).

[ontheway16]

An update with python3;

qgs@qgs-MS-7A74:~/triton/yolov4-triton-tensorrt/clients/python$ python3 client.py -h
Traceback (most recent call last):
  File "client.py", line 8, in <module>
    import tritonclient.grpc as grpcclient
ModuleNotFoundError: No module named 'tritonclient'

Also,

qgs@qgs-MS-7A74:~/triton/yolov4-triton-tensorrt/clients/python$ pip install nvidia-pyindex
Collecting nvidia-pyindex
  Downloading https://files.pythonhosted.org/packages/d0/e3/90965dfab72a8821646d9fa22f3839b436031f28bfec7620456558e8d275/nvidia-pyindex-1.0.4.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-7jR65E/nvidia-pyindex/setup.py", line 25, in <module>
        from pathlib import Path
    ImportError: No module named pathlib

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-7jR65E/nvidia-pyindex/

[philipp-schmidt]

Yes I had similar issues. Make sure you use the most recent upgrade version of pip and also make sure that your default python binary is python3, because nvidia-pyindex seems to be using python scripts under the hood to update the index.

ls -al /usr/bin | grep python

Check to which version the python soft link redirects. It needs to be python3. You can also check via python --version

[ontheway16]

Seems its pointing v2. I added

alias python=python3

to .bashrc but still seems the same with pip install nvidia-pyindex

also;

$ pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)

[philipp-schmidt]

you have to change the softlink to point to python3, aliasing won't help because pip will not run in your bash environment.

Run the next command only if you checked that its valid for you please (check all paths):

sudo ln -sf /usr/bin/python3 /usr/bin/python

this will change the python softlink to point to python3

and then run pip3 instead of pip

[ontheway16]

This one below helped, indeed;

alias python=python3
alias pip=pip3

then;

$ sudo pip3 install --upgrade pip

$ python3 -m pip --version
pip 20.2.3 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)

$ pip install nvidia-pyindex
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Defaulting to user installation because normal site-packages is not writeable
Collecting nvidia-pyindex
  Using cached nvidia-pyindex-1.0.4.tar.gz (5.0 kB)
Building wheels for collected packages: nvidia-pyindex
  Building wheel for nvidia-pyindex (setup.py) ... done
  Created wheel for nvidia-pyindex: filename=nvidia_pyindex-1.0.4-py3-none-any.whl size=4974 sha256=01dfd1eedda4c63c25684a2daa26a86fd35b713bf3a4e6c7e992ce7a4a3e7083
  Stored in directory: /home/qgs/.cache/pip/wheels/c4/ae/b4/3693ecffe187fb687172bdae75d983489ac88c562dde16220e
Successfully built nvidia-pyindex
Installing collected packages: nvidia-pyindex
Successfully installed nvidia-pyindex-1.0.4

But not for client.py, yet.

[philipp-schmidt]

Look into the updated client README

[ontheway16]

Success, thank you.

(yolov4-triton) qgs@qgs-MS-7A74:~/triton/yolov4-triton-tensorrt/clients/python$ python client.py image data/dog.jpg
Running in 'image' mode
Creating buffer from image file...
Invoking inference...
Done
Received result buffer of size (1, 7001, 1, 1)
Naive buffer sum: 43541.35546875
Detected objects: 4