`florence-2-vision` Application Crash on `IGX` with `qt_app.py`

Smart-Surgery-Eason commented 1 month ago

Description

I tried to follow the tutorial, with the florence-2-vision example. Knowing that the full QT UI app is not yet supported for iGPU. But I need to use UI in my app, and I perfer PySide. So, I decide to give a try.

I'm just wondering how to run the florence-2-vision example on IGX. Anyone tried it before?

Hardware Discription

Hardware: IGX Developer Kit
Software: IGX-SW 1.0 Production Release

Steps to Reproduce

Step 1 - clone the repo

git clone https://github.com/nvidia-holoscan/holohub.git
cd holohub

Step 2 - switch branch

git fetch --tags
git checkout -b v2.4.0 tags/holoscan-sdk-2.4.0

Step 3 - modify code

sed -i '/RUN MAX_JOBS=4 python3 -m pip install --no-cache-dir --no-build-isolation flash-attn/s/^/# /' ./applications/florence-2-vision/Dockerfile
sed -i 's/v2.3.0/v2.4.0/g' ./dev_container

Step 4 - build container

./dev_container build --docker_file ./applications/florence-2-vision/Dockerfile

Step 5 - launch container

./dev_container launch

Step 6 - build application

./run build florence-2-vision

Step 7 - launch application

./run launch florence-2-vision python

Output

A GUI window pop up for seconds and the app crash, with log below:

smartsurgery@smartsurgery-igx:~$ ./run launch florence-2-vision python
Run environment: export PYTHONPATH=${PYTHONPATH}:/../../../python/lib:/workspace/holohub/build/florence-2-vision/python/lib:/workspace/holohub && export HOLOHUB_DATA_PATH=/workspace/holohub/data && export HOLOSCAN_INPUT_PATH=/opt/nvidia/holoscan/data
Run workdir: cd /workspace/holohub/build/florence-2-vision
Run command: python3 /workspace/holohub/applications/florence-2-vision/qt_app.py
Run command args: 
[command] export PYTHONPATH=${PYTHONPATH}:/../../../python/lib:/workspace/holohub/build/florence-2-vision/python/lib:/workspace/holohub && export HOLOHUB_DATA_PATH=/workspace/holohub/data && export HOLOSCAN_INPUT_PATH=/opt/nvidia/holoscan/data
[command] cd /workspace/holohub/build/florence-2-vision
[command] python3 /workspace/holohub/applications/florence-2-vision/qt_app.py
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/usr/local/lib/python3.10/dist-packages/torchvision/image.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSs'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
  warn(
[info] [fragment.cpp:586] Loading extensions from configs...
[info] [gxf_executor.cpp:249] Creating context
/workspace/holohub/applications/florence-2-vision/qt_app.py:114: DeprecationWarning: 'exec_' will be removed in the future. Use 'exec' instead.
  sys.exit(app.exec_())
[info] [gxf_executor.cpp:1971] Activating Graph...
[info] [gxf_executor.cpp:2003] Running Graph...
[info] [gxf_executor.cpp:2005] Waiting for completion...
2024-09-12 03:18:07.092 INFO  gxf/std/greedy_scheduler.cpp@191: Scheduling 5 entities
Opening in BLOCKING MODE 
Fatal Python error: _PyGILState_NoteThreadState: Couldn't create autoTSSkey mapping
Python runtime state: initialized

Thread 0x0000ffff747df120 (most recent call first):
  File "/workspace/holohub/applications/florence-2-vision/qt_app.py", line 32 in run

Thread 0x0000ffff8cb9b6c0 (most recent call first):
  File "/workspace/holohub/applications/florence-2-vision/qt_app.py", line 114 in <module>
./run: line 83:   425 Aborted                 (core dumped) python3 /workspace/holohub/applications/florence-2-vision/qt_app.py
[command] export PYTHONPATH=/opt/nvidia/holoscan/python/lib:/workspace/holohub/benchmarks/holoscan_flow_benchmarking && export HOLOHUB_DATA_PATH="" && export HOLOSCAN_INPUT_PATH="/opt/nvidia/holoscan/data"

jjomier commented 1 month ago

Thanks for the report @Smart-Surgery-Eason. Can you confirm that you are running on IGX iGPU? We have seen the same issue on AGX (Jetson) and will look into it.

wendell-hom commented 1 month ago

@Smart-Surgery-Eason, I was investigating this issue and found that this might be related to a v4l2 issue. On your IGX, please run the following commands and then try running the florence-2-vision application again.

cd /usr/lib/aarch64-linux-gnu/
ls -l libv4l2.so.0.0.999999
sudo rm libv4l2.so.0.0.999999
sudo ln -s libv4l2.so.0.0.0.0  libv4l2.so.0.0.999999

nvidia-holoscan / holohub