search

dusty-nv / jetson-inference

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
https://developer.nvidia.com/embedded/twodaystoademo
MIT License
7.71k stars 2.97k forks source link

the JetPack 5 Pytorch 2.1.0 wheel (torch.sparse) seems to be inconsistent with the official release #1780

Open luo-kaguya opened 8 months ago

luo-kaguya commented 8 months ago

I try to use the torch.sparse to achieve the accelerate of semi-structed (2:4) sparsity. This feature is in the Pytorch 2.1.0 release version. But JetPack 5 Pytorch 2.1.0 wheel lack of the new features.

This is the Pytorch 2.1.0 docs image

This is the (Github Pytorch 2.1.0) torch.sparse.init.py image

This is the (JetPack 5 Pytorch 2.1.0) torch.sparse.init.py lack of from torch.sparse import to_sparse_semi_structured, SparseSemiStructuredTensor image image

I check another version (JetPack 6 Pytorch 2.1.0), it has the new feature. So how can I get the new version Pytorch of Jetpack5? image

Looking forward to your reply, thank you!

dusty-nv commented 8 months ago

@luo-kaguya I believe the JetPack 5 version of that PyTorch 2.1 wheel was built before PyTorch 2.1 was officially released.

You can rebuild PyTorch from source using the latest 2.1 sources, either by following the build instructions from this post below, or this pytorch builder dockerfile:

luo-kaguya commented 7 months ago

@dusty-nv Thanks for your response, I followed your advice and tried to rebuild the torch-2.1.0 from the source. I get the wheel and seem to install it on my jetson-orin with cuda-11.4. However, it can't use the cuda. I am not sure what happened because the Cuda is available when I install the official wheel. I hope that maybe the official torch-2.1.0 wheel can be updated to the latest version. Thanks a lot. image image image image

dusty-nv commented 7 months ago

@luo-kaguya my guess is that PyTorch didn't' detect CUDA Toolkit on your Jetson when it was compiling. Soon after the build process begins, PyTorch will print out a configuration summary before it actually starts compiling - check that to make sure it detects CUDA and cuDNN. Did you build PyTorch 'by hand' or with my Dockerfile builder?

luo-kaguya commented 7 months ago

@dusty-nv I checked the configuration summary, and there is indeed a lack of Cuda paths. I compile it again and it can be detected. Your advice is very constructive, thanks a lot.