PaddleSharp 🌟

English | 简体中文

💗 .NET Wrapper for PaddleInference C API, support Windows(x64) 💻, NVIDIA Cuda 10.2+ based GPU 🎮 and Linux(Ubuntu-22.04 x64) 🐧, currently contained following main components:

PaddleOCR 📖 support 14 OCR languages model download on-demand, allow rotated text angle detection, 180 degree text detection, also support table recognition 📊.
PaddleDetection 🎯 support PPYolo detection model and PicoDet model 🏹.
RotationDetection 🔄 use Baidu's official text_image_orientation_infer model to detect text picture's rotation angle(0, 90, 180, 270).
PaddleNLP ChineseSegmenter 📚 support PaddleNLP Lac Chinese segmenter model, supports tagging/customized words.
Paddle2Onnx 🔄 Allow user export ONNX model using C#.

NuGet Packages/Docker Images 📦

Release notes 📝

Please checkout this page 📄.

Infrastructure packages 🏗️

NuGet Package 💼	Version 📌	Description 📚
Sdcb.PaddleInference		Paddle Inference C API .NET binding ⚙️

Native packages 🏗️

Package	Version 📌	Description
Sdcb.PaddleInference.runtime.win64.mkl		win64+mkldnn
Sdcb.PaddleInference.runtime.win64.openblas		win64+openblas
Sdcb.PaddleInference.runtime.win64.openblas-noavx		no AVX, for old CPUs
Sdcb.PaddleInference.runtime.win64.cu120-sm86-89		for NVIDIA 30/40 series
Sdcb.PaddleInference.runtime.win64.cu120-sm80		for NVIDIA A100/A10
Sdcb.PaddleInference.runtime.win64.cu120-sm61-75		for NVIDIA 10/20 series

Note: cu120 means CUDA 12.0, it's compiled in CUDA 12.0.1/cuDNN 8.9.7.29/Tensor RT 8.6.1.6 version.

Linux OS packages(preview):

Package	Version 📌	Description
Sdcb.PaddleInference.runtime.linux-loongarch64		Loongnix GCC 8.2 Loongarch64
Sdcb.PaddleInference.runtime.linux64.mkl.gcc82		Linux-x64 GCC 8.2(tested in Ubuntu 22.04)

Be aware, as the Linux operating system cannot modify the value of LD_LIBRARY_PATH at runtime. If dependent dynamic libraries (such as libcommon.so) are loaded before the main dynamic library (such as libpaddle_inference_c.so), and also due to protobuf errors reported: https://github.com/PaddlePaddle/Paddle/issues/62670

Therefore, all NuGet packages for Linux operating systems are in a preview state, and I'm unable to resolve this issue. Currently, if you are using the NuGet package on Linux, you need to manually specify the LD_LIBRARY_PATH environment variable before running the program, using the following commands:

For x64 CPUs: export LD_LIBRARY_PATH=/<program directory>/bin/Debug/net8.0/runtimes/linux-x64/native:$LD_LIBRARY_PATH
For Loongson 5000 or above CPUs (linux-loongarch64): export LD_LIBRARY_PATH=/<program directory>/bin/Debug/net8.0/runtimes/linux-loongarch64/native:$LD_LIBRARY_PATH

Some of packages already deprecated(Version <= 2.5.0):	Package	Version 📌	Description
Sdcb.PaddleInference.runtime.win64.cuda102_cudnn76_tr72_sm61_75		win64/CUDA 10.2/cuDNN 7.6/TensorRT 7.2/sm61+sm75
Sdcb.PaddleInference.runtime.win64.cuda118_cudnn86_tr85_sm86_89		win64/CUDA 11.8/cuDNN 8.6/TensorRT 8.5/sm86+sm89
Sdcb.PaddleInference.runtime.win64.cuda117_cudnn84_tr84_sm86		win64/CUDA 11.7/cuDNN 8.4/TensorRT 8.4/sm86
Sdcb.PaddleInference.runtime.win64.cuda102_cudnn76_sm61_75		win64/CUDA 10.2/cuDNN 7.6/sm61+sm75
Sdcb.PaddleInference.runtime.win64.cuda116_cudnn84_sm86_onnx		win64/CUDA 11.6/cuDNN 8.4/sm86/onnx

Any other packages that starts with Sdcb.PaddleInference.runtime might deprecated.

Baidu packages were downloaded from here: https://www.paddlepaddle.org.cn/inference/master/guides/install/download_lib.html#windows

All Windows packages were compiled manually by me.

Baidu official GPU packages are too large(>1.5GB) to publish to nuget.org, there is a limitation of 250MB when upload to Github, there is some related issues to this:

But You're good to build your own GPU nuget package using 01-build-native.linq 🛠️.

Paddle Devices

Mkldnn - PaddleDevice.Mkldnn()

Based on Mkldnn, generally fast
Openblas - PaddleDevice.Openblas()

Based on openblas, slower, but dependencies file smaller and consume lesser memory
Onnx - PaddleDevice.Onnx()

Based on onnxruntime, is also pretty fast and consume less memory
Gpu - PaddleDevice.Gpu()

Much faster but relies on NVIDIA GPU and CUDA

If you wants to use GPU, you should refer to FAQ How to enable GPU? section, CUDA/cuDNN/TensorRT need to be installed manually.

FAQ ❓

Why my code runs good in my windows machine, but DllNotFoundException in other machine: 💻

Please ensure the latest Visual C++ Redistributable was installed in Windows (typically it should automatically installed if you have Visual Studio installed) 🛠️ Otherwise, it will fail with the following error (Windows only):
```
DllNotFoundException: Unable to load DLL 'paddle_inference_c' or one of its dependencies (0x8007007E)
```
If it's Unable to load DLL OpenCvSharpExtern.dll or one of its dependencies, then most likely the Media Foundation is not installed in the Windows Server 2012 R2 machine:
Many old CPUs do not support AVX instructions, please ensure your CPU supports AVX, or download the x64-noavx-openblas DLLs and disable Mkldnn: PaddleDevice.Openblas() 🚀
If you're using Win7-x64, and your CPU does support AVX2, then you might also need to extract the following 3 DLLs into C:\Windows\System32 folder to make it run: 💾
- api-ms-win-core-libraryloader-l1-2-0.dll
- api-ms-win-core-processtopology-obsolete-l1-1-0.dll
- API-MS-Win-Eventing-Provider-L1-1-0.dll
You can download these 3 DLLs here: win7-x64-onnxruntime-missing-dlls.zip ⬇️

How to enable GPU? 🎮

Enable GPU support can significantly improve the throughput and lower the CPU usage. 🚀

Steps to use GPU in Windows:

(for Windows) Install the package: Sdcb.PaddleInference.runtime.win64.cu120* instead of Sdcb.PaddleInference.runtime.win64.mkl, do not install both. 📦
Install CUDA from NVIDIA, and configure environment variables to PATH or LD_LIBRARY_PATH (Linux) 🔧
Install cuDNN from NVIDIA, and configure environment variables to PATH or LD_LIBRARY_PATH (Linux) 🛠️
Install TensorRT from NVIDIA, and configure environment variables to PATH or LD_LIBRARY_PATH (Linux) ⚙️

You can refer to this blog page for GPU in Windows: 关于PaddleSharp GPU使用常见问题记录 📝

If you're using Linux, you need to compile your own OpenCvSharp4 environment following the docker build scripts and the CUDA/cuDNN/TensorRT configuration tasks. 🐧

After these steps are completed, you can try specifying PaddleDevice.Gpu() in the paddle device configuration parameter, then enjoy the performance boost! 🎉

Thanks & Sponsors 🙏

崔亮 https://github.com/cuiliang
梁现伟
深圳-钱文松
iNeuOS工业互联网操作系统：http://www.ineuos.net

Contact 📞

QQ group of C#/.NET computer vision technical communication (C#/.NET计算机视觉技术交流群): 579060605

sdcb / PaddleSharp

readme