search

bytedeco / javacv

Java interface to OpenCV, FFmpeg, and more
Other
7.45k stars 1.57k forks source link

Call opencv_imgcodecs.imwrite with GpuMat failed #1987

Closed xuyao178 closed 1 year ago

xuyao178 commented 1 year ago

There is a logic as below:

  1. Get the frame data from video;
  2. Convert frame to Mat;
  3. Init a GpuMat by Mat;
  4. Call opencv_imgcodecs.imwrite to save a picture as jpeg format;

show the code:

String filePath = outputPath + frame.timestamp + ".jpeg";
try (OpenCVFrameConverter.ToMat converter = new OpenCVFrameConverter.ToMat()) {
    Mat mat = converter.convert(frame);
    GpuMat gpuMat = new GpuMat(mat);
    opencv_imgcodecs.imwrite(filePath, gpuMat);
}

After I execute the code, I meet the error as below:

Caused by: java.lang.RuntimeException: OpenCV(4.6.0) /__w/javacpp-presets/javacpp-presets/opencv/cppbuild/linux-x86_64-gpu/opencv-4.6.0/modules/core/src/matrix_wrap.cpp:111: error: (-213:The function/feature is not implemented) You should explicitly call download method for cuda::GpuMat object in function 'getMat_'

Am I using it in the wrong way？

xuyao178 commented 1 year ago

debug log:

Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.javacpp.Loader
Debug: Loading /root/.javacpp/cache/javacpp-1.5.8-linux-x86_64.jar/org/bytedeco/javacpp/linux-x86_64/libjnijavacpp.so
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.javacpp.Pointer
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libva.so.1
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libdrm.so.2
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libva-drm.so.1
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libavutil.so.57
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libjniavutil.so
Debug: Loading class org.bytedeco.ffmpeg.avutil.LogCallback
Debug: Registering org.bytedeco.javacv.FFmpegLogCallback[address=0x7f0d50372d80,position=0,limit=1,capacity=1,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d50372d80,deallocatorAddress=0x7f0cd10f85a0]]
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libswresample.so.4
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libjniswresample.so
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libavcodec.so.59
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libjniavcodec.so
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.global.avformat
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.global.avformat
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libavformat.so.59
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libjniavformat.so
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swscale
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swscale
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libswscale.so.6
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libjniswscale.so
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.global.avformat
Debug: Loading class org.bytedeco.ffmpeg.global.postproc
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.postproc
Debug: Loading library postproc
Debug: Failed to load for postproc@.56: java.lang.UnsatisfiedLinkError: no postproc in java.library.path
Debug: Loading library jnipostproc
Debug: Failed to load for jnipostproc: java.lang.UnsatisfiedLinkError: no jnipostproc in java.library.path
Debug: Loading class org.bytedeco.ffmpeg.global.swscale
Debug: Loading class org.bytedeco.ffmpeg.global.avfilter
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.global.avformat
Debug: Loading class org.bytedeco.ffmpeg.global.postproc
Debug: Loading class org.bytedeco.ffmpeg.global.swscale
Debug: Loading class org.bytedeco.ffmpeg.global.avfilter
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libavfilter.so.8
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libjniavfilter.so
Debug: Loading class org.bytedeco.ffmpeg.global.avdevice
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.global.avformat
Debug: Loading class org.bytedeco.ffmpeg.global.postproc
Debug: Loading class org.bytedeco.ffmpeg.global.swscale
Debug: Loading class org.bytedeco.ffmpeg.global.avfilter
Debug: Loading class org.bytedeco.ffmpeg.global.avdevice
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libavdevice.so.59
Debug: Loading /root/.javacpp/cache/ffmpeg-5.1.2-1.5.8-linux-x86_64.jar/org/bytedeco/ffmpeg/linux-x86_64/libjniavdevice.so
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.global.avformat
Debug: Loading class org.bytedeco.ffmpeg.avformat.Read_packet_Pointer_BytePointer_int
Debug: Registering org.bytedeco.javacv.FFmpegFrameGrabber$ReadCallback[address=0x7f0d503a9510,position=0,limit=1,capacity=1,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d503a9510,deallocatorAddress=0x7f0d5407ce20]]
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.global.avformat
Debug: Loading class org.bytedeco.ffmpeg.avformat.Seek_Pointer_long_int
Debug: Registering org.bytedeco.javacv.FFmpegFrameGrabber$SeekCallback[address=0x7f0d502b2380,position=0,limit=1,capacity=1,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d502b2380,deallocatorAddress=0x7f0d5407d220]]
Debug: Opening org.bytedeco.javacpp.PointerScope@470c4fca
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.global.avformat
Debug: Loading class org.bytedeco.ffmpeg.avformat.AVFormatContext
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.javacpp.PointerPointer
Debug: Registering org.bytedeco.javacpp.PointerPointer[address=0x7f0d50136850,position=0,limit=8,capacity=8,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d50136850,deallocatorAddress=0x7f0db007fa50]]
Debug: Attaching org.bytedeco.javacpp.PointerPointer[address=0x7f0d50136850,position=0,limit=8,capacity=8,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d50136850,deallocatorAddress=0x7f0db007fa50]] to org.bytedeco.javacpp.PointerScope@470c4fca
Debug: Registering org.bytedeco.javacpp.PointerPointer[address=0x7f0d502d28b0,position=0,limit=8,capacity=8,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d502d28b0,deallocatorAddress=0x7f0db007fa50]]
Debug: Attaching org.bytedeco.javacpp.PointerPointer[address=0x7f0d502d28b0,position=0,limit=8,capacity=8,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d502d28b0,deallocatorAddress=0x7f0db007fa50]] to org.bytedeco.javacpp.PointerScope@470c4fca
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.avcodec.AVPacket
Debug: Registering org.bytedeco.ffmpeg.avcodec.AVPacket[address=0x7f0d502d2800,position=0,limit=1,capacity=1,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d502d2800,deallocatorAddress=0x7f0d8c0331a0]]
Debug: Attaching org.bytedeco.ffmpeg.avcodec.AVPacket[address=0x7f0d502d2800,position=0,limit=1,capacity=1,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d502d2800,deallocatorAddress=0x7f0d8c0331a0]] to org.bytedeco.javacpp.PointerScope@470c4fca
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.global.avformat
Debug: Loading class org.bytedeco.ffmpeg.avformat.AVInputFormat
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.javacpp.BytePointer
Info: Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/root/test2.mp4':

Info:   Metadata:

Info:     major_brand     :
Info: mp42
Info:

Info:     minor_version   :
Info: 0
Info:

Info:     compatible_brands:
Info: isommp42
Info:

Info:     creation_time   :
Info: 2020-01-10T11:45:03.000000Z
Info:

Info:     com.android.version:
Info: 9
Info:

Info:   Duration:
Info: 00:00:32.14
Info: , start:
Info: 0.000000
Info: , bitrate:
Info: 9131 kb/s
Info:

Info:   Stream #0:0
Info: [0x1]
Info: (eng)
Info: : Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 8738 kb/s
Info: , SAR 1:1 DAR 16:9
Info: ,
Info: 29.37 fps,
Info: 29.50 tbr,
Info: 90k tbn
Info:  (default)
Info:

Info:     Metadata:

Info:       creation_time   :
Info: 2020-01-10T11:45:03.000000Z
Info:

Info:       handler_name    :
Info: VideoHandle
Info:

Info:       vendor_id       :
Info: [0][0][0][0]
Info:

Info:     Side data:

Info:
Info: displaymatrix: rotation of -90.00 degrees
Info:

Info:   Stream #0:1
Info: [0x2]
Info: (eng)
Info: : Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s
Info:  (default)
Info:

Info:     Metadata:

Info:       creation_time   :
Info: 2020-01-10T11:45:03.000000Z
Info:

Info:       handler_name    :
Info: SoundHandle
Info:

Info:       vendor_id       :
Info: [0][0][0][0]
Info:

Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.global.avformat
Debug: Loading class org.bytedeco.ffmpeg.avformat.AVStream
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.avcodec.AVCodecParameters
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.avcodec.AVCodec
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.global.swresample
Debug: Loading class org.bytedeco.ffmpeg.global.avcodec
Debug: Loading class org.bytedeco.ffmpeg.avcodec.AVCodecContext
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.avutil.AVRational
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.ffmpeg.global.avutil
Debug: Loading class org.bytedeco.ffmpeg.avutil.AVFrame
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.javacpp.IntPointer
Debug: Closing org.bytedeco.javacpp.PointerScope@470c4fca
Debug: Opening org.bytedeco.javacpp.PointerScope@431009c2
Debug: Closing org.bytedeco.javacpp.PointerScope@431009c2
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading /root/.javacpp/cache/openblas-0.3.21-1.5.8-linux-x86_64.jar/org/bytedeco/openblas/linux-x86_64/libgcc_s.so.1
Debug: Loading /root/.javacpp/cache/openblas-0.3.21-1.5.8-linux-x86_64.jar/org/bytedeco/openblas/linux-x86_64/libquadmath.so.0
Debug: Loading /root/.javacpp/cache/openblas-0.3.21-1.5.8-linux-x86_64.jar/org/bytedeco/openblas/linux-x86_64/libgfortran.so.5
Debug: Loading library gfortran
Debug: Failed to load for gfortran@.4: java.lang.UnsatisfiedLinkError: no gfortran in java.library.path
Debug: Loading library gfortran
Debug: Failed to load for gfortran@.3: java.lang.UnsatisfiedLinkError: no gfortran in java.library.path
Debug: Loading /root/.javacpp/cache/openblas-0.3.21-1.5.8-linux-x86_64.jar/org/bytedeco/openblas/linux-x86_64/libopenblas_nolapack.so.0
Debug: Loading /root/.javacpp/cache/openblas-0.3.21-1.5.8-linux-x86_64.jar/org/bytedeco/openblas/linux-x86_64/libjniopenblas_nolapack.so
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading /root/.javacpp/cache/openblas-0.3.21-1.5.8-linux-x86_64.jar/org/bytedeco/openblas/linux-x86_64/libopenblas.so.0
Debug: Loading /root/.javacpp/cache/openblas-0.3.21-1.5.8-linux-x86_64.jar/org/bytedeco/openblas/linux-x86_64/libjniopenblas.so
Debug: Loading class org.bytedeco.opencv.global.opencv_core
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading class org.bytedeco.opencv.global.opencv_core
Debug: Loading library cudart
Debug: Failed to load for cudart@.11.0: java.lang.UnsatisfiedLinkError: no cudart in java.library.path
Debug: Loading library cublasLt
Debug: Failed to load for cublasLt@.11: java.lang.UnsatisfiedLinkError: no cublasLt in java.library.path
Debug: Loading library cublas
Debug: Failed to load for cublas@.11: java.lang.UnsatisfiedLinkError: no cublas in java.library.path
Debug: Loading library cufft
Debug: Failed to load for cufft@.10: java.lang.UnsatisfiedLinkError: no cufft in java.library.path
Debug: Loading library cudnn
Debug: Failed to load for cudnn@.8: java.lang.UnsatisfiedLinkError: no cudnn in java.library.path
Debug: Loading library nppc
Debug: Failed to load for nppc@.11: java.lang.UnsatisfiedLinkError: no nppc in java.library.path
Debug: Loading library nppial
Debug: Failed to load for nppial@.11: java.lang.UnsatisfiedLinkError: no nppial in java.library.path
Debug: Loading library nppicc
Debug: Failed to load for nppicc@.11: java.lang.UnsatisfiedLinkError: no nppicc in java.library.path
Debug: Loading library nppicom
Debug: Failed to load for nppicom@.11: java.lang.UnsatisfiedLinkError: no nppicom in java.library.path
Debug: Loading library nppidei
Debug: Failed to load for nppidei@.11: java.lang.UnsatisfiedLinkError: no nppidei in java.library.path
Debug: Loading library nppif
Debug: Failed to load for nppif@.11: java.lang.UnsatisfiedLinkError: no nppif in java.library.path
Debug: Loading library nppig
Debug: Failed to load for nppig@.11: java.lang.UnsatisfiedLinkError: no nppig in java.library.path
Debug: Loading library nppim
Debug: Failed to load for nppim@.11: java.lang.UnsatisfiedLinkError: no nppim in java.library.path
Debug: Loading library nppist
Debug: Failed to load for nppist@.11: java.lang.UnsatisfiedLinkError: no nppist in java.library.path
Debug: Loading library nppisu
Debug: Failed to load for nppisu@.11: java.lang.UnsatisfiedLinkError: no nppisu in java.library.path
Debug: Loading library nppitc
Debug: Failed to load for nppitc@.11: java.lang.UnsatisfiedLinkError: no nppitc in java.library.path
Debug: Loading library npps
Debug: Failed to load for npps@.11: java.lang.UnsatisfiedLinkError: no npps in java.library.path
Debug: Loading library cudnn_ops_infer
Debug: Failed to load for cudnn_ops_infer@.8: java.lang.UnsatisfiedLinkError: no cudnn_ops_infer in java.library.path
Debug: Loading library cudnn_ops_train
Debug: Failed to load for cudnn_ops_train@.8: java.lang.UnsatisfiedLinkError: no cudnn_ops_train in java.library.path
Debug: Loading library cudnn_adv_infer
Debug: Failed to load for cudnn_adv_infer@.8: java.lang.UnsatisfiedLinkError: no cudnn_adv_infer in java.library.path
Debug: Loading library cudnn_adv_train
Debug: Failed to load for cudnn_adv_train@.8: java.lang.UnsatisfiedLinkError: no cudnn_adv_train in java.library.path
Debug: Loading library cudnn_cnn_infer
Debug: Failed to load for cudnn_cnn_infer@.8: java.lang.UnsatisfiedLinkError: no cudnn_cnn_infer in java.library.path
Debug: Loading library cudnn_cnn_train
Debug: Failed to load for cudnn_cnn_train@.8: java.lang.UnsatisfiedLinkError: no cudnn_cnn_train in java.library.path
Debug: Loading /root/.javacpp/cache/opencv-4.6.0-1.5.8-linux-x86_64-gpu.jar/org/bytedeco/opencv/linux-x86_64-gpu/libopencv_cudev.so.406
Debug: Loading /root/.javacpp/cache/opencv-4.6.0-1.5.8-linux-x86_64-gpu.jar/org/bytedeco/opencv/linux-x86_64-gpu/libopencv_core.so.406
Debug: Loading /root/.javacpp/cache/opencv-4.6.0-1.5.8-linux-x86_64-gpu.jar/org/bytedeco/opencv/linux-x86_64-gpu/libopencv_imgproc.so.406
Debug: Loading /root/.javacpp/cache/opencv-4.6.0-1.5.8-linux-x86_64-gpu.jar/org/bytedeco/opencv/linux-x86_64-gpu/libjniopencv_core.so
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading class org.bytedeco.opencv.global.opencv_core
Debug: Loading class org.bytedeco.opencv.opencv_core.CvSlice
Debug: Registering org.bytedeco.opencv.opencv_core.CvSlice[address=0x7f0d50004270,position=0,limit=1,capacity=1,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d50004270,deallocatorAddress=0x7f0c26cfdeb0]]
Debug: Registering org.bytedeco.opencv.opencv_core.CvSlice[address=0x7f0d5007c290,position=0,limit=1,capacity=1,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d5007c290,deallocatorAddress=0x7f0c26cfdeb0]]
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading class org.bytedeco.opencv.global.opencv_core
Debug: Loading class org.bytedeco.opencv.opencv_core.AbstractArray
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading class org.bytedeco.opencv.global.opencv_core
Debug: Loading class org.bytedeco.opencv.opencv_core.Mat
Debug: Registering org.bytedeco.opencv.opencv_core.Mat[width=1920,height=1080,depth=8,channels=3]
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading class org.bytedeco.opencv.global.opencv_core
Debug: Loading class org.bytedeco.opencv.opencv_core.GpuMat
Debug: Registering org.bytedeco.opencv.opencv_core.GpuMat[address=0x7f0d504d3480,position=0,limit=1,capacity=1,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d504d3480,deallocatorAddress=0x7f0c26d00240]]
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading class org.bytedeco.opencv.global.opencv_core
Debug: Loading class org.bytedeco.opencv.global.opencv_imgproc
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading class org.bytedeco.opencv.global.opencv_core
Debug: Loading class org.bytedeco.opencv.global.opencv_imgproc
Debug: Loading /root/.javacpp/cache/opencv-4.6.0-1.5.8-linux-x86_64-gpu.jar/org/bytedeco/opencv/linux-x86_64-gpu/libjniopencv_imgproc.so
Debug: Loading class org.bytedeco.opencv.global.opencv_imgcodecs
Debug: Loading /root/.javacpp/cache/opencv-4.6.0-1.5.8-linux-x86_64-gpu.jar/org/bytedeco/opencv/linux-x86_64-gpu/libopencv_imgcodecs.so.406
Debug: Loading /root/.javacpp/cache/opencv-4.6.0-1.5.8-linux-x86_64-gpu.jar/org/bytedeco/opencv/linux-x86_64-gpu/libjniopencv_imgcodecs.so
Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d50410d70,deallocatorAddress=0x7f0c26cffe50]
Debug: Collecting org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x0,deallocatorAddress=0x0]
Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d50136850,deallocatorAddress=0x7f0db007fa50]
Debug: Collecting org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x0,deallocatorAddress=0x0]
Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d502d28b0,deallocatorAddress=0x7f0db007fa50]
Debug: Collecting org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x0,deallocatorAddress=0x0]
Debug: Releasing org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x7f0d502d2800,deallocatorAddress=0x7f0d8c0331a0]
Debug: Collecting org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x0,deallocatorAddress=0x0]
xuyao178 commented 1 year ago

GPU and Driver Info: image

saudet commented 1 year ago

That function most likely doesn't support GpuMat. Use a normal Mat instead.

xuyao178 commented 1 year ago

yes, use Mat is working. Is there any way to encode jpeg by GPU?

saudet commented 1 year ago

OpenCV doesn't support that, but it's possible to do that with nvJPEG: https://github.com/bytedeco/javacpp-presets/blob/master/cuda/samples/SampleJpeg.java

xuyao178 commented 1 year ago

yes, I found this sample as you say. It's seems jpeg decode sample only. I got a nvjpeg jpeg encode sample from nvidia.com.

nvjpegHandle_t nv_handle;
nvjpegEncoderState_t nv_enc_state;
nvjpegEncoderParams_t nv_enc_params;
cudaStream_t stream;

// initialize nvjpeg structures
nvjpegCreateSimple(&nv_handle);
nvjpegEncoderStateCreate(nv_handle, &nv_enc_state, stream);
nvjpegEncoderParamsCreate(nv_handle, &nv_enc_params, stream);

nvjpegImage_t nv_image;
// Fill nv_image with image data, let's say 640x480 image in RGB format

// Compress image
nvjpegEncodeImage(nv_handle, nv_enc_state, nv_enc_params,
    &nv_image, NVJPEG_INPUT_RGB, 640, 480, stream);

// get compressed stream size
size_t length;
nvjpegEncodeRetrieveBitstream(nv_handle, nv_enc_state, NULL, &length, stream);
// get stream itself
cudaStreamSynchronize(stream);
std::vector<char> jpeg(length);
nvjpegEncodeRetrieveBitstream(nv_handle, nv_enc_state, jpeg.data(), &length, 0);

// write stream to file
cudaStreamSynchronize(stream);
std::ofstream output_file("test.jpg", std::ios::out | std::ios::binary);
output_file.write(jpeg.data(), length);
output_file.close();

And I try to translate it to java code like below:

public static void checkNvJpeg(String functionName, int result) {
    if (result != NVJPEG_STATUS_SUCCESS) {
        throw new IllegalStateException(String.format("%s returned '%d'", functionName, result));
    }
}

public static void checkCUDA(String functionName, int result) {
    if (result != CUDA_SUCCESS) {
        throw new IllegalStateException(String.format("%s returned '%d'", functionName, result));
    }
}

/**
 * encode jpeg by GPU
 *
 * @param saveToFile
 */
public static void encodeJpegByGpu(File saveToFile) {
    //create handle、state、params
    nvjpegHandle nvjpegHandle = new nvjpegHandle();
    nvjpegEncoderState nvjpegEncoderState = new nvjpegEncoderState();
    nvjpegEncoderParams nvjpegEncoderParams = new nvjpegEncoderParams();
    checkNvJpeg("nvjpegCreateSimple", nvjpegCreateSimple(nvjpegHandle));
    checkNvJpeg("nvjpegEncoderParamsCreate", nvjpegEncoderParamsCreate(nvjpegHandle, nvjpegEncoderParams, null));
    checkNvJpeg("nvjpegEncoderStateCreate", nvjpegEncoderStateCreate(nvjpegHandle, nvjpegEncoderState, null));
    // fill source sample data
    nvjpegImage_t source = new nvjpegImage_t();
    int image_width = 640;
    int image_height = 480;
    long channel_size = image_width * image_height;
    for (int i = 0; i < 3; i++) {
        source.pitch(i, image_width);
        checkCUDA("cudaMalloc", cudaMalloc(source.channel(i), channel_size));
        checkCUDA("cudaMemset", cudaMemset(source.channel(i), 50 * 40 * i, channel_size));
    }

    // Compress image
    CUstream_st stream = new CUstream_st();
    int imgWidth = 640;
    int imgHeight = 480;
    checkNvJpeg("nvjpegEncodeImage",
            nvjpegEncodeImage(nvjpegHandle, nvjpegEncoderState, nvjpegEncoderParams, source, NVJPEG_INPUT_RGB,
                    imgWidth, imgHeight, stream));
    // get compressed stream size
    ByteBuffer nullByteBuffer = null;
    SizeTPointer sizeTPointer = new SizeTPointer(0);
    checkNvJpeg("nvjpegEncodeRetrieveBitstream",
            nvjpegEncodeRetrieveBitstream(nvjpegHandle, nvjpegEncoderState, nullByteBuffer, sizeTPointer, stream));
    // get stream itself
    checkCUDA("cudaStreamSynchronize", cudaStreamSynchronize(stream));
    BytePointer jpegBytePointer = new BytePointer(sizeTPointer.get());
    checkNvJpeg("nvjpegEncodeRetrieveBitstream",
            nvjpegEncodeRetrieveBitstream(nvjpegHandle, nvjpegEncoderState, jpegBytePointer, sizeTPointer, stream));
    // write stream to file
    checkCUDA("cudaStreamSynchronize", cudaStreamSynchronize(stream));
    ByteBuffer imgByteBuffer = jpegBytePointer.asBuffer();
    FileUtil.writeBytes(imgByteBuffer.array(), saveToFile);
}

But failed at cudaMalloc(source.channel(i), channel_size) and return with error code 1. It's seems I call it with bad parameters. Can you help check the code sample?

devjeonghwan commented 1 year ago

Let's try like this

nvjpegImage_t source = new nvjpegImage_t();

int image_width = 640;
int image_height = 480;
long channel_size = image_width * image_height;

for (int i = 0; i < 3; i++) {
    source.pitch(i, image_width);
    BytePointer deviceMemory = new BytePointer();

    checkCUDA("cudaMalloc", cudaMalloc(deviceMemory, channel_size));
    checkCUDA("cudaMemset", cudaMemset(deviceMemory, 50 * 40 * i, channel_size));

    source.channel(i, deviceMemory);
}
devjeonghwan commented 1 year ago

Ok, I wrote fully working encode example.

import org.bytedeco.cuda.cudart.CUctx_st;
import org.bytedeco.cuda.cudart.CUstream_st;
import org.bytedeco.cuda.nvjpeg.*;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.javacpp.PointerPointer;
import org.bytedeco.javacpp.SizeTPointer;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;

import static org.bytedeco.cuda.global.cudart.*;
import static org.bytedeco.cuda.global.nvjpeg.*;

public class Main {
    static class devMalloc extends tDevMalloc {
        final static devMalloc instance = new devMalloc().retainReference();

        @Override
        public int call(PointerPointer pointerPointer, long l) {
            return cudaMalloc(pointerPointer, l);
        }
    }

    static class dev_free extends tDevFree {
        final static dev_free instance = new dev_free().retainReference();

        @Override
        public int call(Pointer pointer) {
            return cudaFree(pointer);
        }
    }

    static class hostMalloc extends tPinnedMalloc {
        final static hostMalloc instance = new hostMalloc().retainReference();

        @Override
        public int call(PointerPointer pointerPointer, long l, int i) {
            return cudaHostAlloc(pointerPointer, l, i);
        }
    }

    static class hostFree extends tPinnedFree {
        final static hostFree instance = new hostFree().retainReference();

        @Override
        public int call(Pointer pointer) {
            return cudaFreeHost(pointer);
        }
    }

    public static void CHECK_CUDA(String functionName, int result) {
        if (result != CUDA_SUCCESS) {
            throw new IllegalStateException(String.format("%s returned '%d'", functionName, result));
        }
    }

    public static void CHECK_NVJPEG(String functionName, int result) {
        if (result != NVJPEG_STATUS_SUCCESS) {
            throw new IllegalStateException(String.format("%s returned '%d'", functionName, result));
        }
    }

    public static void main(String[] args) throws IOException {
        int imageWidth = 640;
        int imageHeight = 480;

        nvjpegDevAllocator_t devAllocator = new nvjpegDevAllocator_t();
        devAllocator.dev_malloc(devMalloc.instance);
        devAllocator.dev_free(dev_free.instance);

        nvjpegPinnedAllocator_t pinnedAllocator = new nvjpegPinnedAllocator_t();
        pinnedAllocator.pinned_malloc(hostMalloc.instance);
        pinnedAllocator.pinned_free(hostFree.instance);

        // Initialize Cuda
        CUctx_st context = new CUctx_st();
        CHECK_CUDA("cuInit", cuInit(0));
        CHECK_CUDA("cuCtxCreate", cuCtxCreate(context, CU_CTX_SCHED_BLOCKING_SYNC, 0));

        // Create Cuda Stream
        CUstream_st stream = new CUstream_st();
        CHECK_CUDA("cuStreamCreate", cuStreamCreate(stream, 0));

        // Create Handle
        nvjpegHandle nvjpegHandle = new nvjpegHandle();
        CHECK_NVJPEG("nvjpegCreateEx", nvjpegCreateEx(NVJPEG_BACKEND_DEFAULT, devAllocator, pinnedAllocator, NVJPEG_FLAGS_DEFAULT, nvjpegHandle));

        // Create Encoder Components
        nvjpegEncoderState nvjpegEncoderState = new nvjpegEncoderState();
        nvjpegEncoderParams nvjpegEncoderParams = new nvjpegEncoderParams();
        CHECK_NVJPEG("nvjpegEncoderParamsCreate", nvjpegEncoderParamsCreate(nvjpegHandle, nvjpegEncoderParams, stream));
        CHECK_NVJPEG("nvjpegEncoderStateCreate", nvjpegEncoderStateCreate(nvjpegHandle, nvjpegEncoderState, stream));
        CHECK_NVJPEG("nvjpegEncoderParamsSetSamplingFactors", nvjpegEncoderParamsSetSamplingFactors(nvjpegEncoderParams, NVJPEG_CSS_444, stream));

        // Create Jpeg Image
        nvjpegImage_t nvjpegImage = new nvjpegImage_t();

        long channelSize = imageWidth * imageHeight;

        for (int i = 0; i < 3; i++) {
            nvjpegImage.pitch(i, imageWidth);
            BytePointer deviceMemory = new BytePointer();

            CHECK_CUDA("cudaMalloc", cudaMalloc(deviceMemory, channelSize));
            // Make Blue Color
            CHECK_CUDA("cudaMemset", cudaMemset(deviceMemory, i == 0 ? 255 : 0, channelSize));

            nvjpegImage.channel(i, deviceMemory);
        }

        // Compress image
        CHECK_NVJPEG("nvjpegEncodeImage", nvjpegEncodeImage(nvjpegHandle, nvjpegEncoderState, nvjpegEncoderParams, nvjpegImage, NVJPEG_INPUT_BGR, imageWidth, imageHeight, stream));

        // Get compressed size
        SizeTPointer jpegSize = new SizeTPointer(1);
        CHECK_NVJPEG("nvjpegEncodeRetrieveBitstream", nvjpegEncodeRetrieveBitstream(nvjpegHandle, nvjpegEncoderState, (BytePointer) null, jpegSize, stream));

        // Retrieve Bitstream
        BytePointer jpegBytePointer = new BytePointer(jpegSize.get());
        CHECK_NVJPEG("nvjpegEncodeRetrieveBitstream", nvjpegEncodeRetrieveBitstream(nvjpegHandle, nvjpegEncoderState, jpegBytePointer, jpegSize, stream));

        // Synchronize Cuda Stream
        CHECK_CUDA("cudaStreamSynchronize", cudaStreamSynchronize(stream));

        // Get bitstream to Java Array
        byte[] bytes = new byte[(int) jpegSize.get()];
        jpegBytePointer.get(bytes);

        // Write
        Files.write(new File("G:/out.jpg").toPath(), bytes);
    }
}

And..

image

xuyao178 commented 1 year ago

Thanks a lot. It's working!