search

asticode / go-astiav

Golang ffmpeg and libav C bindings
MIT License
393 stars 45 forks source link

Implementation for HW Context #32

Closed Cacsjep closed 9 months ago

Cacsjep commented 9 months ago

Hi

this is my first pr ever so sorry if that not correct, i have added hw context hope this follows your rules.

br

Cacsjep commented 9 months ago

I have Missing the hw Transfer, i will add it tomorrow

asticode commented 9 months ago

Let me know when your draft is done so that I know when to start reviewing it 👍

Cacsjep commented 9 months ago

OK, from my side it looks ok, codec context test use CUDA i think most developers that contribute or did tests about this repo have a cuda gpu, or what do u think? I remove my dedicated hw example and update the existing example for demuxing_decoding. I also test it already with my gpu for h264 and h265 decoding.

Cacsjep commented 9 months ago

I hope i understand all correctly and applied the requested changes correctly ;)

Cacsjep commented 9 months ago

i removed the check for pixel format at end because hardwarePixelFormat would be cuda but after transfer the final frame has correctly nv12 so we cant compare them

asticode commented 9 months ago

Perfect! Thanks for sticking with all the changes 👍

I'll merge the PR but I'll add the get_format callback + surely do some minor tweaks in the coming days 👍

Cacsjep commented 9 months ago

Yeah, awesome If i can give some back to you :)

asticode commented 9 months ago

FYI I've added a SetPixelFormatCallback method to CodecContext which is the equivalent to ctx->get_format = ... in ffmpeg's C hardware decoding example.

I've also updated the GO hardware decoding example.

@Cacsjep I don't have access to a GPU-powered machine (I can only confirm that example fails without a GPU 😬), could you confirm the example still works with my changes and confirm frames are really decoded by the hardware?

Cacsjep commented 9 months ago

looks good... we do decoding so you should change h264_nvenc to h264_cuvid decoderCodecName = flag.String("c", "", "the decoder codec name (e.g. h264_cuvid)")

PS C:\Development\go-astiav> go run .\examples\hardware_decoding\ -i .\testdata\video.mp4 -c h264_cuvid -n 0 -t cuda
2024/01/26 15:48:30 ffmpeg log: Opening '.\testdata\video.mp4' for reading (level: 48)
2024/01/26 15:48:30 ffmpeg log: Setting default whitelist 'file,crypto,data' (level: 48)
2024/01/26 15:48:30 ffmpeg log: Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100 (level: 48)
2024/01/26 15:48:30 ffmpeg log: ISO: File Type Major Brand: isom (level: 48)
2024/01/26 15:48:30 ffmpeg log: Unknown dref type 0x206c7275 size 12 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Processing st: 0, edit list 0 - media time: 0, duration: 61440 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Unknown dref type 0x206c7275 size 12 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Processing st: 1, edit list 0 - media time: 0, duration: 240672 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Before avformat_find_stream_info() pos: 380803 bytes read:36769 seeks:1 nb_streams:2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: nal_unit_type: 7(SPS), nal_ref_idc: 3 (level: 48)
2024/01/26 15:48:30 ffmpeg log: nal_unit_type: 8(PPS), nal_ref_idc: 3 (level: 48)
2024/01/26 15:48:30 ffmpeg log: nal_unit_type: 7(SPS), nal_ref_idc: 3 (level: 48)
2024/01/26 15:48:30 ffmpeg log: nal_unit_type: 8(PPS), nal_ref_idc: 3 (level: 48)
2024/01/26 15:48:30 ffmpeg log: nal_unit_type: 5(IDR), nal_ref_idc: 3 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Format yuv420p chosen by get_format(). (level: 48)
2024/01/26 15:48:30 ffmpeg log: Reinit context to 320x192, pix_fmt: yuv420p (level: 40)
2024/01/26 15:48:30 ffmpeg log: skip whole frame, skip left: 0 (level: 48)
2024/01/26 15:48:30 ffmpeg log: All info found (level: 48)
2024/01/26 15:48:30 ffmpeg log: After avformat_find_stream_info() pos: 658 bytes read:69537 seeks:2 frames:2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded lib: nvcuda.dll (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuInit (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDriverGetVersion (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDeviceGetCount (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDeviceGet (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDeviceGetAttribute (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDeviceGetName (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDeviceComputeCapability (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuCtxCreate_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuCtxGetCurrent (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuCtxSetLimit (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuCtxPushCurrent_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuCtxPopCurrent_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuCtxDestroy_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemAlloc_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemAllocPitch_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemAllocManaged (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemsetD8Async (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemFree_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemcpy (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemcpyAsync (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemcpy2D_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemcpy2DAsync_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemcpyHtoD_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemcpyHtoDAsync_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemcpyDtoH_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemcpyDtoHAsync_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemcpyDtoD_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMemcpyDtoDAsync_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuGetErrorName (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuGetErrorString (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuCtxGetDevice (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDevicePrimaryCtxRetain (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDevicePrimaryCtxRelease (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDevicePrimaryCtxSetFlags (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDevicePrimaryCtxGetState (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDevicePrimaryCtxReset (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuStreamCreate (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuStreamQuery (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuStreamSynchronize (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuStreamDestroy_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuStreamAddCallback (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuStreamWaitEvent (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuEventCreate (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuEventDestroy_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuEventSynchronize (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuEventQuery (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuEventRecord (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuLaunchKernel (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuLinkCreate (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuLinkAddData (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuLinkComplete (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuLinkDestroy (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuModuleLoadData (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuModuleUnload (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuModuleGetFunction (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuModuleGetGlobal (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuTexObjectCreate (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuTexObjectDestroy (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuGLGetDevices_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuGraphicsGLRegisterImage (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuGraphicsUnregisterResource (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuGraphicsMapResources (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuGraphicsUnmapResources (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuGraphicsSubResourceGetMappedArray (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuGraphicsResourceGetMappedPointer_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDeviceGetUuid (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDeviceGetUuid_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDeviceGetLuid (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDeviceGetByPCIBusId (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDeviceGetPCIBusId (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuImportExternalMemory (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDestroyExternalMemory (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuExternalMemoryGetMappedBuffer (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuExternalMemoryGetMappedMipmappedArray (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMipmappedArrayGetLevel (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuMipmappedArrayDestroy (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuImportExternalSemaphore (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuDestroyExternalSemaphore (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuSignalExternalSemaphoresAsync (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuWaitExternalSemaphoresAsync (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuArrayCreate_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuArray3DCreate_v2 (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuArrayDestroy (level: 48)
2024/01/26 15:48:30 ffmpeg log: Cannot load optional cuEGLStreamProducerConnect (level: 48)
2024/01/26 15:48:30 ffmpeg log: Cannot load optional cuEGLStreamProducerDisconnect (level: 48)
2024/01/26 15:48:30 ffmpeg log: Cannot load optional cuEGLStreamConsumerDisconnect (level: 48)
2024/01/26 15:48:30 ffmpeg log: Cannot load optional cuEGLStreamProducerPresentFrame (level: 48)
2024/01/26 15:48:30 ffmpeg log: Cannot load optional cuEGLStreamProducerReturnFrame (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuD3D11GetDevice (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuD3D11GetDevices (level: 48)
2024/01/26 15:48:30 ffmpeg log: Loaded sym: cuGraphicsD3D11RegisterResource (level: 48)
2024/01/26 15:48:31 ffmpeg log: Format cuda chosen by get_format(). (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded lib: nvcuvid.dll (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidGetDecoderCaps (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidCreateDecoder (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidDestroyDecoder (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidDecodePicture (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidGetDecodeStatus (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidReconfigureDecoder (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidMapVideoFrame64 (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidUnmapVideoFrame64 (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidCtxLockCreate (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidCtxLockDestroy (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidCtxLock (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidCtxUnlock (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidCreateVideoSource (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidCreateVideoSourceW (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidDestroyVideoSource (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidSetVideoSourceState (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidGetVideoSourceState (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidGetSourceVideoFormat (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidGetSourceAudioFormat (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidCreateVideoParser (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidParseVideoData (level: 48)
2024/01/26 15:48:31 ffmpeg log: Loaded sym: cuvidDestroyVideoParser (level: 48)
2024/01/26 15:48:31 ffmpeg log: CUVID capabilities for h264_cuvid: (level: 40)
2024/01/26 15:48:31 ffmpeg log: 8 bit: supported: 1, min_width: 48, max_width: 4096, min_height: 16, max_height: 4096 (level: 40)
2024/01/26 15:48:31 ffmpeg log: 10 bit: supported: 0, min_width: 0, max_width: 0, min_height: 0, max_height: 0 (level: 40)
2024/01/26 15:48:31 ffmpeg log: 12 bit: supported: 0, min_width: 0, max_width: 0, min_height: 0, max_height: 0 (level: 40)
2024/01/26 15:48:31 ffmpeg log: Invalid pkt_timebase, passing timestamps as-is. (level: 24)
2024/01/26 15:48:31 ffmpeg log: Format cuda chosen by get_format(). (level: 48)
2024/01/26 15:48:31 ffmpeg log: Formats: Original: cuda | HW: cuda | SW: nv12 (level: 40)
2024/01/26 15:48:31 ffmpeg log: CUDA texture alignment: 512 (level: 48)
2024/01/26 15:48:31 new frame: stream 0 - pts: 0 - transferred: true
2024/01/26 15:48:31 new frame: stream 0 - pts: 512 - transferred: true
asticode commented 9 months ago

Great thanks! I've updated the example's flag description 👍