search

Xilinx / video-sdk

https://xilinx.github.io/video-sdk
Other
32 stars 14 forks source link

XMA] WARNING: ffmpeg xmapluginlib Session id: 2, type: encoder. CU cmd is still pending. Cu might be stuck #47

Closed Mudpuppy12 closed 1 year ago

Mudpuppy12 commented 2 years ago

We had an interesting issue in which we could no longer process videos on the hardware. The messages had the following kernel errors listed below.

Any attempt to try a new encoding would fail. It required a reboot for the driver to be able to talk to the hardware again and process videos.

Any insights would be appreciated.

[ec2-user@ip-10-0-0-99 ~]$ xbutil examine System Configuration OS Name : Linux Release : 4.14.232-177.418.amzn2.x86_64 Version : #1 SMP Tue Jun 15 20:57:50 UTC 2021 Machine : x86_64 CPU Cores : 12 Memory : 22590 MB Distribution : Amazon Linux 2 GLIBC : 2.26 Model : vt1.3xlarge

XRT Version : 2.11.691 Branch : 2021.1 Hash : 3e695ed86d15164e36267fb83def6ff2aaecd758 Hash Date : 2021-11-19 02:16:32 XOCL : 2.11.691, 3e695ed86d15164e36267fb83def6ff2aaecd758 XCLMGMT : unknown, unknown

Devices present [0000:00:1f.0] : xilinx_u30_gen3x4_base_2 [0000:00:1e.0] : xilinx_u30_gen3x4_base_2

App log

worker7.log:2022-11-16 18:58:19 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 Starting cmd = convert_mp4_to_m3u8 worker7.log:2022-11-16 18:58:19 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'XRT build version: 2.11.691\n' worker7.log:2022-11-16 18:58:19 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'Build hash: 3e695ed86d15164e36267fb83def6ff2aaecd758\n' worker7.log:2022-11-16 18:58:20 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'Build date: 2021-11-19 02:16:32\n' worker7.log:2022-11-16 18:58:20 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'Git branch: 2021.1\n' worker7.log:2022-11-16 18:58:20 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'PID: 21250\n' worker7.log:2022-11-16 18:58:20 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'UID: 1000\n' worker7.log:2022-11-16 18:58:20 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[Wed Nov 16 18:58:19 2022 GMT]\n' worker7.log:2022-11-16 18:58:21 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'HOST: ip-10-0-0-99.ec2.internal\n' worker7.log:2022-11-16 18:58:21 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'EXE: /opt/xilinx/ffmpeg/bin/ffmpeg\n' worker7.log:2022-11-16 18:58:21 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] WARNING: ffmpeg xma-vcu-encoder device warning: !! The specified Level is too low and will be adjusted !!\r\n' worker7.log:2022-11-16 18:58:21 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'\n' worker7.log:2022-11-16 18:58:21 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] WARNING: ffmpeg xma-vcu-encoder device warning: !! The specified Level is too low and will be adjusted !!\r\n' worker7.log:2022-11-16 18:58:21 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'\n' worker7.log:2022-11-16 18:58:32 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] WARNING: ffmpeg xmapluginlib Session id: 2, type: encoder. CU cmd is still pending. Cu might be stuck\n' worker7.log:2022-11-16 18:58:32 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] ERROR: ffmpeg xma-vcu-encoder ERROR:: (xma_encoder_send_frame) frame_done signal not received from hw\n' worker7.log:2022-11-16 18:58:33 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'\n' worker7.log:2022-11-16 18:58:33 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[MPSOC VCU H264 encoder @ 0x27cf000] [error] encoder error: Error : mpsoc_vcu_encode_frame send raw data failed : ffmpeg pid 21250 on device index = 0 cu index = 38\n' worker7.log:2022-11-16 18:58:33 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[fatal] Video encoding failed\n' worker7.log:2022-11-16 18:58:53 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] ERROR: ffmpeg xmapluginlib xma_plg_buffer_write failed. dev_index: 0. xclSyncBO failed. Check device logs for more info. Error: -5\n' worker7.log:2022-11-16 18:58:53 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] ERROR: ffmpeg xma-vcu-encoder xclSyncBO failed -1 in prepare_sk_cmd \n' worker7.log:2022-11-16 18:58:53 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'\n' worker7.log:2022-11-16 18:58:53 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] ERROR: ffmpeg xma-vcu-encoder xma_encoder_close VCU_DEINIT failed\n' worker7.log:2022-11-16 18:58:53 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'\n' worker7.log:2022-11-16 18:58:54 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] ERROR: ffmpeg xmaencoder Error closing encoder plugin. Return code -1\n' worker7.log:2022-11-16 18:58:54 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'\n' worker7.log:2022-11-16 18:59:03 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] ERROR: ffmpeg xmapluginlib xma_plg_buffer_write failed. dev_index: 0. xclSyncBO failed. Check device logs for more info. Error: -5\n' worker7.log:2022-11-16 18:59:03 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] ERROR: ffmpeg xma-vcu-encoder xclSyncBO failed -1 in prepare_sk_cmd \n' worker7.log:2022-11-16 18:59:03 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'\n' worker7.log:2022-11-16 18:59:04 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] ERROR: ffmpeg xma-vcu-encoder xma_encoder_close VCU_DEINIT failed\n' worker7.log:2022-11-16 18:59:04 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'\n' worker7.log:2022-11-16 18:59:04 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA] ERROR: ffmpeg xmaencoder Error closing encoder plugin. Return code -1\n' worker7.log:2022-11-16 18:59:04 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'\n' worker7.log:2022-11-16 18:59:04 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA-Session-Stats] WARNING: Session id: 1, type: scaler still has unused completd cu cmds\n' worker7.log:2022-11-16 18:59:05 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 FFmpegCommand : b'[XMA-Session-Stats] WARNING: Session id: 3, type: encoder still has unused completd cu cmds\n' worker7.log:2022-11-16 18:59:05 INFO RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 ---------- Returned Code = 1 ------- for cmd = ffmpeg -loglevel repeat+level+error -nostdin -i /tmp/33456cb0-76cf-4091-a919-4d3c040b99c1@arbcbhy7/output.mp4 -filter_complex "multiscale_xma=outputs=2: out_1_width=852: out_1_height=480:out_1_rate=full: out_2_width=1280: out_2_height=720:out_2_rate=full [vid1][vid2];[0:1]asplit=outputs=2[aud1][aud2]" -map "[vid1]" -b:v 2000k -c:v mpsoc_vcu_h264 -map "[aud1]" -c:a aac -map "[vid2]" -b:v 6000k -c:v mpsoc_vcu_h264 -map "[aud2]" -c:a aac -f hls -hls_time 10 -hls_playlist_type event -master_pl_name master.m3u8 -var_streammap "v:1,a:1,name:720p v:0,a:0,name:480p" -y /tmp/33456cb0-76cf-4091-a919-4d3c040b99c1@arbcbhy7/stream%v.m3u8 worker7.log:2022-11-16 18:59:05 ERROR RequestId: 33456cb0-76cf-4091-a919-4d3c040b99c1 - Error in renderVideo - 'NoneType' object is not iterable

Messages

Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 804783000 Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :2/2/0 Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 802134000 Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800f07000 Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 804b97000 Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 802548000 Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800f0d000 Nov 16 19:00:02 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:00:02 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:00:57 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:00:57 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:02:33 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:02:33 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:02:47 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:02:47 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:02:51 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:02:51 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800005000 Nov 16 19:05:18 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:05:18 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:07:07 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:07:07 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:07:21 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:07:21 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:09:20 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:09:20 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:11:10 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:11:10 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:11:25 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:11:25 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:14:25 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:14:25 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:16:17 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:16:17 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:19:35 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:19:35 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:21:24 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:21:24 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:22:27 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 19:22:27 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800002000 Nov 16 19:28:17 ip-10-0-0-99 kernel: NMI watchdog: Perf event create on CPU 0 failed with -2 Nov 16 19:28:17 ip-10-0-0-99 kernel: acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM Nov 16 19:28:20 ip-10-0-0-99 kernel: xocl: module verification failed: signature and/or required key missing - tainting kernel Nov 16 19:28:32 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: cu.u.41943041 ffff8885b6694010 cu_probe: xocl_intc_cu_request failed, err: -16 Nov 16 19:28:32 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: cu.u.41943042 ffff8885b6694810 cu_probe: xocl_intc_cu_request failed, err: -16 Nov 16 19:28:32 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: cu.u.41943043 ffff8885b6694c10 cu_probe: xocl_intc_cu_request failed, err: -16 Nov 16 19:28:32 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: cu.u.41943044 ffff8885b6ec8010 cu_probe: xocl_intc_cu_request failed, err: -16 Nov 16 19:28:32 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: cu.u.41943045 ffff8885b67d9c10 cu_probe: xocl_intc_cu_request failed, err: -16 Nov 16 19:28:39 ip-10-0-0-99 kernel: xocl 0000:00:1e.0: cu.u.41943047 ffff8885b6692410 cu_probe: xocl_intc_cu_request failed, err: -16 Nov 16 19:28:39 ip-10-0-0-99 kernel: xocl 0000:00:1e.0: cu.u.41943048 ffff8885c723fc10 cu_probe: xocl_intc_cu_request failed, err: -16 Nov 16 19:28:39 ip-10-0-0-99 kernel: xocl 0000:00:1e.0: cu.u.41943049 ffff8885b6696810 cu_probe: xocl_intc_cu_request failed, err: -16 Nov 16 19:28:39 ip-10-0-0-99 kernel: xocl 0000:00:1e.0: cu.u.41943050 ffff8885c4db8010 cu_probe: xocl_intc_cu_request failed, err: -16 Nov 16 19:28:39 ip-10-0-0-99 kernel: xocl 0000:00:1e.0: cu.u.41943051 ffff8885b6eca010 cu_probe: xocl_intc_cu_request failed, err: -16

Mudpuppy12 commented 2 years ago

More details

--- messages --

Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl:xdma_process_requests: Request completion timeout Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl:xdma_process_requests: Request completion timeout Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 804783000 Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: 0, 0x43218c000 Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :2/2/0 Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 802134000 Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: 0, 0x5a04a1000 Nov 16 18:58:42 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: 1, 0x522434000 Nov 16 18:58:52 ip-10-0-0-99 kernel: xocl:xdma_process_requests: Request completion timeout Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800f07000 Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: 0, 0x2075e8000 Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl:xdma_process_requests: Request completion timeout Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 804b97000 Nov 16 18:58:53 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: 0, 0x544300000 Nov 16 18:58:54 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: ffff8885c81910a0 _kds_fini_client: Client pid(21215) has 1 opening context Nov 16 18:58:54 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: ffff8885c81910a0 kds_del_context: Client pid(21215) del context CU(0x0) Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl:xdma_process_requests: Request completion timeout Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 802548000 Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: 0, 0x470010000 Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl:xdma_process_requests: Request completion timeout Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl:xdma_xfer_submit: Request Processing failed, :1/1/0 Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: DMA failed, Dumping SG Page Table, ep addr 800f0d000 Nov 16 18:59:03 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: dma.xdma.u.5242881 ffff8885c48eb810 xdma_migrate_bo: 0, 0x547d06000 Nov 16 18:59:04 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: ffff8885c81910a0 _kds_fini_client: Client pid(21109) has 1 opening context Nov 16 18:59:04 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: ffff8885c81910a0 kds_del_context: Client pid(21109) del context CU(0x0) Nov 16 18:59:04 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: ffff8885c81910a0 _kds_fini_client: Client pid(21250) has 1 opening context Nov 16 18:59:04 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: ffff8885c81910a0 kds_del_context: Client pid(21250) del context CU(0x0) Nov 16 18:59:16 ip-10-0-0-99 dhclient[3037]: XMT: Solicit on eth0, interval 123460ms. Nov 16 18:59:52 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: ffff8885c81910a0 kds_add_context: Client pid(21326) add context CU(0xffffffff) shared(true) Nov 16 18:59:52 ip-10-0-0-99 kernel: xocl 0000:00:1f.0: ffff8885c81910a0 kds_del_context: Client pid(21326) del context CU(0xffffffff)

Mudpuppy12 commented 2 years ago

Full log for 11/16

https://www.dropbox.com/s/pjcsr34hchr4rxa/error2.log?dl=0

NastoohX commented 2 years ago

Hi, Thank you for bringing this to our attention. If you can provide responses to the following questions, it would be greatly appreciated: 1- Is this a recurring problem? 2- Were you able to ascertain that problem occurred immediately upon execution of certain FFmpeg command(s) or not? 3- Would you be able to share the FFmpeg command(s) that led to this event? Cheers,

Mudpuppy12 commented 2 years ago

Hello!

Fortunately, It is not a recurring problem. We re-ran the ffmpeg commands and it did not happen a 2nd time. I am thinking perhaps a kernel issue/hickup?

NastoohX commented 1 year ago

Hi, Thank you again for bringing this issue to our attention. We have opened an internal ticket to investigate this further. If any new information comes to light, it will be added here. Cheers,

NastoohX commented 1 year ago

Hi, As this issue now has an internal ticket, it will be closed, until further developments. Cheers,.