Closed oiramario closed 5 years ago
Hi @oiramario
For the RGB sensor, the native format is YUYV (fourcc 'YUY2').
The D415 also offers lower quality RGB stream from the left infrared camera. This stream is available at up to 720p and suffers from projector interference. On the bright side, it is pixel-aligned with depth and projector pattern can be mostly removed with a special preset. For this stream the format is UYVY.
If I understand the problem correctly, you are getting 0 complete frames at 1080p from V4L2 at all, right? Is this only when depth streaming as well, or also when streaming color alone?
I do believe this capability was functional on the Up-Board. If you'd like, I can double-check, but don't have one next to me right now.
Something I think might be relevant - LKML Asynchronous UVC
To rule out librealsense, you could test the system using something like yavta
In general, the Incomplete video frame detected!
warning originates in the uvc_video
Kernel Module (it is a flag on the buffer object if I recall correctly). However, if we forgot a thread doing while(true);
somewhere in librealsense it would certainly be a problem :)
Otherwise, librealsense should add very little overhead when configured as you described.
@ev-mp
Hi @dorodnic
I tried to set RS2_FORMAT_UYVY, but librealsense returned an error:
terminate called after throwing an instance of 'rs2::error'
what(): Couldn't resolve requests
I'm getting few frames at 1080p when streaming color alone, depth stream not test yet. I think there should be no problem with the depth of the 720P format Z16.
10/01 14:43:23,244 DEBUG [547323199920] (archive.cpp:287) CallbackFinished,Color,5,DispatchedAt,1547131403244.034912
10/01 14:43:23,244 DEBUG [547323199920] (archive.cpp:294) Frame Callback [Color#5] overdue. (Duration: 110.381592ms, FPS: 30, Max Duration: 32ms)
10/01 14:43:23,244 DEBUG [547323199920] (archive.cpp:501) CallbackStarted,Color,6,DispatchedAt,1547131403244.155273
10/01 14:43:23,244 DEBUG [547323199920] (archive.cpp:287) CallbackFinished,Color,0,DispatchedAt,1547131403244.255859
10/01 14:43:23,244 DEBUG [547323199920] (archive.cpp:287) CallbackFinished,Color,0,DispatchedAt,1547131403244.338379
10/01 14:43:23,244 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 967432 out of 4147455 bytes (23%)
10/01 14:43:23,250 DEBUG [547314807216] (archive.cpp:287) CallbackFinished,Color,0,DispatchedAt,1547131403250.352783
10/01 14:43:23,250 DEBUG [547314807216] (archive.cpp:294) Frame Callback [Color#0] overdue. (Duration: 1547131403250.352783ms, FPS: 30, Max Duration: 32ms)
10/01 14:43:23,288 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 2966280 out of 4147455 bytes (71%)
10/01 14:43:23,302 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:23,336 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:23,370 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:23,403 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:23,436 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:23,470 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:23,504 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:23,537 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:23,571 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:23,604 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:23,638 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:23,672 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:23,713 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 1196808 out of 4147455 bytes (28%)
10/01 14:43:23,739 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:23,775 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 639752 out of 4147455 bytes (15%)
10/01 14:43:23,815 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 1426184 out of 4147455 bytes (34%)
10/01 14:43:23,838 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:23,872 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:23,910 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 770824 out of 4147455 bytes (18%)
10/01 14:43:23,941 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:23,974 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:24,008 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:24,041 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:24,104 DEBUG [547323199920] (sensor.cpp:450) FrameAccepted,Color,7,0,7,Arrived,1547131404071.235107 1547131404103.760986 diff - 32.525879 ,TS,1547131404103.776611,TS_Domain,System Time last_frame_number 6 last_timestamp 1547131403230.192871
10/01 14:43:24,107 DEBUG [547323199920] (archive.cpp:501) CallbackStarted,Color,7,DispatchedAt,1547131404107.559082
10/01 14:43:24,108 DEBUG [547323199920] (sync.cpp:143) DISPATCH (TS: )--> Color 7 1547131404103.776611
10/01 14:43:24,108 DEBUG [547323199920] (sync.cpp:551) clean inactive stream in (TS: )Depth Infrared Infrared Color
10/01 14:43:24,108 DEBUG [547323199920] (sync.cpp:551) clean inactive stream in (TS: )Depth Infrared Infrared Color
10/01 14:43:24,108 DEBUG [547323199920] (sync.cpp:551) clean inactive stream in (TS: )Depth Infrared Infrared Color
10/01 14:43:24,108 DEBUG [547323199920] (sync.cpp:551) clean inactive stream in (TS: )Depth Infrared Infrared Color
10/01 14:43:24,108 DEBUG [547323199920] (metadata-parser.h:153) Metadata mismatch - actual: 0x0, expected: 0x80000005 (Intel RGB Control)
10/01 14:43:24,108 DEBUG [547323199920] (sync.cpp:143) DISPATCH (TS: I Depth I Infrared I Infrared I Color )--> Color 7 1547131404103.776611
10/01 14:43:24,108 DEBUG [547323199920] (sync.cpp:551) clean inactive stream in (TS: I Depth I Infrared I Infrared I Color )Color
10/01 14:43:24,109 DEBUG [547323199920] (metadata-parser.h:153) Metadata mismatch - actual: 0x0, expected: 0x80000005 (Intel RGB Control)
10/01 14:43:24,109 DEBUG [547323199920] (sync.cpp:99) I Color--> Color 7, 1547131404103.776611
10/01 14:43:24,109 DEBUG [547323199920] (sync.cpp:258) SYNC (TS: I Depth I Infrared I Infrared I Color )--> Color 7 1547131404103.776611
10/01 14:43:24,109 DEBUG [547323199920] (metadata-parser.h:153) Metadata mismatch - actual: 0x0, expected: 0x80000005 (Intel RGB Control)
10/01 14:43:24,109 DEBUG [547323199920] (sync.cpp:516) fps 1 Color 7 1547131404103.776611
10/01 14:43:24,109 DEBUG [547323199920] (sync.cpp:529) (TS: I Depth I Infrared I Infrared I Color )Color 7 1547131404103.776611 fps 1 gap 1000.000000 next_expected: 1547131405103.776611
10/01 14:43:24,109 DEBUG [547323199920] (sync.cpp:258) SYNC (TS: )--> Color 7 1547131404103.776611
10/01 14:43:24,110 DEBUG [547323199920] (metadata-parser.h:153) Metadata mismatch - actual: 0x0, expected: 0x80000005 (Intel RGB Control)
10/01 14:43:24,110 DEBUG [547323199920] (sync.cpp:516) fps 1 Color 7 1547131404103.776611
10/01 14:43:24,110 DEBUG [547323199920] (sync.cpp:529) (TS: )Color 7 1547131404103.776611 fps 1 gap 1000.000000 next_expected: 1547131405103.776611
10/01 14:43:24,110 DEBUG [547323199920] (archive.cpp:287) CallbackFinished,Color,0,DispatchedAt,1547131404110.459229
10/01 14:43:24,110 DEBUG [547323199920] (archive.cpp:294) Frame Callback [Color#0] overdue. (Duration: 1547131404110.459229ms, FPS: 30, Max Duration: 32ms)
10/01 14:43:24,110 DEBUG [547323199920] (syncer-processing-block.cpp:28) SYNCED: Color 7, 1547131404103.776611
10/01 14:43:24,110 DEBUG [547323199920] (archive.cpp:501) CallbackStarted,Color,7,DispatchedAt,1547131404110.847168
10/01 14:43:24,111 DEBUG [547323199920] (archive.cpp:287) CallbackFinished,Color,6,DispatchedAt,1547131404111.008789
10/01 14:43:24,111 DEBUG [547323199920] (archive.cpp:294) Frame Callback [Color#6] overdue. (Duration: 868.772949ms, FPS: 30, Max Duration: 32ms)
10/01 14:43:24,111 DEBUG [547323199920] (archive.cpp:501) CallbackStarted,Color,7,DispatchedAt,1547131404111.299072
10/01 14:43:24,111 DEBUG [547323199920] (archive.cpp:287) CallbackFinished,Color,0,DispatchedAt,1547131404111.460205
10/01 14:43:24,111 DEBUG [547323199920] (archive.cpp:287) CallbackFinished,Color,0,DispatchedAt,1547131404111.592285
10/01 14:43:24,111 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:24,117 DEBUG [547314807216] (archive.cpp:287) CallbackFinished,Color,0,DispatchedAt,1547131404117.408447
10/01 14:43:24,117 DEBUG [547314807216] (archive.cpp:294) Frame Callback [Color#0] overdue. (Duration: 1547131404117.408447ms, FPS: 30, Max Duration: 32ms)
10/01 14:43:24,142 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:24,176 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,209 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,242 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,278 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,313 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,346 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,380 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:24,413 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,446 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,481 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,514 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,548 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,581 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,615 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,649 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,683 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,713 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:24,749 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 377608 out of 4147455 bytes (9%)
10/01 14:43:24,785 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 443144 out of 4147455 bytes (10%)
10/01 14:43:24,823 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 1458952 out of 4147455 bytes (35%)
10/01 14:43:24,849 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,883 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:24,916 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:24,951 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:24,989 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 1131272 out of 4147455 bytes (27%)
10/01 14:43:25,015 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:25,048 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:25,082 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,118 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,151 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,185 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,218 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,252 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,286 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,320 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,353 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,387 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,420 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,454 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,487 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,521 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,554 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,588 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,622 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,655 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,689 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,723 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,756 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,789 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,824 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,856 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 410376 out of 4147455 bytes (9%)
10/01 14:43:25,891 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:25,922 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:25,955 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:25,990 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 377608 out of 4147455 bytes (9%)
10/01 14:43:26,025 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:26,059 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,098 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 1327880 out of 4147455 bytes (32%)
10/01 14:43:26,126 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:26,158 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:26,190 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:26,227 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,258 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:26,293 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,325 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 246536 out of 4147455 bytes (5%)
10/01 14:43:26,361 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:26,395 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,428 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 181000 out of 4147455 bytes (4%)
10/01 14:43:26,462 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,495 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,529 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,563 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,596 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,630 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,663 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,697 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,730 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,764 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 213768 out of 4147455 bytes (5%)
10/01 14:43:26,796 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:26,830 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:26,864 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:26,897 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:26,931 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
^Cselect err: Interrupted system call
10/01 14:43:26,964 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:26,998 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:27,032 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 279304 out of 4147455 bytes (6%)
10/01 14:43:27,082 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 2409224 out of 4147455 bytes (58%)
10/01 14:43:27,099 INFO [547323199920] (types.h:936) Incomplete video frame detected!
Size 443144 out of 4147455 bytes (10%)
10/01 14:43:27,126 INFO [547323199920] (backend-v4l2.cpp:923) Stream finished
10/01 14:43:27,134 INFO [547636520624] (archive.cpp:354) The user was holding on to 1 frames after stream 0x0x2e6dd2d0 stopped
10/01 14:43:27,136 INFO [547636520624] (backend-v4l2.cpp:1473) ~v4l_uvc_meta_device
10/01 14:43:27,136 INFO [547636520624] (backend-v4l2.cpp:1473) ~v4l_uvc_meta_device
10/01 14:43:27,138 INFO [547636520624] (backend-v4l2.cpp:1473) ~v4l_uvc_meta_device
10/01 14:43:27,139 DEBUG [547636520624] (archive.cpp:287) CallbackFinished,Color,7,DispatchedAt,1547131407139.421631
10/01 14:43:27,139 DEBUG [547636520624] (archive.cpp:294) Frame Callback [Color#7] overdue. (Duration: 3031.862549ms, FPS: 30, Max Duration: 32ms)
10/01 14:43:27,139 INFO [547636520624] (archive.cpp:364) All frames from stream 0x0x2e6dd2d0 are now released by the user
If you can, I recommend that you check it out on Up-Board.
I understand what you mean, the problem should be in the UVC protocol here, after all, ZED supports 4K resolution. I will try to read the references you give, but I think this is really important, it is best done by your development team.
[Please Ignore - RealSense system comment]
Patch link not synced just now. Here is the link: https://lkml.org/lkml/2018/1/3/699
Not yet. I'll try the patch next week.
@Oiramario Good. Looking forward to your update!
Hi guys, in this question, the rockchip linux kernel I used has already hit the patch you mentioned. See these below: FROMLIST: media: uvcvideo: Protect queue internals with helper FROMLIST: media: uvcvideo: queue: Support asynchronous buffer handling
Then what's your suggestions?
@Oiramario Please try to add below two patches.
1. modify_urb_number_and_uvc_packet.patch --- a/drivers/media/usb/uvc/uvcvideo.h +++ b/drivers/media/usb/uvc/uvcvideo.h @@ -172,9 +172,9 @@
/ Number of isochronous URBs. / -#define UVC_URBS 5 +#define UVC_URBS 32 / Maximum number of packets per URB. / -#define UVC_MAX_PACKETS 32 +#define UVC_MAX_PACKETS 16 / Maximum status buffer size in bytes of interrupt URB. /
2. rk3399-irqchip-xhci-irq-142-to-all-cpu.patch
--- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -421,6 +421,12 @@ static void __init gic_dist_init(void) affinity = gic_mpidr_to_affinity(cpu_logical_map(smp_processor_id())); for (i = 32; i < gic_data.irq_nr; i++) gic_write_irouter(affinity, base + GICD_IROUTER + i * 8); +
/ irq 142 routed to all cpus /
affinity = affinity | 0x80000000;
gic_write_irouter(affinity, base + GICD_IROUTER + 142 * 8);
/ Decrease irq 142 priority value from default 0x0a to 0x09 /
writel_relaxed(0xa090a0a0, base + GICD_IPRIORITYR + rounddown(142, 4)); }
static int gic_populate_rdist(void) @@ -658,7 +664,8 @@ static int gic_set_affinity(struct irq_data d, const struct cpumask mask_val, reg = gic_dist_base(d) + GICD_IROUTER + (gic_irq(d) * 8); val = gic_mpidr_to_affinity(cpu_logical_map(cpu));
gic_write_irouter(val, reg);
if (gic_irq(d) != 142)
gic_write_irouter(val, reg);
/*
I'll check this out.
XiangyuX Sun, Jan 28, 10:04 PM
@oiramario Any good news? Looking forward to your update. Thanks!
@oiramario Can the patch work for you? Looking forward to your result. Thanks!
@oiramario Does it works? Please update. Thanks!
@RealSenseCustomerSupport I just came back from vacation and I will check this out next week.
@oiramario Any update? Thanks!
Sorry, the confirmation of the patch is delayed because of some other work. There is a problem with the patch content you posted, and it cannot be used directly after copying it. Referring to the code, I re-created the patch and successfully patched it into the kernel. However, I found that in the kernel v4.4.143 used by rockchip, the macro UVC_MAX_STATUS_SIZE is 16, and 32 of the content you sent, I am not very clear about this part, I hope you can help confirm it. Next week I will confirm the modified kernel on the board of rockchip 3399, thank you.
modify_urb_number_and_uvc_packet.txt rk3399-irqchip-xhci-irq-142-to-all-cpu.txt
@oiramario The difference I think is due to different kernel version. You can have a try with your setting of UVC_MAX_STATUS_SIZE
@oiramario How's it going? Any update? Thanks!
@oiramario Any good news?
@oiramario Now unroot android solution is included in 2.19 librealsense. No root needed and no patch needed. Please have a try to see if this can fix your issue. Thanks! https://github.com/IntelRealSense/librealsense/tree/master/wrappers/android
@oiramario , sorry to hijack this thread... I'm trying to do what you seem to have succeeded in doing, i.e getting a RealSense device to work on a NanoPI M4. Please let me know if you would be happy to outline the basic steps that are required... I have successfully build the realsense libs/tools and I can grab images from some of the /dev/video* devices created, but when trying the rs-enumerate-devices script I'm just getting "No device detected. Is it plugged in?"... I guess I have to patch and rebuild a custom kernel?
@landersson What's OS you getting the issue, android or linux?
I'm using the the standard Ubuntu 18.04 based "FriendlyDesktop" image for NanoPI M4. Default kernel 4.4.143. Basically the same setup as oiramario initially reported.
@oiramario Sorry, my mistake. I've thought the issue was found on Android. Will check this issue on Ubuntu and update you later.
@oiramario , sorry to hijack this thread... I'm trying to do what you seem to have succeeded in doing, i.e getting a RealSense device to work on a NanoPI M4. Please let me know if you would be happy to outline the basic steps that are required... I have successfully build the realsense libs/tools and I can grab images from some of the /dev/video* devices created, but when trying the rs-enumerate-devices script I'm just getting "No device detected. Is it plugged in?"... I guess I have to patch and rebuild a custom kernel?
Follow these steps:
1. apt-get install git libssl-dev libusb-1.0-0-dev pkg-config
2. git clone --depth 1 https://github.com/IntelRealSense/librealsense.git
3. cd librealsense
4. sudo cp config/99-realsense-libusb.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules && udevadm trigger
5. mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_WITH_TM2=false -DBUILD_GRAPHICAL_EXAMPLES=false -DBUILD_EXAMPLES=false
make -j$(nproc) && make install
6. rs-enumerate-devices
More detail: https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md
Nanopi-m4 uses kernel 4.4, so we don't need to do anything like patch or rebuild custom kernel.
@oiramario Any good news?
Sorry, I have been stuck in some kernel-related issues recently, but I will take time to verify this issue as soon as possible.
Thanks @oiramario, those are pretty much exactly the steps I took too, but I'm still getting "No device detected..." when running rs-enumerate-devices. Running as sudo doesn't help.
My camera is a 435i, so I guess that must be why we're getting different results. Apart from the camera model, our setups are nearly identical (except you have 4G RAM while I have 2).
@oiramario , did you connect the depth camera via some kind of powered USB hub, or straight to the Nano PI M4?
I'm thinking perhaps the NanoPI can't provide enough power via USB to run the D435i...
I didn't have any USB hub here, just plugged D415 into NanoPi-M4. Could you post some logs or shots?
Ok, I tried with a powered USB3 hub, but that didn't change anything. I've attached an strace of rs-enumerate-devices, the output of "lsusb -vvv" and the end of dmesg... Let me know if there's anything else that can help you figure out why this doesn't work. Thanks.
@oiramario Did you get chance to try again with the kernel patch?
@landersson How many USB ports on your NanoPI? Did you try them all? Seen from the result of lsusb, the camera has been connected and recognized and rs-enumerate should work. Please double check if "sudo cp config/99-realsense-libusb.rules /etc/udev/rules.d/ " executed successfully.
The Nano M4 have 4 USB3 ports.. I've tried them all as well as connecting the D435i using an external, powered USB3 hub.The udev script has been properly installed.
Just took some time to look further into this... With logging enabled, I see this message:
22/03 03:19:33,133 WARNING [547723022352] (ds5-factory.cpp:617) DS5 group_devices is empty.
Looking at src/ds5/ds5-factory.cpp:591, it looks like my D435i is disabled because it's a HID device, but no HID sensors were detected. I assume the HID sensors (gyro and accelerometer) would be detected if with a patched kernel, so I guess I'll just have to go ahead and rebuild the kernel.
@oiramario, if you have any pointers in regards to rebuilding the kernel for the NanoPI M4, that would be great.
@landersson What's the FW version of your D435i? Could you please try to connect it to one PC to see if it's 5.10.13+ and if it can work normally? Please be noted that "D435i requires FW version 5.10.13+ and is currently supported on Linux and Windows 10 only. "
@landersson If need HID related pacth, please refer to https://github.com/IntelRealSense/librealsense/tree/master/scripts
Please select the patch accordingly such as https://github.com/IntelRealSense/librealsense/blob/master/scripts/realsense-hid-ubuntu-bionic-master.patch
realsense-viewer on my Ubuntu 18.04 PC says: Firmware Version: 05.10.13.00
In order to end up with a kernel that is going to work well on the NanoPI M4 SOC, I assume I will have to use a custom kernel that have proper support for this board, such as:
https://github.com/friendlyarm/kernel-rockchip
This is based on Linux 4.4, so I guess I have to apply the realsense patches to this kernel manually.
@landersson Yes, you're right. You need apply the patch manually. Any update please let us know. Thanks!
@landersson Any update? Thanks!
Hi, I just applied the patches to the standard NanoPI M4 RockChip kernel (4.4.154)... I had to fix up a few of the changes manually. After booting with the patched kernel, rs-enumerate-devices finds my D435i and realsense-viewer works pretty well too. I do get a lot of "Incomplete video frame detected" warnings if I try to use the depth stream in all but the lowest resolutions/frame rates. realsense-viewer also seem to be using up a fair bit of CPU even when no streams are enabled.
I haven't had time to investigate and reduce the number of incomplete frames... any suggestions are welcome. Color seem to be able to run as much higher resolutions without getting incomplete frames... Is the color stream being transferred as compressed video, while depth is uncompressed? Or is it more likely that CPU bottlenecks are causing incomplete depth frames?
@landersson Glad to see the progress! What's the resolution and fps of depth when you see the incomplete video frame detected? Any other streaming at the same time when you get incomplete frames?
@oiramario Any progress from your side? Thanks!
@landersson @oiramario Any update? Thanks!
Regardless of whether the color stream is running or not, the depth stream in only running with out incomplete frames in the lowest resolution.
I have run out of time to work more on this for the moment, but we will probably get back to this at a later stage. As far as I'm concerned, please feel free to close this issue. I will open up a new issue related to NanoPI M4 depth stream issues later on if I'm unable to resolve the incomplete frame problems myself.
@oiramario If no other questions, will close the ticket accordingly. Thanks!
@RealSenseCustomerSupport I'm back! I'll into this next week and feed back to you.
Issue Description
rs-enumerate-devices
I plug D415 into USB3.0. Using GLESv2 + GBM + DRM to show color stream with Zero-Copy. For the purpose of reducing the transmission bandwidth, I set the format RS2_FORMAT_YUYV to display the color stream.
When set resolution to 1280 x 720 it can running fast with 30fps. However, when the resolution is set to 1920 x 1080, the log shows a lot of Incomplete video frame detected. The problem occurred after the startup program, the entire motherboard and CPU did not overheat.
I have reviewed the description of such a problem in the forum, pointing out that the problem is due to insufficient usb capacity, or the CPU load is too high. However, the official recommendation of intel to run the D400 device is AAEON UP Board (see link below), its ability is equivalent to RK3399, so I don't think it is the reason for hardware capabilities. https://click.intel.com/intelr-realsensetm-depth-camera-d415-aaeonr-up-board.html
Also, what is the native format of the D415 color stream? Is it YUYV or UYVY? If it is because of the lack of computing power due to format conversion, or the problem of USB transmission bandwidth, I can directly use the native format and then quickly convert through GPU. Please give your suggestion.
Code attached below:
The log file shows that: