Is it possible to stream `IntArray` data captured by usb camera?

[eomgerm]

Hi, I'm working on implementing Live Streaming App using external USB camera. My camera(OAK-1) doesn't support UVC, so I used external library called depthai. I finally retrieved IntArray data from camera, now I need to stream data using RTSP.

Is any examples streaming IntArray data using this library? Thanks.

[pedroSG94]

Hello,

First of all, you need know the exact format of the IntArray (ARGB, RGB, YUV, raw h264, etc)

After that, if you are using raw h264 you can stream it directly using RtspClient like in this post: https://github.com/pedroSG94/RootEncoder/issues/1033#issuecomment-1008749330 If you are using other format not encoded you will need encode it and then use the above example.

[eomgerm]

@pedroSG94 Is it impossible to send H265 Data? My camera only supports H265

[pedroSG94]

Yes, you can do it. The code is exactly the same that with h264 but in your case you need change the way you detect if the frame is a keyframe and the way you extract sps, pps and vps. This is the way to detect a keyframe with h265: https://github.com/pedroSG94/RootEncoder/blob/master/library/src/main/java/com/pedro/library/base/recording/BaseRecordController.java#L87 This is the way to get sps, pps and vps: https://github.com/pedroSG94/RootEncoder/blob/master/encoder/src/main/java/com/pedro/encoder/video/VideoEncoder.java#L406

[eomgerm]

Thanks! I realized that my camera supports both h264 and h265. Now I'm working on getting H264/265 packet from my cam. If I get any update I'll let you know.

[eomgerm]

@pedroSG94 Hello, I finally got H264 Frame(ByteArray) from camera. As you mentioned, I extracted NALU type, but the type was always 9

Below is the first 32bytes(Integer value) of the frame I got. The first 5th byte (I think this is the NALU type) is always 9.

Is any other way that extracts sps, pps automatically?

[pedroSG94]

Hello,

Your case is a bit special (not common). All your frames start with Access unit delimiter. That nalu must be discarded. You have a table with all nalu type here: https://yumichan.net/video-processing/video-compression/introduction-to-h264-nal-unit/ In you case you should discard the first 6 bytes (h264 header, naltu type and byte of Access unit delimiter)

After that, the next nalu type is 33 or 39 (1 or 7 nalu type). 1 is a normal frame (no idr or no keyframe) and 7 is a SPS. You will need wait for a PPS and a keyframe to start (sometimes PPS and SPS could be on start of a keyframe bytearray).