[HOW-TO] low latency Network stream with 90deg rotation

[merasil]

Hi,

I have a rpi zero 2 w that I want to use as a surveillance camera. I bought a camera module 3 and already tried the example mjpeg server 2. This all worked fine, if it wasn't for the fact that I can't manage to output a 90 degree rotated image. I would like to film in portrait mode, because I can cover the important areas better... Google has brought absolutely nothing, or if only with the old raspivid stack... I can not imagine that I'm the only one with the problem. Does anyone have any ideas? The Stream has to be low latency, cause i am only intressted in Live Images.

[davidplowman]

I'm afraid that rotating by 90 degrees is difficult. How are you playing back or storing the videos? The easiest workaround is often to ask the playback software to rotate the video for you, or to set the "rotate" metadata flag if you are storing the videos.

[davidplowman]

Actually if you're using mjpeg_server_2.py, the following quick hack seems to work. It just splices an Exif section into the output file which requests a rotation, which browsers I've tried seem to support.

Add the following code near the top of the file:

import piexif

exif_bytes = piexif.dump({'0th': {piexif.ImageIFD.Orientation: 6}})
exif_len = len(exif_bytes) + 2
blob = bytes.fromhex('ffe1') + exif_len.to_bytes(2, 'big') + exif_bytes

(6 and 8 are the values for 90/270 degree rotation) and just before self.frame = buf add the line

            buf = buf[:2] + blob + buf[2:]

[merasil]

hi, thanks for the suggestions! i will try it later but i think that due to the fact that only the data transfer via HTTP works and i never see the image in the browser (i am grabbing the stream.mjpg), the exif settings will probably be ignored. My goal is to create a stream for two different systems. The first system is a server that uses the stream for face recognition. The rpi is too weak for this. The second system is a homebridge adapter that integrates the stream into homekit. The second system is why i need the images to be in correct rotation, cause the adapter cant do it for me :( I am still quite new in the rpi camera community and therefore do not know the best method to achieve these goals. The mjpeg server py examples were suitable for both. I can also try other outputs (ffmpeg?) than a HTTP server stream. Maybe a TCP listener or something similar will work? Do you have a tip for that? I had a look at libcamera and apparently a rotation by 90 should work with transpose and h/vflip. Is this not implemented in picamera2 yet or is it libcamera?

[davidplowman]

I can't say that I have any specific tips, I'm afraid. Generally I think we see things like rotations being handled more at the playing/receiving end these days, whereas in the past it was more common, I think, for the server/encoder actually to flip the images over. So mostly the "orientation" metadata is much better respected now than it used to be. But obviously things are likely to be a bit different with every different use case.

[merasil]

ok, but then again: it seems that libcamera can rotate the image. At least when I look at the C function "Transform". A transpose + vflip should rotate the image 90 degrees. So is it because of picamera2 which doesn't wrap/implement transpose (yet) or libcamera in general?

[davidplowman]

The libcamera implementation for the Raspberry Pi does not support transpose, I'm afraid.