search

scottlamb / retina

High-level RTSP multimedia streaming library, in Rust
https://crates.io/crates/retina
Apache License 2.0
218 stars 46 forks source link

Add MJPEG support #88

Closed zanshi closed 2 months ago

zanshi commented 9 months ago

Hi!

I've added MJPEG support to the crate based on RFC 2435.

It didn't fit in perfectly with the existing architecture (would maybe be nicer to add a new ImageParameters struct), but I think it's good enough?

I've verified that it works on two camera models from FlexWatch and Parantek. It also works with the GStreamer RTSP server.

I fuzzed it over the night and it didn't find anything.

Appreciate feedback and ideas for improvements!

zanshi commented 6 months ago

Sorry for the late reply as well! Apparently my github notifications were turned off. I've made a few commits to fix the issues you pointed out.

zanshi commented 6 months ago

Apologies for taking so long to get back to you on this. It looks really nice! I have some minor comments below.

It didn't fit in perfectly with the existing architecture (would maybe be nicer to add a new ImageParameters struct), but I think it's good enough?

How would ImageParameters differ from VideoParameters? We can do an API revision if necessary, but fwiw I don't see anything too wrong with using VideoParameters.

Good point. It would probably just make the API more complicated. The idea to change rfc6381_codec to an Option sounds like a good idea.

scottlamb commented 6 months ago

What would you say to extending the example client to exercise this in some fashion—like writing a sequence of .jpg files? As I mentioned before, this all looks like very high-quality work. I'd like to just be able to give it a spin for myself, in part to way to check in following Retina changes if I've accidentally broken it...

zanshi commented 5 months ago

That sounds like a good idea. I'll try to get time to do that soon!

zanshi commented 5 months ago

I've added a test and an example. The example works pretty much like the mp4 one except that you specify an output directory to the JPEGs.

scottlamb commented 2 months ago

Finally getting through all the open PRs. Sorry yours has been waiting so long. Very nice work.

I've added a test and an example. The example works pretty much like the mp4 one except that you specify an output directory to the JPEGs.

Thank you! I don't think any of my cameras do MJPEG so I did a bit of hunting to come up with something to test against. I ended up with this Rube Goldberg machine:

  1. run mediamtx
  2. run ./client-record-format-mjpeg from gortsplib to copy packets from udp port 8000 to
  3. run gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080,format=I420 ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=9000

...and then found gstreamer's rtspclientsink to simplify:

  1. run mediamtx
  2. run gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080,format=I420 ! jpegenc ! rtspclientsink location=rtsp://localhost:8554/mystream

In the process I discovered what seems to be code for muxing jpegs to .mp4 format, in mediamtx's internal/playback/mp4/track.go. Might be interesting to make the mp4 example do this sometime.