search

scottlamb / moonfire-nvr

Moonfire NVR, a security camera network video recorder
Other
1.24k stars 137 forks source link

H.265 support #33

Open scottlamb opened 6 years ago

scottlamb commented 6 years ago

I think this is fairly simple to add (very similar to src/h264.rs) but there are a couple caveats:

It'd be nice to have though. My Dahua camera supports H.265, and I understand it's a significant improvement in video quality per byte.

scottlamb commented 5 years ago

Ahh, there's a draft of ISO/IEC 14496-15 here.

jlpoolen commented 4 years ago

Some points regarding High Efficiency Video Coding (HEVC), or spcifically H.264 vs. H.265:

Source: https://medium.com/advanced-computer-vision/h-264-vs-h-265-a-technical-comparison-when-will-h-265-dominate-the-market-26659303171a

See also: http://trac.ffmpeg.org/wiki/Encode/H.265

scottlamb commented 4 years ago

Yeah, those might be good considerations to add to a future "choosing your codec" section. I definitely don't want to drop H.264 support so folks would be able to choose (stream-by-stream) which they use. They could for example do video analytics on H.264 sub streams and have H.265 main streams that Moonfire NVR never decodes, only passes along to the client.

IIRC the Raspberry Pi 4 has hardware decoding support and a matching license for both H.264 and H.265. So I think neither the patent nor the decoding speed are necessarily a problem in some common scenarios. I'm not a lawyer and haven't sought legal advice on this, though, so take that with a grain of salt.

Also, on-NVR video analytics will always be optional. They're not even fully implemented yet (#30) (though I really want to fix that). And we can support on-camera video analytics (#29) as a CPU-friendlier alternative.

scottlamb commented 3 years ago

I'm demoting this from the 1.0? milestone to wishlist. Neither Chrome nor Firefox supports H.265 (https://caniuse.com/hevc). That means Moonfire NVR couldn't just store H.265 from the cameras and stream it unmodified to those browsers as it does for H.264. When using H.265, we'd have to do one of the following:

makoven commented 3 years ago

  1. Mozilla (and possibly google) will not support HEVC while it is encumbered by patents (link)

  2. Chips&Media Launches AV1 Video Encoder Hardware (link)

So it is possible that in a couple of years cheaper (since no royalties) cameras with AV1 support will appear :)

jlpoolen commented 2 years ago

Scott indicated that Reolink has upgraded their version of LIVE555 used as the rtsp servers in the Reolink cameras. The previous version was from 2013 and recent inquiry and discussion apparently caused Reolink to upgrade the server contained therein.

(I'll note I tried to reach out to the owner of the LIVE555 project about Reolink and my inquiries went unanswered and the maintainer refuses to allow posting from my gmail account despite the fact that I'm a developer with a major software company for 22 years. I'm too tired of trying to maintain my own mail server, it's not worth the time.)

Reolink currently is offering an 8 megapixel camera which send h.265 as its main stream. I posed a question about h.264 support and on October 8th a responder identified by Amazon as a "ReolinkDirect Seller " responded with:

  Hello, for Reolink 8MP cameras, the compression type of main-stream is H.265 while the sub-stream is H.264.

Given Scott's rapport with Reolink, I'd like to suggest he confirm with Reolink this design decision: h.265 for main and h.264 for sub streams. I have older Reolink cameras which have the LIVE555 2013 version and streams are really full of interruptions, in part caused by a possible network interface problem with my Raspberry Pi. Any rate, just to make the problem go away, I'd like to replace the problem cameras with the current offerings. Having cameras where I have them has proven invaluable: captures 2nd class felony theft from a contractor's truck, squatters, car thieves, and homeless people creating small warming fires and indecently exposing themselves. Just the presence of the cameras and the police advising trespassers that we have it all captured on video makes them wary of returning. Moonfire-nvr has come through capturing full streams where Reolink's capture-per-motion-detection-settings have missed a lot. All this is to say h.265 would be welcomed and it appears it's the direction camera manufacturers are heading. But, and that's a big one, h.265 is patented technology and I cannot opine about the legality of decoding. Approaching the licensing manager for h.265 might be something to consider to see what their position is about a project such as moonfire-nvr decoding streams that have been legitimately encoded --- maybe they only want to collect license fees from the encoding side?

scottlamb commented 2 years ago

I actually tried out one of Reolink's 8MP cameras for a bit. I wrote a bit about it on the wiki (just added a bit more). tl;dr: I recommend the 5MP models over the 8MP models. The 8MP has the H.265 problem, and it just doesn't have a large enough sensor for that many pixels anyway. I expect it to have worse images than the 5MP models at night.

All the other manufacturers I've seen have let the user choose between H.265 and H.264 in the camera's config UI. I hope Reolink starts doing the same.

But, and that's a big one, h.265 is patented technology and I cannot opine about the legality of decoding. Approaching the licensing manager for h.265 might be something to consider to see what their position is about a project such as moonfire-nvr decoding streams that have been legitimately encoded --- maybe they only want to collect license fees from the encoding side?

Even if they said "sure, open source projects can do whatever", there'd still be the technical problem. Other than Safari, browsers don't support H.265. Presumably this is also due to the licensing but it doesn't really matter why; they just don't. So we're stuck with the options I mentioned above.

MikePfaff commented 2 years ago

According to Reolink, their 8MP cameras will automatically switch from H.265 to H.264, if the stream resolution is lower than 8MP:

Yes, for the new version 8MP camera, if the resolution is higher than 8MP, the video format will switch to H.265. If it is less than 8MP, it will be H.264.

One user has confirmed it on his RLC-820A and Reolink posted it on a RLC-842A thread, which leads me to believe that this undocumented "lower stream resolution to get H.264" trick might also work on some (or even all) other Reolink 8MP (and higher MP?) camera models?

jlpoolen commented 1 year ago

The Reolink Duo is really two cameras in one unit where the feeds are from two sensors positioned such that they cover 180 degrees with no gaps. Their streams are stitched together into one stream which is 8MP and encoded using H.265. The really neat thing about this camera is that you just place it some place where you want 180 degree coverage, like near a straight sidewalk, and its done, you do not have to install two cameras (and most cameras do you support 90 degree viewing angles).

So the nature of the 8MP beast has changed with this shift of stitching together two 4 MP streams to create a panorama view. I have an extensive video surveillance system and one problem I have faced is what happens a bit further down the street when a person of interest goes beyond my camera's field of view. The 180 degree coverage is an important advancement.

jlpoolen commented 1 year ago

Here's an example of the day and night versions of the sub stream: firefox_2023-04-16_20-50-04

firefox_2023-04-16_20-48-12

scottlamb commented 1 year ago

Yeah, that stitching is nice for the situation. In terms of Moonfire support, there is another user interested in implementing H.265 in Retina, and from there it'd be pretty easy to support Moonfire recording H.265. But actually playing it back in a web UI is another matter. Browser support is still bad: https://caniuse.com/hevc

In theory we could also add support for realtime transcoding in Moonfire, e.g. decoding the H.265 and encoding as H.264, given adequate hardware. The Rockchip RK3588 should be able to do this for example, or some desktop-class GPUs, or add-in accelerator cards such as NETINTs. (Coincidentally, I was talking with them in person today. Their older "Logan" generation T432 cards are down to $300 iirc.)

jlpoolen commented 1 year ago

Several weeks ago I spent some time with Perl using the API and if I recall, I was able to easily pull segments from Moonfire. I'm probably the out-in-left-field use case, but having the H.265 files may be useful even though they cannot be played back on most web browsers at this time. I still have to compare the quality of the "main" feed at H.265 vs. "sub" feed at H.264 to see if there is an appreciable difference. With the 180 degree camera and the ability to see way beyond, having better resolution could be a real plus. I'll update here with my findings and examples supporting.

jlpoolen commented 1 year ago

Here's a study documenting the difference between the 4MP feed encoded with H.264 and the 8MP feed encoded with H.265: https://salemdata.net/jlp/reolink/h265/reolink_duo_h265_h264_comparison.html

I spoke with a police officer on an unrelated matter and he mentioned that many people share videos with them, but the lack of resolution practically makes the videos useless. So from law enforcement's perspective, the higher the resolution and ability to discern numbers and features, the better. It would be nice to be able to use Moonfire-nvr to preserve the H.265 feeds even though they cannot be displayed in the project's web interface. I'd use the existing H.264 content to review knowing that if I have a really important segment, I can retrieve a noticeably better quality stream to hand to law enforcement. But I live in a pretty high crime area... that's the price I pay for cheaper real estate.

scottlamb commented 1 month ago

The h265 branch has a version of this. Feedback welcome, if anyone cares to give it a spin. There's no schema change necessary; the db format was already codec-agnostic.