codecov[bot]
commented
1 year ago Codecov Report
Patch coverage has no change and project coverage change: +0.23 :tada:
Comparison is base (
76ba048) 58.08% compared to head (2964ae2) 58.32%.
Additional details and impacted files
```diff @@ Coverage Diff @@ ## master #470 +/- ## ========================================== + Coverage 58.08% 58.32% +0.23% ========================================== Files 62 62 Lines 3691 3736 +45 ========================================== + Hits 2144 2179 +35 - Misses 1420 1430 +10 Partials 127 127 ``` | [Impacted Files](https://codecov.io/gh/pion/mediadevices/pull/470?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion) | Coverage Δ | | |---|---|---| | [pkg/driver/camera/camera\_linux.go](https://codecov.io/gh/pion/mediadevices/pull/470?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-cGtnL2RyaXZlci9jYW1lcmEvY2FtZXJhX2xpbnV4Lmdv) | `28.57% <0.00%> (-1.87%)` | :arrow_down: | | [pkg/prop/prop.go](https://codecov.io/gh/pion/mediadevices/pull/470?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-cGtnL3Byb3AvcHJvcC5nbw==) | `74.00% <ø> (+1.33%)` | :arrow_up: | | [pkg/codec/vpx/vpx.go](https://codecov.io/gh/pion/mediadevices/pull/470?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-cGtnL2NvZGVjL3ZweC92cHguZ28=) | `83.59% <0.00%> (+0.08%)` | :arrow_up: | | [pkg/codec/x264/x264.go](https://codecov.io/gh/pion/mediadevices/pull/470?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-cGtnL2NvZGVjL3gyNjQveDI2NC5nbw==) | `65.51% <0.00%> (+0.40%)` | :arrow_up: | | [pkg/codec/openh264/openh264.go](https://codecov.io/gh/pion/mediadevices/pull/470?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-cGtnL2NvZGVjL29wZW5oMjY0L29wZW5oMjY0Lmdv) | `81.69% <0.00%> (+0.53%)` | :arrow_up: | | [pkg/io/video/convert\_cgo.go](https://codecov.io/gh/pion/mediadevices/pull/470?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-cGtnL2lvL3ZpZGVvL2NvbnZlcnRfY2dvLmdv) | `71.23% <0.00%> (+2.57%)` | :arrow_up: | | [pkg/io/video/convert.go](https://codecov.io/gh/pion/mediadevices/pull/470?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-cGtnL2lvL3ZpZGVvL2NvbnZlcnQuZ28=) | `72.80% <0.00%> (+6.13%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion):umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
Problem Description
This library sets a buffer size of 1 upon opening cameras in Linux. This will always result in stale frames. For streaming this isn't a big deal since the staleness of a frame could be as little as about 30ms assuming a camera is capable of streaming at 30fps or even less for higher frame rates.
For taking snapshots this is much more problematic. Consider the following use case: a user wants to take a snapshot once a day at noon. Every image would be a day behind. That is, Monday's snapshot would be from Sunday, Sunday's snapshot would be from Saturday, and so forth.
Simply making an extra call in the application wrapping this one is difficult since
I do not think it’s possible to add unbuffered support to blackjack/webcam either. According to the V4L spec (link) there are three ways to read or stream frames from cameras:
Method 2 is what blackjack uses currently. Both methods 2 and 3 require the allocation of buffers. Method 1 doesn’t use buffers but it isn’t as widely supported as 2 and usually not supported for USB webcams. To further drive home this point, the following is what the V4L spec says about reading frames from cameras with read/write functions
In short, read/write is bad and isn’t widely supported. Streaming using mmap is good and is widely supported.
Solution
To fix this issue I made a simple assumption: if the user is reading frames at <1fps they are taking snapshots and not streaming. The solution then is to save a timestamp of the last time we read a frame from the camera. If it was more than a second ago, clear the buffer. If it was less than a second ago, it's business as usual. This adds virtually no overhead for streaming but does add a small amount for taking snapshots. Testing this locally with a C270 USB webcam resulted in <4ms of additional latency per snapshot. If you're taking snapshots every second or greater this extra overhead is negligible.
If there are better ways to fix this problem please let me know. I also plan to investigate ways to fix the same issue on Darwin which has a buffer size of 3.
Note: The example code for streaming MJPEG images in blackjack/webcam solves this issue by discarding the first frame it receives: the stale image. See their code here and here. That seems to be the workaround.