Closed YBonline closed 1 year ago
How many cameras are seeing objects / active at one time? This is a lot to fit in such a small space. It's possible there needs to be more attempts to find an appropriate layout based on the number of cameras that a user has.
Is there a way to tell that directly? Based off my historical memory, I feel like there was usually about 10 cameras on average showing motion. On rare occasions, it would occasionally get up to about 18 cameras at a time. Frequently 5 of them were panoramic as I tend to have those outside where wind blowing tree branches and such activated the motion detection.
My current last birdseye update did successfully lay out 6 cameras on the screen, but none of them were panoramic. I don't think since the update I've had more then 3 of the panoramic cameras appear at once, and I'm fairly certain there is more then that active whenever the updates aren't happening.
I'll also update tonight, usually at night I only have 2-4 with motion at a time (and almost always those are all panoramic)
To be clear motion doesn't matter since you have the objects mode defined for birdseye (unless they are seeing false positives, which is possible)
If you have that many panoramic cameras it may be worth changing the birdseye resolution / aspect ratio to be 32:9 instead of 16:9
I don't know if it would be considered a "false positive" or not but many of the cameras have parked cars in the view, and during motion, even if the cars don't move, they have always shown up on the birdseye view.
I believe I have 6 panoramic cameras currently, however they are all located outside and have roads in their field of view, so they are by far my most likely to have motion/objects. Many of my normal aspect ratio cameras are indoors, so they tend to never have false motion events.
The monitor I have setup to view the cameras full time is 16:9 so I would prefer to keep that aspect ratio, but I went ahead and tested with your recommended 32:9 ratio. The error rate seems to have gone way down and I'm getting frequent updates of the layout, but even at that ratio, there is still a fair number of that error occuring.
I also feel like it might be presenting the issue a bit more though. Here's a screenshot of the layout which seems less then ideal:
That's very odd, no idea how the layout would be like that. I'll have to see if I can recreate it which should hopefully give some insight into what's happening.
I've been continuing to experiment and tried disabling birdseye view on all the pano cameras, and I went a full 20 minutes now without errors and very sane looking layouts. I got up to 12 cameras active during the time in which they switch to nightmode (back on 16:10 for this testing).
I enabled just one pano and sure enough, it error'd super quick.
My other observation with my setup (unrelated to my original issue, but a result of me actually getting to use the new birdseye view) is I needless to say have a ton of cameras appearing and disappearing all the time. With this new birdseye view, the cameras all move around every single time one appears or disappears, which makes it much harder to keep track of whats going on in each camera if I'm looking at it. The old birdseye view only redid the full layout when the grid size changed. Realistically, in my use, during the day that usually meant it was I believe a 4x4 grid the entire day, and during motion events, cameras would (almost) never move on the screen during the events. Using this for just 10 minutes its much harder for me to keep track of what I'm looking at with the constant resorting/movement of cameras. Also, thinking out loud, does the order I have the cameras in my config file matter to the layout?
I know you put way more thought into this then me, but I'm curious if it might be less error prone and simpler, and easier to handle if it kept the grid layout from the previous version, but made certain cameras take up multiple spots on the grid based off their aspect ratio?
I've been continuing to experiment and tried disabling birdseye view on all the pano cameras, and I went a full 20 minutes now without errors and very sane looking layouts. I got up to 12 cameras active during the time in which they switch to nightmode (back on 16:10 for this testing).
I enabled just one pano and sure enough, it error'd super quick.
To be clear I worked on this birdseye revamp specifically because I have a pano camera and wanted it to work better with the layout. Having that many cameras is just not something I had in mind while making it, and I'll have to work on it some more to account for this larger use case.
My other observation with my setup (unrelated to my original issue, but a result of me actually getting to use the new birdseye view) is I needless to say have a ton of cameras appearing and disappearing all the time. With this new birdseye view, the cameras all move around every single time one appears or disappears, which makes it much harder to keep track of whats going on in each camera if I'm looking at it. The old birdseye view only redid the full layout when the grid size changed. Realistically, in my use, during the day that usually meant it was I believe a 4x4 grid the entire day, and during motion events, cameras would (almost) never move on the screen during the events. Using this for just 10 minutes its much harder for me to keep track of what I'm looking at with the constant resorting/movement of cameras. Also, thinking out loud, does the order I have the cameras in my config file matter to the layout?
This change for the birdseye view does not affect the changing of the cameras / that behavior in any way. The amount of time that a camera stays in the layout has not changed. The order of the cameras in the layout is alphabetical but can be configured using
birdseye:
order: X
where lower numbers show up first in the layout.
I know you put way more thought into this then me, but I'm curious if it might be less error prone and simpler, and easier to handle if it kept the grid layout from the previous version, but made certain cameras take up multiple spots on the grid based off their aspect ratio?
That is mostly the goal of this however this revamp also supports portrait aspect cameras which makes things more complicated.
Think of the scenario where you have 6 cameras (which would result in a 3x3 grid previously). If the first two cameras are pano cameras then you can't fit them both on the first row and one of the pano cameras needs to go to the second row. But that means there is a grid spot in the top right that is now not taken up by anything. The order feature means that re-ordering the cameras is not an option so now there is wasted space.
Okay I have just found the problem with your layout, 1920x536 is a non-standard aspect ratio of 240x67 which is why the layout was acting so weird. I remade the layout you showed above with standard aspect ratios and it worked just fine as expected
I just pushed up a PR to force cameras with non-standard aspect ratios into standard aspect ratios, this should fix what you are experiencing
Thank you so much for the super quick fix on the actual issue I reported! I'll test it in a few minutes, but on my other comment (which if you want me to make a new issue to discuss it, let me know, I know I drifted to another topic)
To be clear I worked on this birdseye revamp specifically because I have a pano camera and wanted it to work better with the layout. Having that many cameras is just not something I had in mind while making it, and I'll have to work on it some more to account for this larger use case.
I understand that and believe this is still an improvement, just looking for ways it could be even better as the birdseye view, at least from using it for a short amount of time, seems to be more "annoying" to watch a camera then before, but if its a pano camera, it is a HUGE improvement.
This change for the birdseye view does not affect the changing of the cameras / that behavior in any way. The amount of time that a camera stays in the layout has not changed.
I think it does, I think I just explained myself poorly, let me try with an example: Lets say the old grid method was going to have a 2x2 grid, lets call the positions: [1] [2] [3] [4] Lets have 5 cameras, in alphabetical order, A, B, C, D, E
We're going to have 3 sequential events scenarios.... (a), (b), and (c)
(a) Cameras A, B, C, D all have motion to start, no motion on E (b) Motion ends on B and C, no motion on E (c) Motion occurs on C and E, no motion on B
Under the old layout, we would have this sequence of events (a) 1=A 2=B 3=C 4=D (b) 1=A 2=blank 3=blank 4=D (c) 1=A 2=C 3=E 4=D
Under the new layout, lets assuming they're all square for simplicity so we still have the 2x2, we'd have: (a) 1=A 2=B 3=C 4=D (b) 1=A 2=D 3=blank 4=blank (c) 1=A 2=C 3=D 4=E
Now, lets imagine I'm sitting and watching the birdseye view. The events on A, B, C, and E all seem uninteresting to me, but D contains a suspicious event, so thats the only one I'm really tracking. Under the old layout, I keep my eyes camera D, which stays in position 4 the entire time, without touching anything Under the new layout, for me to track camera D, I start in position 4, then it moves to position 2, then it moves to position 3.
If you can imagine this with 28 cameras, cameras late on the list are going to be moving constantly during their motion event, which didn't happen under the old system. I'm not sure what the best answer is, but I hope this use case gets some though.
So I think two things got conflated. There has not been any change in the amount of time that a camera is shown based on activity (as it is configured, motion or object). Before and now a camera will be shown for 30 after the last activity.
However, in the previous implementation where the grid meant that birdseye did not need to care about camera resolutions; it would just replace or remove a camera based on when it expired without shuffling other cameras around (as you said above).
This is very complicated with the new layout because a newly added camera may not fit where the old camera was.
I have a few thoughts:
order
are the same then the list can not be sorted and that will result in layouts that are more stable. So I applied your fix and after 15 minutes, I have not had the originally reported bug (yay!), but I did want to share a screenshot since I think its still struggling with the layout with a large number of pano cameras:
Movement issues aside, about half the screen is blank, and it seems fairly obvious that better ordering could result in the videos being probably at least 30% bigger, I imagine I could easily get this layout into 3 full width rows that would probably consume 90% of my screen instead of less than 50% of it
No doubt, if you're following the ordering feature, its going to be really, really hard, but realistically, the old birdseye view didn't really honor the order if you weren't in continuous mode, and I personally find it hard to think of a good reason that order feature is at all useful when you're on a dynamic birdseye layout. The ordering feature seems to only make sense in the event that you are running birdseye in continuous mode. And if you really are running in continuous mode and care about the layout, the more "normal" configurable layout that is seen in most other NVR software probably makes more sense anyways (and is presumably easier to implement for those that want that instead of trying to make the Birdseye feature an all in one item)
No doubt, if you're following the ordering feature, its going to be really, really hard, but realistically, the old birdseye view didn't really honor the order if you weren't in continuous mode
The previous birdseye did not have an ordering feature. The ability to order cameras is new for 0.13 as well.
The ordering feature seems to only make sense in the event that you are running birdseye in continuous mode.
Have to say I disagree, I certainly want my main front camera shown first in the layout because that is the camera I care about most. Ordering it first makes it always show first so the layout is much more predictable.
The ordering feature was also added and implemented by another user due to requests from other users with a similar perspective.
Okay, I discovered a bug that was still causing issues, try the latest commit
Yup, I just went ahead and tried that and didn't seem to help too much
Tried putting together a similar setup to yours on the latest commit and this is what I see
The last screenshot was without the latest commit and with your suggestion of changing the height to 540, but this one is with that latest commit (and height still set to 540):
I am picking purposely when its doing especially bad layouts. When it first starts up and most cameras show, it looks more similar to yours:
That's where I get very confused, because using the same resolutions / ordering (for the first handful) I see this
what is your birdseye resolution and what are the resolutions of the different cameras in that first layout?
Aha! So when you had me change my birdseye layout to 32:9, when I switched it back, I decided to go to 16:10 (1920x1200) since thats what my monitor actually was, instead of the original 16:9 which was default. When I went back to 16:9 (1920x1080) it started doing a LOT better job. This was the worst I got over 3 minutes (instead of the prior examples):
Do you still want the individual camera resolutions or does that give you what you need?
Okay that makes sense, I'll look tomorrow to see if there's some issue with that
@YBonline Okay, latest commit should handle that as well
Alright, seems to work at the 16:10 aspect ratio with no noticeable bugs, however I just looked at CPU usage and noticed it was way up, and is on the frigate.output process. When I first upgraded to this dev branch my load average seemed to stay around 20... I'm now going about 35. I can try to step through the patches and see which one seems to cause if if you want me to (or its possible its unrelated, but I have 300% CPU from frigate.output.
Here's screenshots of htop before and after this patch, I guess I don't necessarily know what "normal" is since birdseye is broken in the before:
@YBonline Try my latest commits, in my testing it dramatically reduces the CPU usage after an initial caching period
Seems to be between the before and after in my previous ones:
I just pushed another commit which should help a bit more. But at a certain point you are going to have a lot of birdseye CPU usage with that many cameras, and the improvements that have been made in this issue means that more cameras can fit / take up more area which of course uses more CPU to create the image for birdseye.
There is a separate PR which reduces the CPU usage by changing the method that is used for resizing the camera frames
That last commit seems to have made it a bit worse again after running for 10 min:
The commit couldn't have really made it worse, it simply reduced the calculations for each cameras aspect ratio because they are saved.
You can try running pyspy in the container to see what is using the CPU for the frigate.output process, but at this point I think your CPU usage is just increased due to the increased cameras and resolution that is being used to show the cameras and not in the layout calculation itself.
Describe the problem you are having
I'm on the latest dev branch, with the new "dynamic" birdseye layout, which looks like a great feature, except I keep getting this error in my logs. The layout updates once a in a while successfully, then stays on that layout and keeps producing the error again. I have lots of panoramic cameras, so I'm guessing it has to do with that? Is there any debug mode for the new birdseye layout? 2023-07-03 19:39:02.696843621 [2023-07-03 19:39:02] frigate.output ERROR : Error finding appropriate birdseye layout Nothing else in the log seems to be relevant
Version
0.13.0-12D4A47
Frigate config file
Relevant log output
FFprobe output from your camera
Frigate stats
Operating system
Other Linux
Install method
Docker CLI
Coral version
USB
Network connection
Wired
Camera make and model
many
Any other information that may be helpful
No response