roflcoopter commented 1 year ago

This is a pretty massive PR that brings a number of new features and improvements.

The main feature developed here is the option of having 24/7 recordings. With this comes a new component, storage which will allow you to retain data based on time (like it is today) and based one consumed space. This allows you to use your storage to its maximum.

Another feature of the storage component are tiers. They allow you to store recordings on multiple media. Say you have one 256GB SSD and a 1TB HDD, you can keep the most recent recordings on the SSD for fast access while offloading the older recordings to the HDD for archiving.

All of this will be in the docs soon.

Breaking Changes

Recordings folder structure has changed mp4 file extension is now forced and CONFIG_EXTENSION is removed CONFIG_SEGMENTS_FOLDER is removed in favor of tier paths Thumbnails are now stored in /thumbnails folder (depending on the tier config) CONFIG_FILENAME_PATTERN in CONFIG_THUMBNAIL is removed, stored as ID of the recording

Tasks:

[x] CONFIG_RETAIN is removed in favor of tier max_age
[x] Add a force move to next tier on stop to avoid losing data in RAM when restarting
[x] Cleanup FilesMeta table on shutdown. Solved by a trigger
[x] CONFIG_RETAIN is still used in some places
[x] CONFIG_FOLDER is removed in favor of tier paths
[x] Remove segment cleanup
[x] Remove recordings cleanup
[x] Force mp4
[x] Fix saving of recordings clips create_event_clip
[x] Must change recorder/complete event listener in Recordings UI since it is not triggered if create_event_clip is False
[x] Add a safe mode if storage component fails to load
[ ] Add manual disc sync to make sure that all files are in the database
[x] Fix face recognition pickle error
[x] Add max length to recordings
[x] Make sure that FailedCamera recordings work properly
[x] Remove FailedCamera from _get_tier_config and preceding calls
[x] Fix deletion of recordings
[x] Update docker docs with new VOLUMES
[x] Update UI to use HLS recordings
[x] Dynamic file fetching to find files that have been moved to different tiers
[x] Add orig_ctime to FilesMeta table to store the original creation time of the file, replacing created_at in most places and also sorting on filename column in recordings_to_move_query
[x] Delete recordings from database when files are removed
[x] API for fetching 24/7 events
[x] End HLS playlists if there is a large gap in the segments
[X] Set Recordings end_time/Motion end_time to current time on startup in case of crashes
[x] New UI page called Events
- [x] Timeline
- [x] Events
- [ ] Filters
- [ ] Update timeline using websockets
- [x] Add link from startpage to Events
- [ ] Make clicking the timeline more intuitive

netlify[bot] commented 1 year ago

Deploy Preview for viseron canceled.

Name	Link
Latest commit	b2b8091be798a31e83226e8fb5e9bcb1a9abdf26
Latest deploy log	https://app.netlify.com/sites/viseron/deploys/65f7ecbd5744a80008e096cb

tdashmike commented 6 months ago

Looking forward to this. Do you need help beta testing?

roflcoopter commented 6 months ago

For sure, eventually! It is still in an early stage, it mostly works but some polishing remains before it is ready for testing

jMyles commented 5 months ago

Really appreciate your huge lift on this PR - and this project - @roflcoopter.

roflcoopter commented 5 months ago

Thank you! It's taking a lot of time and effort but I think it will turn out great.

I hope to release an alpha soon when the timeline part is done to get some feedback. Then focus on the remaining parts like migrating old recordings over and the documentation for the new components

roflcoopter commented 3 months ago

Getting close now, just need to make some tweaks on the timeline view!