search

vrcx-team / VRCX

Friendship management tool for VRChat
MIT License
901 stars 167 forks source link

[Feature Request] Database/Log retention period to limit database size, and an offline/read only mode #772

Open i0ntempest opened 2 months ago

i0ntempest commented 2 months ago

Since my original request is too excessive and requires an almost complete rewrite of VRCX (as others commented below), I updated my request to make it simpler. Explain in detail what your suggested feature would be used for. Allow users to set a log retention period, and automatically delete logs older than that period. And an offline/read only mode allowing users to open a database to examine data without making changes.

Describe how it would look if it requires a UI. A few new options:

Explain why people would want to use it. With the default database location being in %appdata%, this would avoid filling up users' C drive. Users may implement periodical backups with backup software or custom script, and the offline mode would allow the user to go back and examine backups.

Original issue: Explain in detail what your suggested feature would be used for. I think a lot of people have VRCX running all the time to log stuff, and the database file could get rather big. I am planning to run it on my home server, so in addition to features requested here, I'd like to request another useful feature that is log rotation. Basically this feature would automatically archive data (with optional compression) that is older than customizable time (say 3 months) from database (maybe with sensitive stuff like passwords removed) to some customizable directory, every some customizable length (say a month), and then remove archived data from the main database. And then, a read only mode mentioned in then issue above, in addition to be able to open another main database at some network location, could also be used to open an archive database without modifying it.

Describe how it would look if it requires a UI. A few new options:

Myrkie commented 2 months ago

this is an excessive amount of requests for a codebase that is not at all ready for it, vrcx would need to be completely rewritten with all of this in mind, and wouldn't at all be an easy drag and drop addition.

DavidLeBonk commented 2 months ago

If you want frequent backups you could manage the database yourself by just writing an application that loads the VRCX sqlite. Copies data to your new table/custom backend and formatting/compressing as you see fit over there?

i0ntempest commented 2 months ago

I'm not a software developer but I have indeed had thoughts about writting a powershell script to manage database backups, using sqlite commands and simple file operations. I'm much more experienced with bash but VRCX only runs on Windows. The process in my head is:

Which should mostly accomplish what I want, but I don't think copying and deleting stuff from the main database while VRCX is open and might be writing to it is a good idea. Thoughts?

DavidLeBonk commented 2 months ago

Well I personally would recommend not using powershell scripts here and choosing a language like C# or python to make an actual application to do this as it will make any changes you desire to make in the future significantly easier.

i0ntempest commented 2 months ago

Okay, I can do a bit of Python. But I'm not familiar with databases so please bear with me: does sqlite handles concurrent writes well? What would happen if I start deleting stuff while VRCX is trying to write to it?

DavidLeBonk commented 2 months ago

Long as it ain't the same element no issues should occur assuming the VRCX codebase uses transactions where possible (I assume it does). Or you could just do periodic close of VRCX duplicate the db and work with that while VRCX boots back up.

i0ntempest commented 2 months ago

Okay - I did a bit of research, and I think I can make a script to do the backup. I'll be updating my request to retention period and an offline mode only. Hope that is doable.