openbikesensor / portal

OpenBikeSensor data collection portal
GNU Lesser General Public License v3.0
36 stars 17 forks source link

Full user data export #215

Open opatut opened 2 years ago

opatut commented 2 years ago

I would like to see an export feature for my account, for moving between instances, or just having a backup of my data. This is required anyway for DSGVO inquiries, so why not add it?

It needs to contain, for now:

I'd say we make a "user_export" job table in the database, scan it the worker, generate a zipfile for each pending job, store this in a temporary folder, then allow the user to download this zipfile from the profile page if the export job is complete. The worker can also delete old export jobs and their related files after a while (say, 7 days or so).

gluap commented 1 year ago

A data-intensive part of this is now available in #321 (that mega-pr also contains bulk download of tracks). It works in real time. I believe adding "all my comments" as well as "all my track details" will still be possible in real time. It would save us having some extra queues and workers - we would only have to add a new api endpoint for the export without having workers, a queue and tables for shuffling around data.