search

MegaAntiCheat / masterbase

API/Data Platform for Ingesting, Storing, and Serving Data through Postgres, and Litestar
11 stars 1 forks source link

Keep Demos on Disk ? #33

Open jayceslesar opened 7 months ago

jayceslesar commented 7 months ago

Might not be the wisest idea to keep demos in the DB, but we can back them out with the following query at least:

\copy (
     SELECT loid, STRING_AGG(data, '' ORDER BY pageno) AS all_data
     FROM pg_largeobject
     JOIN demo_sessions demo ON demo.demo_oid = pg_largeobject.loid
     WHERE demo.session_id = 'some_session_id'
     GROUP BY loid
 ) TO '/home/worker/asdf.dem' WITH BINARY;

It might be beneficial to keep them on disk.

jayceslesar commented 7 months ago

as of now I think this is a bad idea, and that keeping these in the DB is a better bet

megascatterbomb commented 7 months ago

The potential convenience of being able to back up the demos themselves by just backing up all the postgres databases is too valuable imo. Sticking with the db also prevents any large malicious payloads from just existing on the main file system waiting for execution.

jayceslesar commented 6 months ago

Was doing a large export with @kavorite and noticed bad performance issues on exports using the current method, I believe it might be best to fallback to disk or some external storage in prod. Being able to backup is nice, but we can also back up a drive. Memory stays at the same level but CPU gets pretty tanked when querying a demo