search

CSDL-UMD / Rockwell

Rockwell uses the twitter authentication workflow to render a twitter like feed in order to collect information about the users interaction with their feed. It also has an attention check feature to ensure that the user is being observant of their feeds and not simply scrolling through with the intent of finishing quickly.
7 stars 2 forks source link

Persist sessions / tokens data on disk #217

Closed glciampaglia closed 10 months ago

glciampaglia commented 12 months ago

Right now it twauth-web we keep a couple of global dictionaries to store tokens and sessions. The problem is that when the process is terminated, we lose the contents of both dictionaries. To persist on disk we can use the sqlitedict package, which provides a dict-like interface to persist data in an sqlite database.

Rivado-E commented 11 months ago

I added the database connections and the new dictionaries. I chose to make each of the former dictionaries a table for easy migration from the former dictionaries to the sqlitedicts. The only thing left is the process through which a single user's data is deleted.

glciampaglia commented 11 months ago

We have added sqlitedict to replace all global Python dicts. However, some dictionaries (like the ones for access token and access token secret) are actually being initialized from the PostgresQL db. So it would be simpler to just fetch the information from postgres instead. @saumyabhadani95 will review all the new persistent "dicts" and see if any of those can be simply fetched from the DB instead.

Regarding when to close the persistent dict, we will add a try/finally block in the __main__ condition to make sure that the databases are closed cleanly no matter what happens.

Regarding purging entries from the persistent dicts: right now it seems we will not have situations where stale information may prevent participants from completing the task. Furthermore, some of the dicts are written once and never re-used (for example we do not need to consider situations where the same participant is having a different task_id), so it should be OK to keep the delete function empy.