search

gwu-libraries / TweetSets

Service for creating Twitter datasets for research and archiving.
MIT License
25 stars 2 forks source link

Upgrades to Python 3.8 along with dependencies #130

Closed lwrubel closed 3 years ago

lwrubel commented 3 years ago

This upgrades the server/flask-server and worker containers to Python 3.8, but does not touch the Python version in the loader and spark-master containers, which has to wait until Java and Spark are updated to compatible versions.

Note that celery is upgraded to the highest 4.x version possible (4.4.7) and redis has a major version upgrade to 3.x. See the #126 for details and migrating instructions from the redis-py docs.

To test, build the new containers, create extracts, and run tests. Please confirm that a "previous datasets" created before the upgrade is available. Also recommend running a spark load to confirm that it runs without changes.

dolsysmith commented 3 years ago

Tests pass and UI works as expected. Previously created datasets still available.