Also ran a survey of requirements.txt and upgraded/removed some low-hanging fruit:
Whitenoise 3.3.0 broke with the Django version upgrade, so I upgraded it to the most recent version, version 6.6.0. This involved changing the wsgi configuration and the whitenoise middleware.
Django Contact Form was at version 1.4.2, but version 2.1.0 is the current version. The upgrade was simple, but did involve some namespace changes (from contact_form... -> from django_contact_form)
Django Feature Policy was at version 3.0.0 and still added the feature policy header, but this draft http header was actually rejected in favor of the permissions policy header (https://httptoolkit.com/blog/renaming-feature-policy-to-permissions-policy/). With this renaming, the package was renamed to Django Permissions Policy. I went ahead and changed this package and upgraded to the most recent version, 4.19.0 and made some updates to our settings.py, accordingly.
Removed simplejson - we were only using it in one place in the codebase and I couldn't see any reason we weren't using the builtin json module there.
Removed python2 to python3 compatibility layers:
six==1.10.0, we're using python3, so no python2 to 3 compatibility needed
future==0.16.0, the python3 future is now
Removed exhibitapp dependencies; instead, we should prefer the requirements.txt file in that repository: https://github.com/ucldc/exhibitapp/blob/main-3.2/requirements.txt - I think this will require an update to our CodeBuild buildspec, though, because beanstalk only installs one requirements.txt file in the root of the application version.
These were some easy version bumps with no need for codebase changes:
retrying v1.3.3 to v1.3.4
requests v2.21.0 to v2.31.0
django-health-check v3.9.0 to v3.18.1
These packages were already at their most current versions (which are quite old):
django-easy-pjax==1.3.0
django-recaptcha2==1.4.1
Finally, I skipped over any upgrades to redis packages django-redis v4.7.0 (most recent version 5.4.0), redis v2.10.5 (most recent version 5.0.3), hiredis v0.2.0 (most recent version 2.3.2) - these were all big version jumps, and I don't have a complete understanding of how exactly Calisphere and Django use redis.
I also skipped upgrades to aws-xray-sdk 0.93. This was a major jump to v2.13.0, and I don't understand how Calisphere is related to AWS X-Ray.
I also skipped upgrades to other critical underlying packages - mysqlclient 1.3.12 (most recent version is 2.2.4), elasticsearch 7.13.4 (most recent version is 8.13.0), and django-statis-sitemaps 4.2.1 (most recent version is 5.0.0). Since these are all major version upgrades that are also critical to our application, I'd like to perform those upgrades in isolation.
This is a duplicate of: https://github.com/ucldc/public_interface/pull/316, but rebased on this update to
mainhttps://github.com/ucldc/public_interface/commit/0bab9d69593521ff26d8fbd00f7d89bfcd4c986a, and including an update tomysqlclientin response to beanstalk error messagesAttributeError: module 'MySQLdb.constants.FIELD_TYPE' has no attribute 'JSON', more details here: https://github.com/ucldc/public_interface/pull/321[This PR is paired with: https://github.com/ucldc/exhibitapp/pull/31]
Also ran a survey of requirements.txt and upgraded/removed some low-hanging fruit:
from contact_form...->from django_contact_form)simplejson- we were only using it in one place in the codebase and I couldn't see any reason we weren't using the builtinjsonmodule there.Removed python2 to python3 compatibility layers:
Removed
exhibitappdependencies; instead, we should prefer the requirements.txt file in that repository: https://github.com/ucldc/exhibitapp/blob/main-3.2/requirements.txt - I think this will require an update to our CodeBuild buildspec, though, because beanstalk only installs onerequirements.txtfile in the root of the application version.These were some easy version bumps with no need for codebase changes:
These packages were already at their most current versions (which are quite old):
Finally, I skipped over any upgrades to redis packages django-redis v4.7.0 (most recent version 5.4.0), redis v2.10.5 (most recent version 5.0.3), hiredis v0.2.0 (most recent version 2.3.2) - these were all big version jumps, and I don't have a complete understanding of how exactly Calisphere and Django use redis.
I also skipped upgrades to aws-xray-sdk 0.93. This was a major jump to v2.13.0, and I don't understand how Calisphere is related to AWS X-Ray.
I also skipped upgrades to other critical underlying packages - mysqlclient 1.3.12 (most recent version is 2.2.4), elasticsearch 7.13.4 (most recent version is 8.13.0), and django-statis-sitemaps 4.2.1 (most recent version is 5.0.0). Since these are all major version upgrades that are also critical to our application, I'd like to perform those upgrades in isolation.