Vulnerabilities and dependencies

[nutjob4life]

The P5 portal uses a number of packages (and Python) itself which may be out-of-date or may have security vulnerabilities. A security scan revealed the following:

flask                 2.1.3      2.2.5     python  GHSA-m2qf-hxjv-5gpq  High      
gunicorn              20.1.0     22.0.0    python  GHSA-w3h3-4rj7-4ph4  High      
pillow                9.5.0      10.0.1    python  GHSA-j7hp-h8jx-5ppr  High      
pillow                9.5.0      10.0.0    python  GHSA-8ghj-p4vj-mr35  High      
pillow                9.5.0      10.0.1    python  GHSA-56pw-mpj4-fxww  High      
pillow                9.5.0      10.2.0    python  GHSA-3f63-hfp8-52jq  High      
tiff                  4.5.1-r0             apk     CVE-2024-7006        High      
tiff                  4.5.1-r0             apk     CVE-2023-52356       High      
tiff                  4.5.1-r0             apk     CVE-2023-52355       High      
werkzeug              2.0.3      2.2.3     python  GHSA-xg9f-g7g7-2323  High      
werkzeug              2.0.3      3.0.3     python  GHSA-2g68-c3qc-8985  High      
nghttp2-libs          1.57.0-r0            apk     CVE-2024-28182       Medium    
pillow                9.5.0      10.3.0    python  GHSA-44wm-f244-xhp3  Medium    
tiff                  4.5.1-r0             apk     CVE-2023-6277        Medium    
tiff                  4.5.1-r0             apk     CVE-2023-41175       Medium    
tiff                  4.5.1-r0             apk     CVE-2023-40745       Medium    
tiff                  4.5.1-r0             apk     CVE-2023-3164        Medium    
tiff                  4.5.1-r0             apk     CVE-2015-7313        Medium    
wagtail               5.2.3      5.2.6     python  GHSA-jmp3-39vp-fwg8  Medium    
tiff                  4.5.1-r0             apk     CVE-2023-6228        Low       

We don't have any critical vulnerabilities thankfully, but 11 "high" vulnerabilities have slipped by us and we need to address them ASAP.

It would also be nice to check to see if we can upgrade any of the following:

• [x] Python 3.11 → 3.12
• [x] celery and redis
• [x] Django 4.2 → 5.1
• [x] Django auth LDAP
• [x] Django-redis
• [x] pillow 9.5 → 10.4
• [x] psycopg2 2.9 → whatever the next iteration of this package is
• [x] wagtail 5.2 → 6.2
• [x] wagtail-favicon
• [x] wagtail-metadata
• [x] wagtail-menus 3.1 → 4.0

Postponing this for now:

• Elasticsearch → OpenSearch

[nutjob4life]

Note: Python 3.13 is now out too but is not yet supported by Wagtail, Django, etc. In fact, neither is Python 3.12. We should stay with Python 3.11.

• Pillow can be upgraded to 10.3
• psycopg2 can be upgraded to 2.9.9

[nutjob4life]

Since I first began investigating these vulnerabilities, a new critical one appeared:

py3-click             8.1.7-r0              apk     CVE-2015-8768        Critical  

But we're not even using py3-click so I'm not sure why this has appeared!

[nutjob4life]

Okay, I've fixed what I can; the remaining vulnerabilities are:

py3-click             8.1.7-r0              apk     CVE-2015-8768        Critical  
flask                 2.1.3       2.2.5     python  GHSA-m2qf-hxjv-5gpq  High      
gunicorn              21.2.0      22.0.0    python  GHSA-w3h3-4rj7-4ph4  High      
tiff                  4.6.0-r0              apk     CVE-2024-7006        High      
tiff                  4.6.0-r0              apk     CVE-2023-52356       High      
werkzeug              2.0.3       2.2.3     python  GHSA-xg9f-g7g7-2323  High      
werkzeug              2.0.3       3.0.3     python  GHSA-2g68-c3qc-8985  High      
werkzeug              2.3.8       3.0.3     python  GHSA-2g68-c3qc-8985  High

For these I have the following mitigation statements:

• py3-click — vulnerable file click/install.py is not installed and therefore not vulnerable
• flask — false positive, 2.3.3 installed by OS
• gunicorn — 21.2.0 is the latest avalable from the OS
• tiff — no newer alternative available, but we do not use TIFF images anyway
• werkzeug — 2.3.8 is latest available from the OS

[nutjob4life]

The remaining upgrades are "nice-to-haves":

• wagtail-favicon 0.3.0 is the latest available and still works even with Wagtail 6.2.2 (tested locally)
• wagtail-metadata 5.0.0 is the latest and work with Wagtail 6.2.2 (also tested locally)
• wagtailmenus 4.0.1 seems to work great, so putting in this upgrade
• django_celery_results 2.5.1 is the latest (from May 2023), Celery gets upgraded to 5.4.0, Redis must stay at 5.2.0
• wagtail upgraded to 5.2.6
• django upgraded to 5.0.9 (this is the big one!)
• django-auth-ldap upgraded to 5.1.0

[nutjob4life]

@hoodriverheather @tloubrieu-jpl okay, I've made a ton of changes to support all of the above ↑

These security updates and modernization changes are now live on the dev portal. Because this affected lots of the core software (Django, Wagtail, etc.), there could be unexpected effects on various parts.

I ran the automated tests and got all 258 passing. But of course, since I came up with the automated tests, they have my bias.

If you could "kick the tires" and make sure your favorite parts of the portal are still working the way you expect, I'd appreciate it (@hoodriverheather you might especially look at the metadata forms and other forms, as the form machinery got updated).

[nutjob4life]

Hi folks, I'm going to go ahead and merge this into the main development tree because we have just the one dev platform and I've got another feature I need to have ready for testing by next week's (November 15th) code freeze

The dev portal will be down for about 15 minutes

[hoodriverheather]

@nutjob4life The LabCAS Documentation Page under Data and Resources/Informatics/ is not displaying the icons and giving me Not found error when I try to click on all links except - LabCAS EDRN Cancer Biomarker Data Commons (LabCAS) Data

Link to page: https://edrn-dev.jpl.nasa.gov/portal/renaissance/data-and-resources/informatics/labcas-cancer-biomarker-data-commons/

Screenshots:

[nutjob4life]

@hoodriverheather that's to be expected; it's because that page is raw HTML and won't work anywhere except at NCI.

We can fix this, but it's a lower priority item.

[hoodriverheather]

@nutjob4life Sounds good. Just trying to click on most things. :)

[hoodriverheather]

@nutjob4life Looks good to me. Nice job!

[nutjob4life]

@hoodriverheather thanks for being so thorough :relieved

[nutjob4life]

Reopening this because the fix to #380 was to undo a lot of these changes.