[Bug]: ValueError on aqt install-qt

itsrutledge-sudo commented 3 months ago

Bug description

Produced on my machine when running: /home/runner/.local/bin/aqt install-qt --outputdir ~/Qt linux desktop 6.7.0 linux_gcc_64

Expected behavior

Expected it to install.

aqt and python version

INFO : aqtinstall(aqt) v3.1.18 on Python 3.10.12 [CPython GCC 11.4.0]

Operating System

Linux/Unix

Relevant log output

runner@ptec-grp-m-pipeline-runner:~$ /home/runner/.local/bin/aqt install-qt --outputdir ~/Qt linux desktop 6.7.0 linux_gcc_64
INFO    : aqtinstall(aqt) v3.1.18 on Python 3.10.12 [CPython GCC 11.4.0]
ERROR   : [digital envelope routines] unsupported
Traceback (most recent call last):
  File "/home/runner/.local/lib/python3.10/site-packages/aqt/installer.py", line 182, in run
    args.func(args)
  File "/home/runner/.local/lib/python3.10/site-packages/aqt/installer.py", line 449, in run_install_qt
    run_installer(qt_archives.get_packages(), base_dir, sevenzip, keep, _archive_dest)
  File "/home/runner/.local/lib/python3.10/site-packages/aqt/installer.py", line 1149, in run_installer
    queue = multiprocessing.Manager().Queue(-1)
  File "/usr/lib/python3.10/multiprocessing/managers.py", line 723, in temp
    token, exp = self._create(typeid, *args, **kwds)
  File "/usr/lib/python3.10/multiprocessing/managers.py", line 606, in _create
    conn = self._Client(self._address, authkey=self._authkey)
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 508, in Client
    answer_challenge(c, authkey)
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 755, in answer_challenge
    digest = hmac.new(authkey, message, 'md5').digest()
  File "/usr/lib/python3.10/hmac.py", line 184, in new
    return HMAC(key, msg, digestmod)
  File "/usr/lib/python3.10/hmac.py", line 60, in __init__
    self._init_hmac(key, msg, digestmod)
  File "/usr/lib/python3.10/hmac.py", line 67, in _init_hmac
    self._hmac = _hashopenssl.hmac_new(key, msg, digestmod=digestmod)
ValueError: [digital envelope routines] unsupported
ERROR   : aqtinstall(aqt) v3.1.18 on Python 3.10.12 [CPython GCC 11.4.0]
Working dir: `/home/runner`
Arguments: `['/home/runner/.local/bin/aqt', 'install-qt', '--outputdir', '/home/runner/Qt', 'linux', 'desktop', '6.7.0', 'linux_gcc_64']` Host: `uname_result(system='Linux', node='ptec-grp-m-pipeline-runner', release='5.15.0-1068-azure-fips', version='#77+fips1-Ubuntu SMP Tue Jun 25 15:25:32 UTC 2024', machine='x86_64')`
===========================PLEASE FILE A BUG REPORT===========================
You have discovered a bug in aqt.
Please file a bug report at https://github.com/miurahr/aqtinstall/issues
Please remember to include a copy of this program's output in your report.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

ddalcino commented 2 months ago

I'm not super confident I know what's happening here, but:

[digital envelope routines] unsupported is usually an error you get from OpenSSL when you're asking it to use an algorithm it doesn't support.
Your uname string mentions fips a couple times, in the release and version section.
As I understand it, the FIPS version of OpenSSL is not allowed to use MD5. MD5 hasn't been secure for over a decade.
In this case, Python is trying to make an MD5 digest, for multiprocessing purposes, not security.

I am drawing the conclusion that when Python calls out to OpenSSL to make the MD5 for non-security purposes, it fails for security reasons. IMHO this is an issue with Python, not aqtinstall. Go ahead and google "Python", "FIPS" and "multiprocessing"; I'm betting this guy's blog will pop up a lot, suggesting ways to rebuild Python: https://www.gyanblog.com/security/

I'm not sure how to fix this conclusively, but here are some possible workarounds:

Set concurrency: 1 in your aqt settings.ini file: see https://aqtinstall.readthedocs.io/en/latest/configuration.html#settings Installation could take around 4x longer than usual, since you won't be using concurrency.
Use a non-FIPS system.
Try using a different version of Python. Perhaps there's a newer version that knows how to run a multiprocessing workload without running afoul of FIPS-enabled OpenSSL.

Good luck, and please let us know how it went!

miurahr commented 2 months ago

I think hmac.new() can take usedforsecurity argument when Python 3.9 and later.

https://docs.python.org/3/library/hashlib.html#hashlib.new

Your log tells us

  digest = hmac.new(authkey, message, 'md5').digest()

no usedforsecurity=false there. This can be improved to add it.

Python manual expresses that

Changed in version 3.12: For any of the MD5, SHA1, SHA2, or SHA3 algorithms that the linked OpenSSL does not provide we fall back to a verified implementation from the HACL* project.

https://docs.python.org/3/library/hashlib.html#hash-algorithms

You may have a chance to try Python 3.12.x

miurahr commented 2 months ago

I found that Python 3.12 and later improved the multiprocessing with stronger HMAC

a change commit
but ticket multiprocessing.connection challenge implicitly uses MD5

I think it is worth trying python 3.12 and later.

miurahr / aqtinstall