Open rroctavian opened 6 months ago
The suggestion has a lot of merit... The issue is that
For background, the "should be usable without cryptography" requirement comes from the idea that python-tuf wants to be potentially vendorable in pip (and cryptography as natively built package is not acceptable there)... It could be acceptable that python-tuf had a default cryptography dependency but we just expect the vendoring process to break that dependency (and make sure it's easy to do). The issue here would be, how do we ensure python-tuf remains easily "usable without cryptography" in the future: I suppose we can somehow make sure there are still tests that run without cryptography
@jku, thank you for getting back.
Given pyca/cryptography's page says
Our goal is for it (i.e. pyca/cryptography) to be your "cryptographic standard library".
I was wondering if pyca/cryptography might have a timeline of their own to become vendorable-friendly?
- python-tuf should also be usable in apps that do not want cryptography
... python-tuf wants to be potentially vendorable in pip (and cryptography as natively built package is not acceptable there)...
Please let me know if I am getting the point here. The way I see it, python-tuf
would either:
pyca/cryptography
bits that are needed.
Are there, by any chance, any issues/conversations with active initiatives on this topic? I'd be happy to have a read and fill in my gaps.
I am slightly concerned that this issue might be off-putting to new users and have an impact on the adoption of python-tuf
in favour of e.g. go-tuf CLI, thus resulting in fewer eyes here. The reason is that the problem arises in the first few minutes of using the package and it is might not be trivial to explain initially. One would either need to perform break-point debugging to find the import error or would need to have the logging properly configured. It does not help that securesystemslib logger has to be grabbed and forced to output at DEBUG level, otherwise the issue goes unnoticed in the first read of the logs. That is also not obvious.
The way I see it, we either live with the current inconvenience while working on becoming pip-vendorable, or patch it to help new users and continue thinking about the best solution.
Would it be acceptable at all to list securesystemslib[crypto] as a dependency in the meantime? I am curious to hear thoughts on this.
I think this might be a good idea, and I'm even thinking python-tuf might not have to do anything here except add the dependency: With the current state of the code I'm fairly confident python-tuf will not accidentally start requiring cryptography in code in the future assuming securesystemslib does not do so -- so we'd just want to ensure that securesystemslib has tests that ensure it works without cryptography (it might already, I did not check). This should ensure that vendoring both (for ed25519 verificiations) is possible.
@lukpueh opinions on this?
That said, I'll point out a detail about this bit:
I am slightly concerned that this issue might be off-putting to new users and have an impact on the adoption of python-tuf in favour of e.g. go-tuf CLI
There is a reason for python-tuf being "harder to install" than a CLI app: python-tuf is not an application but a library. We don't expect end-users to ever install it, we expect:
securesystemslib[gcpkms,hsm,sigstore]
). The application is the only one who knows what is going to be needed, python-tuf can never do that 100%Now, developer experience does matter even for libraries and it is true that e.g. any tutorial would have to start with "first remember to install securesystemslib[cryptography]"... This is not ideal so I think your idea is good.
I guess it's ok. If someone wants to vendor tuf for a pure-python environment it should be easy for them to just ignore that dependency, as long as our code supports it (securesystemslib
does have tests for this).
It will become harder though to use tuf in a pure-python environment without vendoring dependencies. But I don't know if that use case exists. If it does, it should be rare enough to justify improving DX for everyone else.
If we do this, we need to update the installation docs here: https://theupdateframework.readthedocs.io/en/latest/INSTALLATION.html
@jku @lukpueh what's the conclusion, are we still okay on adding this? with above comments, It looks that we were agreed on adding it. If yes, I would like to work.
Issue Description
The purpose of this issue is to kindly ask whether listing
securesystemslib[crypto]
as a dependency would improve the out-of-the-box experience withpython-tuf
.I noticed that
pip3 install tuf
did not upgrade the pre-existing, oldcryptography==3.4.8
present in a standard python installation. (The exact situation is an AWS Ubuntu 22.04 machine where python3 comes pre-installed withcryptography==3.4.8
.) A more recent version likecryptography>=37.0.0
is required to perform atuf.ngclient.Updater.download_target
operation.cryptography>=37.0.0
is listed as a dependency of the custom installsecuresystemslib[crypto]
of securesystemslib, but not for the base install. Unfortunately,tuf==3.1.0
only mentionssecuresystemslib[crypto]
inrequirements/main.txt
, but not as a dependency inpyproject.toml
.Reproduce issue
The issue becomes evident during signature verification processes, where the older cryptography library cannot correctly handle the signatures. Here are relevant snippets from the logs:
Please find attached the python file which generates the error and its logs below. You should be able to run the python script from anywhere as it has the paths indicated above hard-coded for this example. tuf_import_error_issue.py.txt
Logs
Summary
python-tuf
listssecuresystemslib>=0.26.0
as a dependency but does not specify that it should include the[crypto]
extras.cryptography
library is already installed, installingpython-tuf
does not prompt an upgrade to meetsecuresystemslib[crypto]
's requirements, leading to potential signature verification issues.securesystemslib[crypto]
as a direct dependency forpython-tuf
. This change would ensure that the necessarycryptography
version is installed or upgraded duringpython-tuf
's installation, mitigating issues related to outdated dependencies and improving the out-of-the-box security and reliability ofpython-tuf
, especially in environments where dependency management is crucial.I appreciate that managing dependencies is a delicate balance and I am curious to hear your thoughts. Please let me know if there is any further information I could help with.