Let's shadow a Python release

[agriyakhetarpal]

Title

Let's shadow a Python release: insights from packaging, distribution, and supply-chain security

Describe your Talk

As the Python programming language nears its annual release with version 3.13 in October 2024, there are a bunch of exciting new features, such as a greatly improved interactive read-eval-print-loop (REPL), a just-in-time (JIT) compiler via PEP 744, free-threading builds that disable the global interpreter lock (GIL) via PEP 703, and iOS support via PEP 730.

However, have you ever wondered how one ends up receiving a Python distribution for a particular version with all these features and updates, packaged for convenience and subsequent use for one's desired operating system? Also, what are the intricacies around this process, viz. software project management and security implications? This talk aspires to explore these areas in the Python release process, right from beta versions to the final published builds.

The takeaways from the talk, summarised in a (provisional) outline shall be as follows:

1. (Ten minutes) – the process of releasing a new Python 3.X version:
   • the release and support cycles for Python versions, plus the stakeholders involved
   • releasing backports and security updates
   • downstream packaging and distribution: package management software and ports across platforms
2. (Fifteen minutes) – a cursory look over SLSA (supply-chain levels for software artifacts) provenance and SBOM (software bill-of-materials) for release artifacts in the Python ecosystem
   • the basics of build provenance
   • the SLSA framework and its levels
   • how Python participates and facilitates software provenance
   • SBOM, and SBOM for Python
   • possible upcoming features: index-level support for attestations via PEP 740, standalone Python binaries via PEP 711, and more

[!NOTE] Owing to the varied nature of the word, the use of the term "Python" and its derivatives in this issue/proposal unambiguously refers to CPython, i.e., the most popular implementation of the Python programming language, unless explicitly specified otherwise.

Pre-requisites & reading material

The major prerequisite is an interest in the Python programming language. Some insights into the use of package managers and installation archives might tend to be useful, but will not be needed. No prior knowledge is required or assumptions made w.r.t the concepts of build provenance or SLSA – owing to the advanced nature of these topics, a deep dive into their internals is not planned for inclusion in the material.

Time required for the talk

Thirty minutes; twenty-five for the presentation, five for fielding questions

Link to slides/demos

No response

About you

I am an undergraduate student at the University of Delhi, and a software engineer at Quansight, where my work is aligned towards the packaging and distribution of fundamental open-source software in the PyData ecosystem. My current work assignment is based around @pyodide, a Python implementation designed for the browser based on WebAssembly, and on approaches to interactive documentation for packages and libraries in the Scientific Python ecosystem through it.

My interests include Python packaging, scientific computing, numerical software, compilers and toolchains, and a lot more.

---

Please feel free to reach out to me through my social media handles:

• LinkedIn: https://www.linkedin.com/in/agriyakhetarpal/
• Twitter: https://twitter.com/agriyakhetarpal

or contact me via social messaging apps, such as Telegram: https://t.me/agriyakhetarpal or on Signal: @agriyakhetarpal.01. For enquiries of private disposition, please reach out to me at my email address: agriyakhetarpal [at] outlook [dot] com

Availability

July/August

Any comments

I am unsure about my availability at this time – hence, I am opening this issue as a placeholder in order to indicate my interest in delivering this talk in the meetups that are to be scheduled for either of these months. I will be in a better position to share my availability and my slides closer to the planned date.

[pulsar17]

Hi Agriya, thanks for proposing a talk. Would you be available on 20th July (3rd Saturday) to present?

[agriyakhetarpal]

Hi Ishaan, I will be available to present on the 20th, thank you!

[agriyakhetarpal]

Hi again; I think I'll need the talk to last slightly longer (perhaps a margin of ten or so more minutes). Would this be conducive? I thought it would be nice to let this be known beforehand while the venue and the timings are being decided.

[pulsar17]

Yes, +10 mins is fine, you can go on for longer even if people seem interested. That anyway depends on what happens on the day itself.

The venue is confirmed, it is Sun foundation World class skill center (Blue Line). We haven't announced it yet. The timings will be 1-5 pm.

[agriyakhetarpal]

Thanks for the confirmation on the venue! I'll be there.

[agriyakhetarpal]

Hi, I'm closing the issue since the talk was delivered on 20th July, 2024. Thanks for having me!

[!TIP] The slides shall remain available at the following link: https://slides.com/agriyakhetarpal/pydelhi-july-2024.

Resources for further reading

[!NOTE] These resources aim to complement what was discussed in the slides. A link to this comment has been added to the final slide.

1. Python release PEPs

   • https://peps.python.org/pep-0719/
   • https://peps.python.org/pep-0745/

2. The Python development and release cycle, and related notes

   • https://devguide.python.org/developer-workflow/development-cycle/#devcycle
   • https://endoflife.date/python
   • https://github.com/python/release-tools
   • https://www.youtube.com/watch?v=jhzv5RU56V4
   • https://sethmlarson.dev/security-developer-in-residence-weekly-report-35
   • https://www.youtube.com/watch?v=0fw2wKCS9B4

3. Software Bill-of-Materials, and Python

   • https://www.synopsys.com/blogs/software-security/software-bill-of-materials-bom.html
   • https://sethmlarson.dev/security-developer-in-residence-weekly-report-28
   • https://sethmlarson.dev/security-developer-in-residence-weekly-report-33
   • https://discuss.python.org/t/create-and-distribute-software-bill-of-materials-sbom-for-python-artifacts/39293
   • https://devguide.python.org/developer-workflow/sbom/

4. SLSA and build provenance

   • https://pradeepl.com/blog/slsa/
   • https://slsa.dev/spec/v1.0/
   • https://slsa.dev/spec/v1.0/provenance
   • https://slsa.dev/spec/v1.0/levels
   • https://blog.sigstore.dev/homebrew-build-provenance/
   • https://blog.trailofbits.com/2024/05/14/a-peek-into-build-provenance-for-homebrew/
   • https://devblogs.microsoft.com/nuget/building-a-safer-future-how-nuget-is-tackling-software-supply-chain-threats/

5. Dependency confusion exploits

   • https://github.com pypa pip issues 8606 (not linking the issue so that it isn't visible – please resolve the link yourself)
   • https://www.sentinelone.com/blog/pytorch-dependency-torchtriton-supply-chain-attack/
   • https://pytorch.org/blog/compromised-nightly-dependency/

6. DoS via typosquatting

   • https://blog.phylum.io/a-pypi-typosquatting-campaign-post-mortem/
   • https://www.techtarget.com/searchsecurity/news/366577455/Typosquatting-campaign-malicious-packages-slam-PyPi

7. Ancillary resources

   • https://jfrog.com/blog/leaked-pypi-secret-token-revealed-in-binary-preventing-suppy-chain-attack/
   • https://jfrog.com/blog/malware-civil-war-malicious-npm-packages-targeting-malware-authors/
   • https://arstechnica.com/security/2024/07/code-sneaked-into-fake-aws-downloaded-hundreds-of-times-backdoored-dev-devices/
   • https://checkmarx.com/blog/pypi-is-under-attack-project-creation-and-user-registration-suspended/

8. Scientific Python and Python packaging

   • https://hackmd.io/@scientific-python/spec-8
   • https://github.com/sigstore/sigstore-python
   • https://github.blog/changelog/2024-06-25-artifact-attestations-is-generally-available/

[Animesh-Ghosh]

Thanks for the talk @agriyakhetarpal and also for updating the issue with resources for further reading!