serverless / serverless-python-requirements

⚡️🐍📦 Serverless plugin to bundle Python packages
MIT License
1.09k stars 289 forks source link

Swap pip for uv #830

Open lmmx opened 4 months ago

lmmx commented 4 months ago

Is there an existing issue for this?

Use case description

uv was just released (from the makers of ruff), and is substantially faster than pip, pip-tools, and virtualenv. This seems desirable to speed up serverless builds.

Proposed solution (optional)

Announcing uv: an extremely fast Python package installer and resolver, written in Rust.

uv is designed as a drop-in alternative to pip, pip-tools, and virtualenv.

With a warm cache, uv installs are near-instant. Here, it's > 75x faster than pip and pip-tools.

Source

The “drop-in” claim seems to suggest you might be able to just change the binary i.e. re-alias pip to uv pip

pgrzesik commented 4 months ago

Hey @lmmx - how do you imagine support for uv on the plugin level? As the plugin uses pip that is installed locally, I think "just" aliasing on your own should be enough, but maybe I'm missing some specific use cases. Please let me know what do you think 🙇

Zer0x00 commented 3 months ago

uv doesn't (yet) support the pip install -t flag with which you can specify a target directory.

This flag is used by this project to install the requirements into a cache directory or if you've disabled the cache to have the requirements under .serverless/requirements.

Zer0x00 ➜ /project $ uv pip install -t /project/.serverless/requirements -r /project/.serverless/requirements/requirements.txt

error: unexpected argument '-t' found

  tip: to pass '-t' as a value, use '-- -t'

Usage: uv pip install [OPTIONS] <PACKAGE|--requirement <REQUIREMENT>|--editable <EDITABLE>>

For more information, try '--help'.
pgrzesik commented 3 months ago

Thanks for sharing @Zer0x00 🙇 Unless there will be a very high demand for uv-specific support, I think the plugin shouldn't include anything specific for uv if the general plan for it is to be a drop-in replacement for pip with full compatibility.

If anyone is interested in having support for uv sooner, please vote on the issue with 👍 and/or write a comment below 🙇

JP-Ellis commented 2 days ago

This is something I really would like to see supported. I have recently transitioned from using pip to uv pip with some quite amazing speed ups across the board, and it would be great to have support for uv (or perhaps/eventually switch to uv entirely).

I have had success patching serverless-python-requirements manually with the following patch (be warned though, use at your own risk):

diff --git a/node_modules/serverless-python-requirements/lib/pip.js b/node_modules/serverless-python-requirements/lib/pip.js
index 40140d3..bfc75ab 100644
--- a/node_modules/serverless-python-requirements/lib/pip.js
+++ b/node_modules/serverless-python-requirements/lib/pip.js
@@ -147,7 +147,7 @@ async function installRequirements(targetFolder, pluginInstance, funcOptions) {

   try {
     const dockerCmd = [];
-    const pipCmd = [options.pythonBin, '-m', 'pip', 'install'];
+    const pipCmd = ['uv', 'pip', 'install', '--verbose', '--system'];

     if (
       Array.isArray(options.pipCmdExtraArgs) &&
@@ -159,7 +159,10 @@ async function installRequirements(targetFolder, pluginInstance, funcOptions) {
       });
     }

-    const pipCmds = [pipCmd];
+    const pipCmds = [
+      [options.pythonBin, '-m', 'pip', 'install', 'uv'],
+      pipCmd
+    ];
     const postCmds = [];
     // Check if we're using the legacy --cache-dir command...
     if (options.pipCmdExtraArgs.indexOf('--cache-dir') > -1) {
@@ -196,9 +199,9 @@ async function installRequirements(targetFolder, pluginInstance, funcOptions) {
     if (!options.dockerizePip) {
       // Push our local OS-specific paths for requirements and target directory
       pipCmd.push(
-        '-t',
+        '--target',
         dockerPathForWin(targetFolder),
-        '-r',
+        '--requirement',
         dockerPathForWin(targetRequirementsTxt)
       );
       // If we want a download cache...
@@ -226,7 +229,7 @@ async function installRequirements(targetFolder, pluginInstance, funcOptions) {
     // If we are dockerizing pip
     if (options.dockerizePip) {
       // Push docker-specific paths for requirements and target directory
-      pipCmd.push('-t', '/var/task/', '-r', '/var/task/requirements.txt');
+      pipCmd.push('--target', '/var/task/', '--requirement', '/var/task/requirements.txt');

       // Build docker image if required
       let dockerImage;