Do the commit check before setting up workdir

JonCholas commented 1 year ago

Hello!

Vote on this issue by adding a 👍 reaction
If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

Today, pulumi operator checks if the commit of a stack has changed after creating the working directory. Creating the working directory involves installing all the dependencies just to check a commit and stop there because the commit hasn't changed.

This is a waste of resources and time as when we want to update the stack, we will not see the pulumi operator doing anything until it has finished preparing the working directory for the previous execution.

In the logs(at the bottom), you can see how it

Reconcilling Stack 2023-05-29T03:54:10.676Z
Commit check 2023-05-29T03:55:00.273Z

It takes a minute to check a commit, and is doing it every minute so there is a high chance that everytime we change a config/commit we have to wait up to a minute for the previous run of pulumi check to finish.

Currently you are calling:

revision, err := revisionAtWorkingDir(w.WorkDir())

And then not returning it until the workdir is setup, and therefore, dependencies installed

return revision, sess.setupWorkspace(ctx, w)

Can we move the creation of workdir outside and check the revision earlier?

2023-05-29T03:54:10.676Z - Reconciling Stack - -  -
2023-05-29T03:54:10.696Z - Setting up pulumi workdir for stack - -  -
2023-05-29T03:54:12.107Z - Using local stack - -  -
2023-05-29T03:54:15.483Z - Setting autostack - -  -
2023-05-29T03:54:18.693Z - Initial autostack config - -  -
2023-05-29T03:54:18.694Z - stackConfig loaded - -  -
2023-05-29T03:54:21.984Z - Updated stack config - -  -
2023-05-29T03:54:21.984Z - InstallProjectDependencies - -  -
2023-05-29T03:54:25.880Z - Pip Install - Requirement already satisfied: pip in /tmp/pulumi-working/beta-neara-f515a5a8/deployment-beta-beta/pulumi/venv/lib/python3.9/site-packages (22.0.4)-  -
2023-05-29T03:54:26.012Z - Pip Install - Collecting pip-  -
2023-05-29T03:54:26.021Z - Pip Install -   Using cached pip-23.1.2-py3-none-any.whl (2.1 MB)-  -
2023-05-29T03:54:26.032Z - Pip Install - Requirement already satisfied: setuptools in /tmp/pulumi-working/beta-neara-f515a5a8/deployment-beta-beta/pulumi/venv/lib/python3.9/site-packages (58.1.0)-  -
2023-05-29T03:54:26.349Z - Pip Install - Collecting setuptools-  -
2023-05-29T03:54:26.354Z - Pip Install -   Using cached setuptools-67.8.0-py3-none-any.whl (1.1 MB)-  -
2023-05-29T03:54:26.411Z - Pip Install - Collecting wheel-  -
2023-05-29T03:54:26.413Z - Pip Install -   Using cached wheel-0.40.0-py3-none-any.whl (64 kB)-  -
2023-05-29T03:54:26.520Z - Pip Install - Installing collected packages: wheel, setuptools, pip-  -

....

2023-05-29T03:54:32.967Z - Pip Install - Collecting certifi>=2017.4.17 (from requests<3.0,>=2.21->pulumi-kubernetes<4.0.0,>=3.0.0->pulumi-neara==0.1.0->-r requirements.txt (line 1))-  -
2023-05-29T03:54:32.970Z - Pip Install -   Using cached certifi-2023.5.7-py3-none-any.whl (156 kB)-  -
2023-05-29T03:54:33.371Z - Pip Install - Installing collected packages: types-PyYAML, arpeggio, urllib3, typing-extensions, tomli, six, semver, pyyaml, pyflakes, pycodestyle, protobuf, platformdirs, pathspec, packaging, mypy-extensions, mccabe, isort, idna, grpcio, dill, click, charset-normalizer, certifi, attrs, requests, pulumi, parver, mypy, flake8, black, pulumi-random, pulumi-kubernetes, pulumi-gcp, pulumi-aws, pulumi-eks, pulumi-neara-  -
2023-05-29T03:54:59.447Z - Pip Install -   Running setup.py develop for pulumi-neara-  -
2023-05-29T03:54:59.874Z - Pip Install - Successfully installed arpeggio-2.0.0 attrs-23.1.0 black-23.1.0 certifi-2023.5.7 charset-normalizer-3.1.0 click-8.1.3 dill-0.3.6 flake8-6.0.0 grpcio-1.51.3 idna-3.4 isort-5.12.0 mccabe-0.7.0 mypy-1.2.0 mypy-extensions-1.0.0 packaging-23.1 parver-0.4 pathspec-0.11.1 platformdirs-3.5.1 protobuf-4.23.2 pulumi-3.68.0 pulumi-aws-5.41.0 pulumi-eks-1.0.2 pulumi-gcp-6.56.0 pulumi-kubernetes-3.28.1 pulumi-neara-0.1.0 pulumi-random-4.13.2 pycodestyle-2.10.0 pyflakes-3.0.1 pyyaml-6.0 requests-2.31.0 semver-2.13.0 six-1.16.0 tomli-2.0.1 types-PyYAML-6.0.0 typing-extensions-4.6.2 urllib3-2.0.2-  -
2023-05-29T03:55:00.273Z - Checking current HEAD commit hash - -  -
2023-05-29T03:55:00.273Z - Commit hash unchanged. Will poll again. - -  -

Affected area/feature

rquitales commented 1 year ago

Hi @JonCholas, thank you for reporting this enhancement. Your suggestion to reduce unnecessary computations and network calls sounds like a reasonable change that could greatly benefit the project. We appreciate your input and will add this enhancement to our backlog for further evaluation.

If you're interested, we would also welcome any contributions you may have to speed up the implementation of this enhancement. We value collaboration and would be glad to work with you in landing a pull request that addresses this feature. Please feel free to reach out if you have any questions or need assistance along the way.

diablo39 commented 12 months ago

Idea looks fantastic! When we can expect implementation?

pulumi / pulumi-kubernetes-operator