amonshiz
commented
2 months ago - #161
👈 main
This stack of pull requests is managed by Graphite. Learn more about stacking.
Join @amonshiz and the rest of your teammates on Graphite
Open amonshiz opened 2 months ago
amonshiz
commented
2 months ago 👈mainThis stack of pull requests is managed by Graphite. Learn more about stacking.
Join @amonshiz and the rest of your teammates on Graphite
amonshiz
commented
2 months ago I mean, the change seems to do what it says on the tin. This doesn't really indicate what the VM image changes are - I assume that the CI runs will ensure that the images are good.
That is why I linked to the PR that had the information. Unfortunately there have been a number of changes since the 2024-05-17 and the 2024-06-17 images. I'll try to aggregate those and add to this PR description.
kendalharland
commented
2 months ago I mean, the change seems to do what it says on the tin. This doesn't really indicate what the VM image changes are - I assume that the CI runs will ensure that the images are good.
Fortunately we're not using them just yet. I'll rebase https://github.com/thebrowsercompany/swift-build/pull/159 which will test the images, and once it's mered all future image rolls will be tested and we can safely automate them at some point. This reminds me I should also update the pull_request.paths trigger to include .cirun.yml.
amonshiz
commented
2 months ago I mean, the change seems to do what it says on the tin. This doesn't really indicate what the VM image changes are - I assume that the CI runs will ensure that the images are good.
Fortunately we're not using them just yet. I'll rebase https://github.com/thebrowsercompany/swift-build/pull/159 which will test the images, and once it's mered all future image rolls will be tested and we can safely automate them at some point. This reminds me I should also update the
pull_request.pathstrigger to include.cirun.yml.
Great call! I noticed this failed because of pip not being available for some reason. I expect it is another fall out of the change to how Python 3.9 is installed on arm64 VMs now. https://github.com/thebrowsercompany/arc/pull/27344
amonshiz
commented
2 months ago Ugh. Yeah, that is it. python3 is showing up on the path but not pip.
PS C:\Users\runner> echo $Env:Path
C:\Program Files\PowerShell\7;C:\Program Files\git\usr\bin;C:\Program Files\git\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\Tailscale\;C:\Program Files\WinGet\Links;C:\Program Files\Git\cmd;C:\Program Files\CMake\bin;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\dotnet\;C:\Program Files\Amazon\AWSCLIV2\;C:\Program Files\GitHub CLI\;C:\Users\runner\.cargo\bin;C:\Windows\system32\config\systemprofile\AppData\Local\Microsoft\WindowsApps;C:\Users\runner\AppData\Local\Microsoft\WindowsApps;C:\Users\runner\AppData\Local\Microsoft\WinGet\Links;C:\Users\runner\.dotnet\tools;C:\Program Files\PackageManagement\NuGet\Packages\pythonarm64.3.9.10\tools
PS C:\Users\runner> which python3
/c/Program Files/PackageManagement/NuGet/Packages/pythonarm64.3.9.10/tools/python3
PS C:\Users\runner> python3 --version
Python 3.9.10
PS C:\Users\runner> python3 -m pip --help
Usage:
C:\Program Files\PackageManagement\NuGet\Packages\pythonarm64.3.9.10\tools\python3.exe -m pip <command> [options]
Commands:
install Install packages.
download Download packages.
uninstall Uninstall packages.
freeze Output installed packages in requirements format.
list List installed packages.
show Show information about installed packages.
check Verify installed packages have compatible dependencies.
config Manage local and global configuration.
search Search PyPI for packages.
cache Inspect and manage pip's wheel cache.
index Inspect information available from package indexes.
wheel Build wheels from your requirements.
hash Compute hashes of package archives.
completion A helper command used for command completion.
debug Show information useful for debugging.
help Show help for commands.
General Options:
-h, --help Show help.
--isolated Run pip in an isolated mode, ignoring environment variables and user configuration.
-v, --verbose Give more output. Option is additive, and can be used up to 3 times.
-V, --version Show version and exit.
-q, --quiet Give less output. Option is additive, and can be used up to 3 times (corresponding to WARNING, ERROR, and CRITICAL logging levels).
--log <path> Path to a verbose appending log.
--no-input Disable prompting for input.
--proxy <proxy> Specify a proxy in the form [user:passwd@]proxy.server:port.
--retries <retries> Maximum number of retries each connection should attempt (default 5 times).
--timeout <sec> Set the socket timeout (default 15 seconds).
--exists-action <action> Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.
--trusted-host <hostname> Mark this host or host:port pair as trusted, even though it does not have valid or any HTTPS.
--cert <path> Path to PEM-encoded CA certificate bundle. If provided, overrides the default. See 'SSL Certificate Verification' in pip documentation for more information.
--client-cert <path> Path to SSL client certificate, a single file containing the private key and the certificate in PEM format.
--cache-dir <dir> Store the cache data in <dir>.
--no-cache-dir Disable the cache.
--disable-pip-version-check
Don't periodically check PyPI to determine whether a new version of pip is available for download. Implied with --no-index.
--no-color Suppress colored output.
--no-python-version-warning
Silence deprecation warnings for upcoming unsupported Pythons.
--use-feature <feature> Enable new functionality, that may be backward incompatible.
--use-deprecated <feature> Enable deprecated functionality, that will be removed in the future.Yeah and the NuGet Python install doesn't include any pip executable in that tools directory.
PS C:\Users\runner> dir 'C:\Program Files\PackageManagement\NuGet\Packages\pythonarm64.3.9.10\tools\'
Directory: C:\Program Files\PackageManagement\NuGet\Packages\pythonarm64.3.9.10\tools
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 6/17/2024 10:49 PM DLLs
d----- 6/17/2024 10:49 PM include
d----- 6/18/2024 6:52 PM Lib
d----- 6/17/2024 10:49 PM libs
d----- 6/17/2024 10:49 PM Tools
-a---- 1/17/2022 2:32 PM 25278 LICENSE.txt
-a---- 1/17/2022 2:32 PM 100752 python.exe
-a---- 1/17/2022 2:32 PM 60304 python3.dll
-a---l 6/17/2024 10:49 PM 0 python3.exe
-a---- 1/17/2022 2:32 PM 5529488 python39.dll
-a---- 1/17/2022 2:32 PM 99728 pythonw.exe
-a---- 1/17/2022 2:32 PM 193416 vcruntime140.dll
-a---- 1/17/2022 2:32 PM 56200 vcruntime140_1.dllDang, this change has really been frustrating to work through.
kendalharland
commented
1 month ago Ah, yeah that is annoying. When I get the time I can patch the release binary size workflow to install python instead of assume it comes with the VM, and invoke python3 -m pip explicitly as a workaround, then rebase and merge this PR if you like.
amonshiz
commented
1 month ago That seems like a pretty good work around!
I'm thinking something we can do is also install a wide range of Python versions on the Windows VMs and then just require any workflow that wants to use Python use the setup action we have, and in there on a self-hosted Windows VM it would verify that the desired Python is pre-installed and set the expected venv variables as usual.
kendalharland
commented
1 month ago That sounds like a great idea!
We recently rolled out version 2024.06.17 of the Windows VMs. They moved to more recent Azure OS offerings and implement more complete DataDog Agent observability for the GitHub Runner process.