search

pantsbuild / scie-pants

Protects your Pants from the elements.
https://www.pantsbuild.org/docs/installation
Apache License 2.0
19 stars 18 forks source link

PANTS_BOOTSTRAP_URLS is not honored by scie-pants #243

Closed kingjamm72 closed 11 months ago

kingjamm72 commented 1 year ago

When the following file is created the scie-pants command does not honor the contents:

{
  "ptex": {
    "cpython-3.8.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz": "file:///home/build/ptex/cpython-3.8.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz",
    "cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz": "file:///home/build/ptex/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz"
  }
}

Where the keys were derived from the SCIE=inspect pants | jq .ptex command. The values for the keys point to a local file system for the ptex runtimes.

After creation of this file running the following command yields the original values

PANTS_BOOTSTRAP_URLS=/home/build/ptex-mapping.json SCIE=inspect pants

{
  "cpython-3.8.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz": "https://github.com/indygreg/python-build-standalone/releases/download/20230507/cpython-3.8.16%2B20230507-x86_64-unknown-linux-gnu-install_only.tar.gz",
  "cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz": "https://github.com/indygreg/python-build-standalone/releases/download/20230507/cpython-3.9.16%2B20230507-x86_64-unknown-linux-gnu-install_only.tar.gz"
}

The pants command in use was downloaded using the following:

curl --proto '=https' --tlsv1.2 -fsSL https://static.pantsbuild.org/setup/get-pants.sh | bash

This was done as of August 15 2023

jsirois commented 1 year ago

The SCIE=inspect ... command just shows the lift manifest embedded in the scie (you could also tail -1 <scie> | jq . to view it) - it makes no attempt to run the scie and display any sort of merged manifest.

That aside, this all works as this session shows:

Setup pants fresh:

$ curl --proto '=https' --tlsv1.2 -fsSL https://static.pantsbuild.org/setup/get-pants.sh | bash
Downloading and installing the pants launcher ...
Installed the pants launcher from https://github.com/pantsbuild/scie-pants/releases/latest/download/scie-pants-linux-x86_64 to /home/jsirois/bin/pants

Running `pants` in a Pants-enabled repo will use the version of Pants configured for that repo.
In a repo not yet Pants-enabled, it will prompt you to set up Pants for that repo.
$ rm -rf ~/.cache/nce

Try using custom URLs, 1st fumble with bad JSON:

$ echo '{}' > test.json
$ PANTS_BOOTSTRAP_URLS=test.json pants
Failed to parse ptex config: missing field `ptex` at line 1 column 2
...

Add the missing key:

$ echo '{"ptex":{}}' > test.json
$ PANTS_BOOTSTRAP_URLS=test.json pants
Did not find an URL mapping for file cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz.
...

Add the missing mapping, but a bad one:

$ echo '{"ptex":{"cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz":"file:///etc/hosts"}}' > test.json
$ PANTS_BOOTSTRAP_URLS=test.json pants
...
ERROR: Failed to establish atomic directory /home/jsirois/.cache/nce/466c13d805cfedffe1debdc484ce9cda9c007101044055b736ba37e4fea056b3/locks/configure-d4c43b4551541032e488fa411bf07aed4e69afb36ca1407e191fc12b89491fce. Population of work directory failed: Failed to establish atomic directory /home/jsirois/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz. Population of work directory failed: The tar.gz destination /home/jsirois/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz of size 410 had unexpected hash: 16183c427758316754b82e4d48d63c265ee46ec5ae96a40d9092e694dd5f77ab
$ wc -c /etc/hosts
410 /etc/hosts
$ sha256sum /etc/hosts
16183c427758316754b82e4d48d63c265ee46ec5ae96a40d9092e694dd5f77ab  /etc/hosts

Etc. In other words, the PANTS_BOOTSTRAP_URLS mechanism works, you just need to supply it good json pointing to good files.

jsirois commented 1 year ago

@kingjamm72 I'm going to close this as answered. It seems like you were looking for a debug command to dump what the scie would do, a sort-of dry run. That feature simply does not exist right now.

wfscheper commented 11 months ago

@jsirois, I'd like to re-open this issue as I'm unable to get the scie-pants binary to respect the env variable.

$ PANTS_BOOTSTRAP_VERSION=report pants
0.10.2
$ sha256sum ~/bin/pants
810c2e8cab1958ef17e7a7638efe5f3754dde50d721e32824a6ca89552c86171  /home/user/bin/pants
$ rm -r ~/.cache/nce
$ echo '{}' >test.json
$ PANTS_BOOTSTRAP_URLS=test.json pants
Bootstrapping Pants 2.17.0
Installing pantsbuild.pants==2.17.0 into a virtual environment at /home/user/.cache/nce/fa6ec1ff473e58cf7dff9577ae94c2bde6bf1c7a837c75b928b414c0195eb80e/bindings/venvs/2.17.0
New virtual environment successfully created at /home/user/.cache/nce/fa6ec1ff473e58cf7dff9577ae94c2bde6bf1c7a837c75b928b414c0195eb80e/bindings/venvs/2.17.0.
No goals specified.
Use `pants help` to get help.
Use `pants help goals` to list goals.

This is a blocker for my team to adopt the scie-pants binary (and subsequently pants 2.18.x), as our builds are done with very restricted network access.

jsirois commented 11 months ago

@wfscheper indeed. The issue is fixed here and the fix should be released very soon: https://github.com/pantsbuild/scie-pants/pull/293

huonw commented 11 months ago

The fix in #293 has been released in 0.10.3: https://github.com/pantsbuild/scie-pants/releases/tag/v0.10.3 . Thanks @kingjamm72 and @wfscheper for flagging this, and @jsirois for identifying the connection!