various files which are supposedly installed into the package are not visible when testing the package

[ethanc8]

(continued from https://discord.com/channels/1082332781146800168/1264423283919687722/1265375809015910420)

I am building the package llvm-spirv from my feedstock.

now it's failing in testing with:

 │ │ + $PREFIX/bin/llvm-spirv -help
 │ │ /tmp/.tmpsuRFVk/conda_build.sh: line 10: $PREFIX/bin/llvm-spirv: No such file or directory

but earlier it put bin/llvm-spirv into the package:

 │ │ Files in package:
 │ │   - bin/llvm-spirv
 │ │   - info/about.json
 │ │   - info/hash_input.json
 │ │   - info/index.json
 │ │   - info/licenses/LICENSE.TXT
 │ │   - info/paths.json
 │ │   - info/recipe/LICENSE.txt
 │ │   - info/recipe/build.sh
 │ │   - info/recipe/build_install.sh
 │ │   - info/recipe/install.sh
 │ │   - info/recipe/recipe.yaml
 │ │   - info/recipe/rendered_recipe.yaml
 │ │   - info/recipe/variant_config.yaml
 │ │   - info/tests/0/run_test.sh
 │ │ Creating target folder "/home/runner/work/chipstar-forge-feedstocks/chipstar-forge-feedstocks/output/linux-64"
 │ │ Compressing archive...
 │ │ Archive written to "/home/runner/work/chipstar-forge-feedstocks/chipstar-forge-feedstocks/output/linux-64/llvm-spirv-17.0.2-h44954ab_0.conda"

You can see the full build log at https://github.com/ethanc8/chipstar-forge-feedstocks/actions/runs/10063290370/job/27817743293.

[wolfv]

Regarding this bug - I think we should either make the symlinks relative when writing files to the cache, or replace the prefix when restoring the files.

[ethanc8]

Regarding this bug - I think we should either make the symlinks relative when writing files to the cache, or replace the prefix when restoring the files.

wait, is this related to the cache? I thought it was completely separate. Did the packager install the symlink to the cache instead of installing the actual binary?

In my test script, I might just run ls -l

[wolfv]

Yeah, so we create the cache in some directory, let's say foo, and then move it into a storage location. If there were absolute symlinks, they still reference foo - we need to remove that absolute prefix and replace with the current build prefix. I'll have a fix shortly :)

[wolfv]

I don't know if it's easy to test for you, but this might fix it: https://github.com/prefix-dev/rattler-build/pull/993

[wolfv]

There is at least one more issue - the file / symlink doesn't appear under paths.json. I am taking a look (testing with your recipes, btw).

[wolfv]

OK, with my latest push, it seems to build :)

[ethanc8]

Is there a way to use the main branch with your rust-forge GitHub Action?

[ethanc8]

Ok, I got your branch to work with GitHub Actions, and it got past building libllvmspirv17, but now I get (in the next package):

 ╭─ Running build for recipe: libllvmspirv-17.0.2-h36e50f3_0
 │
 │ ╭─ Fetching source code
 │ │ Validated SHA256 values of the downloaded file!
 │ │ Found valid source cache file.
 │ │ Using extracted directory from cache: "/home/runner/work/chipstar-forge-feedstocks/chipstar-forge-feedstocks/output/src_cache/v17_0_2_e90bd99b"
 │ │ Copying source from url: "/home/runner/work/chipstar-forge-feedstocks/chipstar-forge-feedstocks/output/src_cache/v17_0_2_e90bd99b" to "/home/runner/work/ch
 │ │ ipstar-forge-feedstocks/chipstar-forge-feedstocks/output/bld/rattler-build_libllvmspirv_1722223731/work"
 │ │
 │ ╰─────────────────── (took 0 seconds)
 │
 │ ╭─ Running cache build
 │ │ Cache key: "82aeb19a2c14e2fef0904646420786e7319d81ca02633bdfc0dd17abae1bd791"
 │ │ Restoring cache from "/home/runner/work/chipstar-forge-feedstocks/chipstar-forge-feedstocks/output/build_cache/bld_82aeb19a2c14e2fef0904646420786e7319d81ca
 │ │ 02633bdfc0dd17abae1bd791"
 │ │ Restoring from cache: "include/LLVMSPIRVLib/LLVMSPIRVLib.h"
 │ │ Restoring from cache: "include/LLVMSPIRVLib/LLVMSPIRVExtensions.inc"
 │ │ Restoring from cache: "lib/libLLVMSPIRVLib.so"
Creating symlink from "/home/runner/work/chipstar-forge-feedstocks/chipstar-forge-feedstocks/output/bld/rattler-build_libllvmspirv_1722223731/host_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/libLLVMSPIRVLib.so" to "libLLVMSPIRVLib.so.17"
 │ │
 │ ╰─────────────────── (took 0 seconds)
 │
 ╰─────────────────── (took 0 seconds)
 × error Error building package: failed to open file `/home/runner/work/chipstar-forge-feedstocks/chipstar-forge-feedstocks/output/bld/rattler-build_libllvmspirv
 × error _1722223731/host_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib
 × error /libLLVMSPIRVLib.so`
Current symlink target: "libLLVMSPIRVLib.so.17"
Didn't change symlink target
Error:   × failed to open file `/home/runner/work/chipstar-forge-feedstocks/chipstar-
  │ forge-feedstocks/output/bld/rattler-build_libllvmspirv_1722223731/
  │ host_env_placehold_placehold_placehold_placehold_placehold_placehold_place
  │ hold_placehold_placehold_placehold_placehold_placehold_placeho/lib/
  │ libLLVMSPIRVLib.so`
  ╰─▶ No such file or directory (os error 2)

This definitely seems related. What's going on here is that libLLVMSPIRVLib.so, in the package libllvmspirv, is a symlink to libLLVMSPIRVLib.so.17, in the package libllvmspirv17.

[ethanc8]

It looks like the " various files which are supposedly installed into the package are not visible when testing the package" has been solved; the issue with symlinks between packages is tracked in #994.