"No such file or directory" errors in `toolchain/autoupdate_testdata.py`

geoffromer commented 1 month ago

Description of the bug:

Under certain conditions, toolchain/autoupdate_testdata.py fails with a cascade of "No such file or directory" errors.

What did you do, or what's a simple way to reproduce the bug?

$ git clone https://github.com/carbon-language/carbon-lang
$ cd carbon-lang/
$ git reset --hard 187a3608df
$ bazel build //toolchain
$ git reset --hard HEAD^
$ toolchain/autoupdate_testdata.py

187a3608df is the commit hash of #4209. Other before/after commit pairs would also work, so long as a prelude file is present when bazel build //toolchain is run, but absent when toolchain/autoupdate_testdata.py is run. Note that you might need to clear your Bazel cache to get these steps to work, because non-hermetic Bazel behavior seems to be part of the problem.

What did you expect to happen?

I expected the script to run successfully.

What actually happened?

The script produced an error for each testdata file:

Executing tests from //toolchain/testing:file_test
-----------------------------------------------------------------------------

Error updating toolchain/check/testdata/alias/no_prelude/alias_of_alias.carbon: No such file or directory

Error updating toolchain/check/testdata/array/assign_return_value.carbon: No such file or directory

Error updating toolchain/check/testdata/array/base.carbon: No such file or directory

Error updating toolchain/check/testdata/alias/no_prelude/fail_local_in_namespace.carbon: No such file or directory

Error updating toolchain/check/testdata/alias/no_prelude/export_name.carbon: No such file or directory

Error updating toolchain/check/testdata/array/array_in_place.carbon: No such file or directory

Error updating toolchain/check/testdata/alias/fail_bool_value.carbon: No such file or directory

Error updating toolchain/check/testdata/array/nine_elements.carbon: No such file or directory

Error updating toolchain/check/testdata/as/fail_no_conversion.carbon: No such file or directory

Error updating toolchain/check/testdata/array/fail_bound_negative.carbon: No such file or directory

... and so on.

Any other information, logs, or outputs that you want to share?

I updated this line to log the name of the file, and got the following more illuminating error message:

Error updating toolchain/parse/testdata/struct/fail_dot_string_colon.carbon: Unable to open /usr/local/google/home/gromer/.cache/bazel/_bazel_gromer/6b556be8ddbdbba287a2e1ef86f733d5/execroot/_main/bazel-out/k8-fastbuild/bin/toolchain/install/prefix_root/lib/carbon/../../lib/carbon/core/prelude/operators/as.carbon: No such file or directory

The file in question is a symlink pointing to my git client's core/prelude/operators/as.carbon, which of course doesn't exist, hence the error.

geoffromer commented 1 month ago

I should note, I think there are two separate bugs here:

The fact that toolchain/autoupdate_testdata.py can fail in a clean client.
The fact that the error messages misleadingly suggest that it can't find the testdata files, when the problem is actually a prelude file.

jonmeow commented 1 month ago

From what I can tell of the repro, you do a build, then remove artifacts used as input to the build. bazel doesn't notice the removed artifacts (the prelude files) due to the symlinks.

I think the actual issue is with symlink_filegroup, since the dependency is on the symlink rather than the data file. That's toolchain/install/BUILD. I'll think about if there's another approach there, I know @chandlerc was trying to work around a few issues.

Note, I'd typically define a "clean client" repro as:

git clone https://github.com/carbon-language/carbon-lang
cd carbon-lang/
toolchain/autoupdate_testdata.py

A clean client works fine for me. It's removing prelude files that's not detected.

Note, a simple repro here is to then do:

rm core/prelude/operators/as.carbon
toolchain/autoupdate_testdata.py

bazel should catch this but doesn't.

carbon-language / carbon-lang