Compiler: Error caused by cache file name too long

[gofenix]

hello, when i use scry in vscode.

there are some errors like this:

[Error - 11:46:00 AM] An error was found while searching diagnostics
Unexpected char 'U' at 1:1
Unable to get file info: '/Users/lucas/Documents/demos/crystals/scry/Users/lucas/.cache/crysta/Users-lucas-Documents-demos-crystals-scry-Users-lucas-.cache-crysta-Users-lucas-Documents-demos-crystals-scry-Users-lucas-.cache-crysta-Users-lucas-Documents-demos-crystals-scry-Users-lucas-.cache-crysta-Users-lucas-Documents-demos-crystals-scry-src-scry.cr': File name too long (File::Error)
  from raise<File::Error+>:NoReturn
  from Crystal::System::File::info?<String, Bool>:(Crystal::System::FileInfo | Nil)
  from Dir::mkdir_p<String>:Nil
  from Crystal::CacheDir#directory_for<String>:String
  from Crystal::Compiler#new_program<Array(Crystal::Compiler::Source)>:Crystal::Program
  from Crystal::Compiler#compile<Array(Crystal::Compiler::Source), String>:Crystal::Compiler::Result
  from Crystal::Command#run:(Bool | Nil)
  from __crystal_main
  from main
Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/crystal-lang/crystal/issues

and this is my crystal info:

[11:48:38] lucas:scry git:(master*) $ brew info crystal   
crystal: stable 0.35.1 (bottled), HEAD
Fast and statically typed, compiled language with Ruby-like syntax
https://crystal-lang.org/
/usr/local/Cellar/crystal/0.35.1_1 (1,123 files, 58.4MB) *
  Poured from bottle on 2020-07-11 at 23:03:59
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/crystal.rb
License: Apache-2.0
==> Dependencies
Build: autoconf ✔, automake ✔, libatomic_ops ✘, libtool ✔
Required: gmp ✔, libevent ✔, libyaml ✔, llvm ✔, openssl@1.1 ✔, pcre ✔, pkg-config ✔
==> Options
--HEAD
        Install HEAD version
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Analytics
install: 2,822 (30 days), 5,165 (90 days), 20,500 (365 days)
install-on-request: 2,596 (30 days), 4,725 (90 days), 18,538 (365 days)
build-error: 0 (30 days)

hope your help, thanks

[straight-shoota]

Path lengths on macOS are quite limiting, unfortunately.

I can't tell what exactly causes this without further information, but the very long last path component (Users-lucas-Documents-demos-crystals-scry-Users-lucas-.cache-crysta-Users-lucas-Documents-demos-crystals-scry-Users-lucas-.cache-crysta-Users-lucas-Documents-demos-crystals-scry-Users-lucas-.cache-crysta-Users-lucas-Documents-demos-crystals-scry-src-scry.cr) indicates that the compiler is asked to compile a file that exists at the path that you get when replacing all the - with /. This already long path is not generated by the compiler. The compiler only prepends the path to the temp dir, which makes it exceed the file limit.

So to me it seems like the problem originates somewhere else, maybe scry produces such a long file name?

Still, the compiler should be able to handle this better. At the very least, such a file system error should not be indicated as a compiler bug. It should be a normal compiler error (wrap the error in Crystal::Error). As an additional enhancement, we could consider mitigating the problem by shortening paths for cache files if they threaten to exceed the maximum file name length. Replacing (part of) the cache key by a hash should work for this.

Reproduction is simple by setting CRYSTAL_COMPILER_PATH to a ridiculously long path (4K on linux, 1K on mac) and compiling a file. (NOTE: In that case the compiler should genuinely fail and not try to recover in some way.)

[straight-shoota]

I just noticed that the Program#cache_dir property (which is getting populated when the error occurs) appears to be not actually used anywhere.

$ grep -R cache_dir src/compiler/
src/compiler/crystal/program.cr:    setter cache_dir : String?
src/compiler/crystal/compiler.cr:      program.cache_dir = CacheDir.instance.directory_for(sources)

Removing that doesn't fix the problem though, it just appears later in the codegen stage.