Compiler not closing import files?

[andyarvanitis]

I believe I'm seeing a bug where import files are being opened but never closed during compilation. As a result, I think my code is crashing due to hitting process limits on open files (the macos ulimit -n defaults to 256).

The initial symptom was simply getting a 'Error: Unable to open file:' on one of my imports. I modified runtime.c to print fopen and fclose debug info:

port_type _Cyc_io_open_input_file(void *data, object str, const char *mode)
{
  const char *fname;
  Cyc_check_str(data, str);
  fname = ((string_type *) str)->str;
  make_input_port(p, NULL, CYC_IO_BUF_LEN);
  printf("-- opening %s", fname);
  p.fp = fopen(fname, mode);
  printf(" -> %p\n", p.fp);
  if (p.fp == NULL) {
    Cyc_rt_raise2(data, "+Unable to open file", str);
  }
  return p;
}

object Cyc_io_close_port(void *data, object port)
{
  Cyc_check_port(data, port);
  {
    FILE *stream = ((port_type *) port)->fp;
    if (stream) {
      printf("-- closing");
      printf(" -> %p\n", stream);
      fclose(stream);
   }
...

Resulting in:

$ cyclone  -COPT -Wno-string-compare -I output output/Main/main.scm 

-- opening output/Main/main.scm -> 0x7fff8ed1eb60
-- closing -> 0x7fff8ed1eb60
-- opening /usr/local/share/cyclone/scheme/base.sld -> 0x7fff8ed1eb60
-- opening /usr/local/share/cyclone/scheme/base.sld -> 0x7fff8ed1ebf8
-- closing -> 0x7fff8ed1ebf8
-- opening /usr/local/share/cyclone/scheme/cyclone/common.sld -> 0x7fff8ed1ebf8
-- opening /usr/local/share/cyclone/scheme/cyclone/common.sld -> 0x7fff8ed1ec90
-- closing -> 0x7fff8ed1ec90
-- opening /usr/local/share/cyclone/cyclone/match.sld -> 0x7fff8ed1ec90
-- opening /usr/local/share/cyclone/cyclone/match.sld -> 0x7fff8ed1ed28
-- closing -> 0x7fff8ed1ed28

... many more sld files (mine), all doubled when opening

And I've counted 253 "closing" occurences before the "Unable to open file" error, which seems to jibe with my theory. Bumping up the process limit delays the failure.

I've dug around the codebase a bit, but unfortunately didn't spot any obvious cause. I'd be happy to look more if you have any suspicions, or if you think I'm on the wrong track.

[justinethier]

Thanks for the detailed bug report @andyarvanitis

At a high level what is going on is that during our code optimization phase the variable in the following call:

(close-input-port fp$211$534)

is being replaced with an inlined call to the open-input-file primitive:

(close-input-port (open-input-file dir$208$533))

Obviously not what we want as this prevents the original port from being closed. Let me take a closer look at the optimizer...

[andyarvanitis]

Thanks!

[justinethier]

Alright, this has been fixed on the master branch. It would probably be easiest to rebuild from source using cyclone-bootstrap and then recompile your code using those binaries.

Thanks again for reporting this @andyarvanitis !

[andyarvanitis]

Awesome, thanks! I'll try it tonight!

[andyarvanitis]

Works great, thanks again