Add blocking mode

[ndmitchell]

If Fsatrace could tell me about a file access before it occurred, Shake could do a need before it still built, which would make auto deps much more powerful. Is this feasible? Would require some kind of pipes based protocol probably - you write a line of stdout or to a file, Shake writes something back to say continue.

[ndmitchell]

Or if Fsatrace was a C API

[jacereda]

How would that help with auto deps? Right now I'm notifying after a successful open(), do you mean notifying before the actual open() happens? Something like a callback that allows shake to actually generate the file before the open()?

[ndmitchell]

With the current scheme, I can needed a file (indicate that a file was just used at it's current value). With a blocking API I could instead switch to a need, which is vastly more powerful - and indeed generate the file before the open(), or rebuild it if necessary.

[jacereda]

I still fail to see why needed is less powerful than need. The first time you build, say, a .c file, you don't need the dependencies, the resulting .o is 'dirty' and needs to be generated. After you built it, you invoke needed and get the right dependencies. How does the need approach improve the situation?

[jacereda]

Is the purpose to avoid specifying need dependencies? So a generator doesn't need to specify that it's generating foo.c and let AutoDeps figure it out?

[jacereda]

If this is going to happen I think I prefer to turn it into a library. Otherwise I think the communication overhead can make it a bit slow.

[ndmitchell]

Imagine your .c file includes a .h file that is generated from a .txt file. The first time round, your build fails because the .h file hasn't been generated. A generator must still say what it produces, but now the .c file doesn't have to need the .h file.

[jacereda]

Oh, of course, I messed up the thing. So no more need version.h/config.h/whatever. Sounds good, we can approach it when the thing has proper tests and is working correctly across platforms.

[ndmitchell]

Cool, sounds good. I hope to try and get the tests running sometime this week, and see if there's anything I can do to cover all the kinds of things Shake does (much easier to put the tests in fsatrace so the cause is obvious and you have something to test against).

Note that while a library would make for a nicer and faster interface, it also complicates integrating with a Haskell library, since you really want Shake to be cabal-install'able, and fsatrace requires complex compiler setup that makes that difficult. As a result, an out-of-process API of some description would still be useful, even if it's slower - generally these things are dominated by the time compiling anyway, and communication over pipes (e.g. stdin/stdout) is pretty fast.

[jacereda]

Well, the library could always be opened dynamically if present, the downside being that both apps would be sharing memory (not a small one).

[ndmitchell]

How easy is opening a library dynamically from Haskell? I've done it in C++ apps before, but never tried from Haskell - I guess it involves a separate Windows/Posix bit of code? And does that mean putting C into Shake? However, I'm not against it if it works, and I'm not worried about the sharing of memory - it will remain optional in Shake, although if it does the need thing and does work seamlessly (as it has the potential to do) I could imagine it becoming recommended sooner or later.

[jacereda]

Should be easy, no need to resort to C:

https://hackage.haskell.org/package/unix-2.7.1.0/docs/System-Posix-DynamicLinker.html http://hackage.haskell.org/package/Win32-2.2.2.0/docs/System-Win32-DLL.html

[ndmitchell]

Ok, that's not so bad then.

[ndmitchell]

For info, thinking more, I think the preemptive notification will be exceptionally useful, eg https://github.com/snowleopard/shaking-up-ghc/issues/48#issuecomment-167447745

[jacereda]

I'll convert it to a C API.