ocaml / dune

A composable build system for OCaml.
https://dune.build/
MIT License
1.62k stars 401 forks source link

satisfying a (libraries <x>) dependency with a (rule(target %{cma:<x>}) ...) ? #2871

Closed houndofsound closed 4 years ago

houndofsound commented 4 years ago

this didn't work for me with dune2 but is there a way? is it reasonable? I tried taking a copy of the unix.cma, renaming it, and loading/using it in a toplevel and it worked. I took that to mean the cma is bundling necessary component cmi infos and that was reason enough for it not to complain about lack of unix.cmi.

houndofsound commented 4 years ago

ah, I see that that doesn't hold in general - the cmi files must be available. so a mere %{cma:} isn't right. the reason I was looking for a way to do this was to interpose a rule before the delivery of a library to its dependant.

ghost commented 4 years ago

Could you describe your use case a bit more? In particular, what rule are you trying to interpose?

houndofsound commented 4 years ago

the interposed rule is making a decision about the interface the library presents (without the benefit of an interface file in advance which might fail the build) - code changes may change the cma generated but not its interface and in that event I'd like to prevent triggering an upstream rebuild (via a watched target) by returning the prior version of the library (cached). currently I compile privately and pass the source through for a 2nd build, when necessary, which is wasteful. passing along a completed package would have been tidier.

ghost commented 4 years ago

I feel like what you are trying to do is what dune does already. In particular, if the .cmi files don't change but the .cma changes, reverse library dependencies won't be rebuilt.

houndofsound commented 4 years ago

I muddled my explanation I'm afraid. I'm actually only passing a src file to a library build and I first compile that single src file into a library because that appears to be the most convenient way of a getting a single file compiled (I guess library/executable stanzas are syntactic sugar for a bunch of rules covering byte/native ... and more!). so having built an object file I abandon it (wanting the cmi only) and pass along the src file or an older version, according to some criteria. that way the upstream build can be stopped. my wish was for some way to not abandon the object file but pass it along (or an old version) instead of its src, for use by the library build, which will otherwise duplicate the work.

so my

ghost commented 4 years ago

Do you have a concrete example? It would be easier to reason about this by looking at a concrete repository.

houndofsound commented 4 years ago

unfortunately not yet. the code is a mess and changing again - I was sure I had found the right direction and posted with a view to squeezing out the wrinkles, but now have found a flaw. so all-change again!

so I have closed the 'issue'. but I have a question! the central thing I have been trying to do is keep an interface constraint as long as possible in order to keep dynlink modules loadable. when that doesn't hold any more the action is to remove the constraint entirely and allow the rebuild to happen. but this requires somehow to be able to take the negative of a dependency - in case of failure do something else. I haven' t spotted a way of doing that in the manual yet. or else I need a way to provoke dune-watch to go round the loop again, after I remove the interface constraint, and then it is guaranteed to succeed. are either of those achievable with dune as-is? thanks..

ghost commented 4 years ago

It's not possible to react to the failure of internal build rules. Just to put things in perspective, do I understand correctly that you want to make incremental build fasters? If yes, how much time saving are we talking about here?

houndofsound commented 4 years ago

its about making sensible assurances as to when code gets reloaded. if A B & C comprise a dynlinkable library X and C is itself also made so, with its function stubs left in the library and a counterpart loadable unit, minor changes to C will alter the stubs and cause X to be rebuilt. for instance, the addition of a new function into C that is not yet referenced from outside C. now if A and B were also sub-dynlinkable modules, they would suffer reloading and re-initialisation blowing away state held within themselves. from a user perspective it would be very annoying if A/B setup a GUI, only for such a minor change in C to tear it back down, put it up again. of course one could put some distance between those stubs and the library A/B are part of but it might be a rather artificial /awkward separation. If the C stubs themselves went into a library of sorts, and a dependency analyser existed that could know that C was to become a one-file library but could still work out the deps between A,B and C, then that would put the necessary separation between the C stubs and the A/B components, and the stubs could be reloaded for as long as there was interface compatibility. as it is I currently just withhold the stub file generated, substituting with the old (as exists in the running program) and mli constrain the dynlink sub-unit as long as possible, with the attendant problem of how to coerce dune into going round again after removing that mli constraint (releasing an up-to-date stub file and so allowing the rebuild of the A/B/C library and dynlink sub-unit). I recently posted here about a dune loop I inadvertently invoked seemingly caused by messing with the source tree within an action. that's my best lead at the moment but obviously very hacky. if there are alternative tricks and tips available I would love to hear about them!

ghost commented 4 years ago

I'm finding it difficult to read your last message. Could you please try to make it look nicer? For instance by separating paragraphs, adding titles, code examples, schemas, ... If you cannot do that, that's fine and I'll do an effort on my side, but if you can that would be much appreciated! Thanks

houndofsound commented 4 years ago

I did my best to explain my usage and don't think layout will improve that, but I left an escape clause at the end (in a separate paragraph too!) - I just would like a way to provoke dune-watch into going round the loop again from inside.

ghost commented 4 years ago

What about --auto-promote? That would trigger a rebuild.