Avoiding recompilation when dependencies are unchanged?

[nmoroze]

Hi,

I've just gotten started with FuseSoC and noticed that at least with the Verilator and Icestorm backends, it seems to always recompile the simulator binary/bitstream if the build command is called, even if dependencies haven't changed. I was wondering if it's in scope for FuseSoC to have a feature that prevents this recompilation?

If this is something that would be desirable, I'd be happy to try to contribute a PR if it's something that seems doable. Another possibility I had considered is instead of having FuseSoC handle this responsibility itself, perhaps it could have a command for dumping all dependencies to a file that could be used by a Make wrapper that only calls FuseSoC if those dependencies are out of date (something like .d files sometimes used with Make for C/C++ projects).

Thanks!

[imphil]

Thanks for opening this issue, looks like we didn't have one on that topic yet -- which unfortunately doesn't mean nobody has thought about it :smile: The topic you're raising has been a long-standing issue with FuseSoC and does require a fair amount of work to solve, which we are step by step working on.

• Some dependencies on e.g. environment variables or on external cores are visible in FuseSoC. FuseSoC therefore needs to keep track of them and invalidate the build as necessary.
• All tools are called in edalize: some through Makefiles, others directly from Python, etc. To avoid calling tools if that isn't needed, we first need better dependency tracking in edalize.

It's a large task, and one that requires a fair amount of refactoring in various parts of the code base to get it right -- and it needs to be right: unless we're certain that we're tracking all dependencies, we need to stay conservative and rebuild everything.

So if you're interesting in helping out we can get you started. The first areas of work are in edalize, likely around its ninja integration.

If you're only looking for a reasonably good solution today that writes out a dependency file to be used in a higher-level Makefile (which calls fusesoc), you can have a look at this patch: https://github.com/lowRISC/fusesoc/commit/1fb8c939a0fb165bb1ca96233bcc7fc9b6e78369 (which is used in the FuseSoC version we use in OpenTitan).

[nmoroze]

Gotcha, I figured this feature might be a bit of a project! The OpenTitan patch seems like it could be useful for my purposes - thanks for pointing me towards that.

I'm definitely interested in contributing and have some bandwidth to do so in the short term, although that bandwidth might go away fairly soon. If the edalize groundwork is self-contained I'd be happy to take a crack at it, but I don't think I can promise being able to help see the whole end-to-end feature through. Given that, I totally understand if there'd be too much overhead in me getting familiar with the codebase etc. for it to be worthwhile.

In any case, is it worth keeping this issue open for tracking?

[olofk]

Surprise! This is now supported in the new Flow API. Most of the legacy backends are not yet ported though (help appreciated!) but we're getting there