pulp-platform / snitch_cluster

An energy-efficient RISC-V floating-point compute cluster.
https://pulp-platform.github.io/snitch_cluster/
Apache License 2.0
50 stars 51 forks source link

treewide: Improve docker container build, CI caching and python dependencies #192

Closed fischeti closed 1 month ago

fischeti commented 2 months ago

Docker container

CI

Python dependencies

fischeti commented 1 month ago

Undrafting this again after quite a lot of changes. The PR now touches a lot more than just consolidating python dependencies (see updated description).

colluca commented 1 month ago

I pushed commit 4f89a6f9fb4052b8cceaab57e1a5ab9d530d7795 to automate generation of the dependency list which determines the Verilator model build.

It is based on the List Make Prerequisites Action, which parses Make's database output (from make -pq), to generate the dependency tree for any top-level target (in our case bin/snitch_cluster.vsim). It then lists all leaf nodes of the tree, i.e. all file dependencies the target ultimately depends on, and hashes their contents.

The method is still not 100% correct, e.g. if FESVR_VERSION is changed it will not trigger a cache miss. A solution would be to include the Makefiles themselves in the list of dependencies. The List Make Prerequisites Action could be extended to this end, but probably also the Makefiles would have to be split up to mix the least amount of logic, in order not to trigger unnecessary builds on every Makefile edit.

Nonetheless, the method is at least as precise as the previously proposed method, so I think we can continue with this solution, and eventually refine it in the future, when needed.