Cache bitstreams

[troibe]

Since building the bitstreams takes quite some time but the resulting files are fairly small it would be nice to cache them. Maybe even the additional build files like logs?

This would help a lot with debugging. The person running into an issue can then just share the bitfile with the a verbose name. Maybe we could have something along the lines of board-name-commit-sha-litex - commit-sha-litex-dependencies -checksum of modified files in case working dir is dirty?

Additionally of course rebuilds are not necessary if the files are already in the cache. In the very long term future maybe those cached bitstreams could even be stored online.

[dayjaby]

If we would create release versions, we could simply attach the bitstreams for the most tested boards to the releases.

For release versions we could create a litex_setup.py with all the exact versions of the dependencies. In one of my repos I already do it that way: https://github.com/dayjaby/litex-experiments/blob/master/litex/litex_setup.py

[troibe]

I was thinking of something along those lines as well. If I recall correctly litex once was a single repo so versioning was pretty straight forward.

In one of my repos I already do it that way: https://github.com/dayjaby/litex-experiments/blob/master/litex/litex_setup.py

Oh interesting up to this point I have been using just git checkout...So what does your workflow look like? Do you change the commit sha in the litex_setup.py then run litex_setup.py then do modifications? If you modify a litex dependency and it's not already upstreamed do you still add the local commit sha to litex_setup.py?

[dayjaby]

Well, I am trying to run different OS like Zephyr and NuttX, which already have some support for litex. However, their drivers expect the CSR locations to stay the same, otherwise they won't work which can cause a lot of headache. If you plan to have a setup to test modifiable software on fixed hardware, fixing all the LiteX versions - for which you know they worked once - in one place is a good way to go.

Do you change the commit sha in the litex_setup.py then run litex_setup.py then do modifications?

Exactly! You can run litex_setup.py update to checkout any updates to the sha.

If you modify a litex dependency and it's not already upstreamed do you still add the local commit sha to litex_setup.py?

I keep a fork from the dependency and modify the litex_setup.py to use my fork, e.g. https://github.com/dayjaby/litex-experiments/blob/29ffbc2bb1f60d1b2a1971f858b99a39b5b34909/litex/litex_setup.py#L27

HOWEVER having nice releases would make this manual adapations of litex_setup.py unnecessary. You could just jump to the next release version if you don't want to fiddle with which combination of litex dependency versions might work well together.

Just an example from another environment that I'm familiar with: ROS (robotic operating system) has several release versions: Kinetic, Lunar, Melodic, Noetic

Each release version runs well on exactly one ubuntu version: Lunar runs on Ubuntu 16, Melodic on Ubuntu 18, Noetic on Ubuntu 20. Interestingly they chose Long Term Support Ubuntu versions.

If you plan to modify an OS or run an entire project on a LiteX-based board, this approach makes sense: Use a very stable LiteX version as a foundation for all your work. "Very stable" could mean that several features like drivers (uart, gpio, spi, ...) are tested on several OS (zephyr, linux, ...) on several boards (arty a7, ...). Another benefit is that a person trying Zephyr OS could just grab a stable release of LiteX and not being exposed to the risk that he clones a master branch where there is still ongoing work.