Have Makefile support toolchain versions

[adumont]

I want the Makefile to support env vars to build specific versions of the tools in the toolchain.

For example, YOSYS_VERSION for Yosys.

It would then do git checkout $YOSYS_VERSION It could be a branch, a tag or a commit ID. If no version specified, checkout master branch.

That combined with travis.yml env sections, it will run a build for every line.

env:

• YOSYS_VERSION=foo BAR=bar
• YOSYS_VERSION=bar BAR=foo

[eine]

@adumont, you might find https://github.com/antonblanchard/ghdl-yosys-blink interesting. It uses docker images provided in ghdl/docker to avoid building and install yosys, nextpnr, etc. locally. His use case is for VHDL sources, but the same Makefile structure and docker images should work for this project.

[adumont]

I build the toolchain, because it's fun, also because it's useful.

The sources and built toolchain are cached in Travis CI, so each build I only rebuild what might have changed in the upstream sources of any of the tools of the toolchain (yosys, arachne-pnr,...). My wish here is to leverage the multi jobs in Travis CI, one build for each environment variable set. So I could have a build with all the latest versions of se in the toolchain, like now, as well as a build with some fixed stable version of the toolchain. This is because sometimes, build fails, not due to my changes, but because of regression bugs in the toolchain.

Re using docker, I have a feeling it's not going to work from within Travis CI environment which is already inside a docker. But not sure.

I'll have a look at this suggestion. Thanks for the feedback

El mié., 5 feb. 2020 1:47, eine notifications@github.com escribió:

@adumont https://github.com/adumont, you might find https://github.com/antonblanchard/ghdl-yosys-blink interesting. It uses docker images provided in ghdl/docker to avoid building and install yosys, nextpnr, etc. locally. His use case is for VHDL sources, but the same Makefile structure and docker images should work for this project.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/adumont/hrm-cpu/issues/50?email_source=notifications&email_token=AACRLWSUANXT7OUNZBS57ZTRBIEDJA5CNFSM4KPN6RFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKZXYYA#issuecomment-582188128, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACRLWWDMHYVPFBV74M6TVDRBIEDJANCNFSM4KPN6RFA .

[eine]

Docker images do work in Travis. Precisely, the main use case of ghdl/docker images is CI environments. Anyway, I misunderstood the purpose of your setup. I thought that those install scripts were for users to install/update the tools locally. Since they are for you to tinker with and learn about CI/Travis, it's ok.

[adumont]

What I like of the idea of using docker images is that it's easy to call a version or another... So maybe I can think of a mixed setup... Interesting

El mié., 5 feb. 2020 8:19, eine notifications@github.com escribió:

Docker images do work in Travis. Precisely, the main use case of ghdl/docker images is CI environments. Anyway, I misunderstood the purpose of your setup. I thought that those install scripts were for users to install/update the tools locally. Since they are for you to tinker with and learn about CI/Travis, it's ok.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/adumont/hrm-cpu/issues/50?email_source=notifications&email_token=AACRLWTB23TFYGNS7M6L6SLRBJSATA5CNFSM4KPN6RFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK2M2OY#issuecomment-582274363, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACRLWXN2FGMLC3BXKYA5KDRBJSATANCNFSM4KPN6RFA .

[eine]

Actually, in ghdl/docker we generate images for multiple distributions and with the different backends supported by GHDL. Then, it is straightforward to test a single design in multiple environments. So, it's what you mean, but with a different variable (distro/backend instead of yosys version). Using docker images is kind of like caching the dependencies. Moreover, you can run locally exactly the same environment that is executed in the CI service. This is very useful for debugging build issues.

The point is that we used to have the CI setup with Travis only, but it was quite limiting because a single YAML file is supported. You can have multiple jobs and stages in a single workflow, but you cannot define multiple independent workflows. Hence, we changed to GitHub Actions (GHA) a few months ago. It is functionally equivalent, but up to 20 different workflows can be triggered and each of them can take up to 6h, instead of Travis' 50min limit. If you are going to investigate creating and consuming multiple docker images, I suggest to use GHA instead.

[adumont]

Thanks for the tip, I didn't know GHA. Sounds interesting!

El mié., 5 feb. 2020 9:00, eine notifications@github.com escribió:

Actually, in ghdl/docker we generate images for multiple distributions and with the different backends supported by GHDL. Then, it is straightforward to test a single design in multiple environments. So, it's what you mean, but with a different variable (distro/backend instead of yosys version). Using docker images is kind of like caching the dependencies. Moreover, you can run locally exactly the same environment that is executed in the CI service. This is very useful for debugging build issues.

The point is that we used to have the CI setup with Travis only, but it was quite limiting because a single YAML file is supported. You can have multiple jobs and stages in a single workflow, but you cannot define multiple independent workflows. Hence, we changed to GitHub Actions (GHA) a few months ago. It is functionally equivalent, but up to 20 different workflows can be triggered and each of them can take up to 6h, instead of Travis' 50min limit. If you are going to investigate creating and consuming multiple docker images, I suggest to use GHA instead.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/adumont/hrm-cpu/issues/50?email_source=notifications&email_token=AACRLWUIJCG6DCSXEIZNP7LRBJWYVA5CNFSM4KPN6RFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK2PWEQ#issuecomment-582286098, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACRLWVMAYPCHABMFCV6ZB3RBJWYVANCNFSM4KPN6RFA .

[adumont]

Implemented somehow in https://github.com/adumont/hrm-cpu/pull/61/commits/191f6b915f09e9d211cded1629b2733847c266ee

env variables that can be set for make:

• DOCKER_YOSYS="adumont/yosys:yosys-0.8"
• DOCKER_YOSYS="adumont/yosys:yosys-0.9"
• DOCKER_YOSYS="ghdl/synth:beta yosys" if DOCKER_YOSYS is unset, will build yosys from source (master)

and ARACHNEPNR=1 . If not set, will use nextpnr-ice40 docker image by ghdl project (ghdl/synth:nextpnr-ice40)

[eine]

Great! I'm glad you decided to optionally use ghdl images.

An additional hint: arachne-pnr is deprecated (https://github.com/YosysHQ/arachne-pnr#arachne-pnr). However, I know from experience that building nextpnr is not as straightforward. You can use https://github.com/ghdl/docker/blob/master/dockerfiles/cache_pnr#L116-L129 (which is the dockerfile we use to build ghdl/docker:nextpnr* images) as a reference to build nextpnr locally (from source). That's for Debian Buster, so it should be similar on other Debian-based distros. That makefile is slighly complex, and you need to follow the chain of stages to gather all the required dependencies. Should you need any help, please let me know.

[adumont]

Yeah, saw that, nextpnr takes a long time to build though... Plus the image is huge. I tried quickly to build it on alpine, but failed. IIRC I couldn't satisfied the eigen3 requirement... So I decided to use yours instead.

Thanks for the tips

El mié., 12 feb. 2020 1:12, eine notifications@github.com escribió:

Great! I'm glad you decided to optionally use ghdl images.

An additional hint: arachne-pnr is deprecated ( https://github.com/YosysHQ/arachne-pnr#arachne-pnr). However, I know from experience that building nextpnr is not as straightforward. You can use https://github.com/ghdl/docker/blob/master/dockerfiles/cache_pnr#L116-L129 (which is the dockerfile we use to build ghdl/docker:nextpnr* images) as a reference to build nextpnr locally (from source). That's for Debian Buster, so it should be similar on other Debian-based distros. That makefile is slighly complex, and you need to follow the chain of stages to gather all the required dependencies. Should you need any help, please let me know.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/adumont/hrm-cpu/issues/50?email_source=notifications&email_token=AACRLWWPQAYXVUE6ZZX56UDRCM5G7A5CNFSM4KPN6RFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELOWBWA#issuecomment-584933592, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACRLWWB4Y5RNEQYMKZTE4LRCM5G7ANCNFSM4KPN6RFA .

[eine]

That's interesting indeed. If you ever get to build it on alpine, please, ping me. Most of "extra" ghdl docker images are based on Debian Buster because we cannot ensure that all the tools work on alpine. However, should it be true some day, I'd like to change.