Add Ultra96 and upgrade Vivado

[the-snowwhite]

@dkhughes @cdsteinkuehler Adding the Ultra96V1 and newer Ultra96V2 to the mksocfpga hardware nest requires an update of the Vivado (at least to 2018.2) version This includs all current Xilinx projects to keep things as simple as possible.

---

I would like to propose settling for (the latest)Version 2019.1 as this (as something new) includes license free partial re-configuration (PR) which I certainly would like to play with... I already have the Ultra96 up and running in the 2019.1 environment(rt-patched).

machinekit@mksocfpga-ultra96-soc:~$ uname -a
Linux mksocfpga-ultra96-soc 4.19.0-rt1 #1 SMP PREEMPT RT Sat Sep 14 22:25:13 UTC 2019 aarch64 GNU/Linux

I have a Vivado 2019.1 Dockerimage (that can run the full gui) this I have (so far) managed to reduce in size to "only" 26 GB as an starting point.

[dkhughes]

I messed around with this a little while back but the large image size (48GB) pushed me away temporarily. If you have gotten it down to a more manageable 26GB, I'd be happy to address the project generation stuff for zynq. I imagine the ultrascale stuff is similar as well.

[the-snowwhite]

Thanks that would be great I've pushed it to dockerhub I's just as large as downloading the offline installer :-)

---

So fat I've only been able to upload a raw Ultra96 project and export the project and bd tcl files.

[the-snowwhite]

@dkhughes what are the RATES[4..0] signals used for ? In the quartus projects they are omitted and omitting them in the Vivado projects would give 5 I/O's more...

[dkhughes]

RATES is related to the DPLL and interrupt driven (waithread) stuff I use in some designs. @mhaberler and I were using it to sync the main thread to an external timing source to reduce jitter.

[the-snowwhite]

oh I remember a bit now, unfortunately @mhaberler took his leave while I was recovering so I failed to notice any conclusion of the irq work, but its beneficial ?

[the-snowwhite]

I have now created a more final fresh project to a fresh ultra96-raw branch here The make_bitfile.sh process is working (modded for ultra96).(no dts or bootgen stuff)

[the-snowwhite]

So far the ultra96 is confined to 1 pin file: The Ultra96 boot setup process is to: generate a BOOT.BIN containing u-boot, arm-firmware and the bitfile (with petalinux) generate a image.ub containing the linux kernel and the devicetree (with mkimage)

Place the 2 files in the boot partition, and the fpga is programmed from boot.

---

I have not put any effort into seeing if its possible to boot bitfile-less (on a linux desktop). I recon it will pay more off to look into having a separate PR file for each config and I expect the current hm2_soc.ol dts overlay implementation only needs a slight change to the dts file template to load partial bitfiles.

[the-snowwhite]

Ok I have now finalized the initial addition of Ultra96 V1 and V2 in the new branch ultra96 and deleter all the other (work) branches.

[the-snowwhite]

Have just finished upgrading the zynq projects to 2019.1 in this branch and running full build now locally.

[the-snowwhite]

OK @dkhughes Got the build.vivado.sh to run fully in the new thesnowwhite/vivado-docker in this branch Is it possible for you to test if the zynq bitfiles work correctly ?

[dkhughes]

Yes, I will try it as soon as I can find a little time. Downloading the 25GB file failed and I didn't look into why, but I should be free this week to figure it out.

[the-snowwhite]

OK I updated the docker image the other day perhaps you tried pulling while I was updating the image

[the-snowwhite]

Btw by downloading do you mean:

docker pull thesnowwhite/bionic-vivado:2019.1

?

[ArcEye]

@the-snowwhite The latest mksocfpga build failed for vivado. I checked the server and the arceye/vivado docker is missing, AGAIN!

The last time this happened it took months for @dkhughes and I to get a working docker back up there, don't know what is happening or who is/how it is getting deleted.

I should have the backup copy, but if you are going to replace it, I will wait until that replacement is ready and change the settings to pull that.

Meanwhile the quartus packages are building but the vivado are not, will have to do a seperate vivado build of the last 2 PRs once sorted.

[ArcEye]

As an interim measure, I have found the docker image backup and am trying to push it to dockerhub temporarily.

If I can do that, the builder will pull it and then can complete the builds for vivado. About 1/10th through the 25GB so far, with 3 false starts where dockerhub errored and went back to the beginning again :(

[the-snowwhite]

@ArcEye I somehow didn't get notifications of your recent messages here. The new vivado docker is "only" 26Gb in total (I think the former is around 48GB) Since it does not require a license I have uploaded it to dockerhub I have also tried testing automated building from the linked github repo which failed only due to the missing Vivado install file.

Uploading the 26 Gig install file to a github account is not an option, and placing the valid download link (wget method) may be is problematic ? as you have to sign in and press yes to an U.S. Government Export Approval before getting the download link:

U.S. Government Export Approval
U.S. export regulations require that your First Name, Last Name, Company Name and Shipping 
Address be verified before Xilinx can fulfill your download request.  Please provide accurate and complete information. 
Addresses with Post Office Boxes and names/addresses with Non-Roman Characters with accents such as grave, tilde or colon are not supported by US export compliance systems.

[ArcEye]

I have now switched to a direct sftp upload to the server of the old image ( it is 24GB or 14.3GB in a gzip ), dockerhub kept failing part way through and restarting from zero. :sob:

Once yours is tested and available in dockerhub, pulling it will not be the problem, but the vivaldo export stuff may be. Have to wait and see.

[the-snowwhite]

Well so far I haven't worked out a solution for how to program the U96 fpga other than baking it into the BOOT.BIN file that is delivered in the boot folder of the sd-image(Xilinx method). And starting machinekit with the no-fw method So for the first U96 iteration I haven't looked mush at the vivaldo export stuff as it's not strictly needed to go anywhere yet....

[ArcEye]

I managed to get the vivado:2015.4-jessie docker image back onto the server eventually and installed However running it results in this error

09:26:14 $ docker run --tty --detach --workdir /var/lib/jenkins/workspace/mksocfpga-vivado --volume /var/lib/jenkins:/var/lib/jenkins:rw --volume $WORKSPACE:/work:rw --volume /var/lib/jenkins/workspace/mksocfpga-vivado:/work:rw --volume /tmp:/tmp:rw --net bridge --add-host dockerhost:172.17.0.1 --env BUILD_CAUSE=MANUALTRIGGER --env BUILD_CAUSE_MANUALTRIGGER=true --env BUILD_DISPLAY_NAME=#52 --env BUILD_ID=52 --env BUILD_NUMBER=52 --env BUILD_TAG=jenkins-mksocfpga-vivado-52 --env "BUILD_TIMESTAMP=2019-10-03 10:26:08 CEST" --env BUILD_URL=https://jenkins.machinekit.io/job/mksocfpga-vivado/52/ --env CLASSPATH= --env EXECUTOR_NUMBER=1 --env GIT_AUTHOR_EMAIL=jenkins@machinekit.io --env "GIT_AUTHOR_NAME=Machinekit Jenkins Git user" --env GIT_BRANCH=origin/master --env GIT_COMMIT=b0a8492dcee8aae947ed7b82f5604dfb0840d157 --env GIT_COMMITTER_EMAIL=jenkins@machinekit.io --env "GIT_COMMITTER_NAME=Machinekit Jenkins Git user" --env GIT_PREVIOUS_COMMIT=b0a8492dcee8aae947ed7b82f5604dfb0840d157 --env GIT_PREVIOUS_SUCCESSFUL_COMMIT=fdc9ddc3a42cc2b6f4a1302ef8e6ca8fc23a4bc0 --env GIT_URL=https://github.com/machinekit/mksocfpga.git --env GITCACHE=/var/cache/gitcache --env HUDSON_HOME=/var/lib/jenkins --env HUDSON_SERVER_COOKIE=a148cdc1ba063112 --env HUDSON_URL=https://jenkins.machinekit.io/ --env JENKINS_SERVER_COOKIE=a148cdc1ba063112 --env JENKINS_URL=https://jenkins.machinekit.io/ --env JOB_BASE_NAME=mksocfpga-vivado --env JOB_DISPLAY_URL=https://jenkins.machinekit.io/job/mksocfpga-vivado/display/redirect --env JOB_NAME=mksocfpga-vivado --env JOB_URL=https://jenkins.machinekit.io/job/mksocfpga-vivado/ --env "NODE_LABELS=amd64 debian docker jessie mah.priv.at master" --env NODE_NAME=master --env ROOT_BUILD_CAUSE=MANUALTRIGGER --env ROOT_BUILD_CAUSE_MANUALTRIGGER=true --env RUN_CHANGES_DISPLAY_URL=https://jenkins.machinekit.io/job/mksocfpga-vivado/52/display/redirect?page=changes --env RUN_DISPLAY_URL=https://jenkins.machinekit.io/job/mksocfpga-vivado/52/display/redirect --env WORKSPACE=/var/lib/jenkins/workspace/mksocfpga-vivado arceye/vivado:2015.4-jessie /bin/cat

09:26:16 docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"/bin/cat\": stat /bin/cat: no such file or directory": unknown.
09:26:16 FATAL: Failed to run docker image

@dkhughes @the-snowwhite Can you shed light on why a docker would start with a /bin/cat command with no targets or why this is unable to be found?

When I try to run the docker locally I get a similar error which suggests a more systemic problem beyond the actual command it starts with.

docker run --rm -it --privileged -v $PWD:$PWD -w $PWD arceye/vivado /bin/bash
docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown.

So at present no vivado packages.

[the-snowwhite]

The only idea that pops up here is: data corruption on the server (failing hdd/sdd) ?

[ArcEye]

Unlikely, happens on the server and my local machine, plus the file being looked for is within the docker, not in the server.

It is possible that my copy of the docker was corrupted in storing it to a tgz. I have had something similar relating to entry points in dockers before, but cannot remember how I resolved it.

If @dkhughes still has his copy of the docker, we can try uploading that, or wait for the new docker to come on line.

[the-snowwhite]

Well I hope @dkhughes can verify that the former zynq projects have kept their integrity in the new docker and function as supposed to. Then we can let the old docker just hideaway....

[the-snowwhite]

@ArcEye BTW

Meanwhile the quartus packages are building but the vivado are not, will have to do a seperate vivado build of the last 2 PRs once sorted.

The last 2 commits were Quartus only ..

[dkhughes]

I'm trying to download the image again it now. Only 25Mbps connection though, so it's going to take a while.

I saw you said you baked the image into boot.bin. Why did you go that road instead of using the overlay framework like the existing projects? The newest xilinx kernels support it out of the box with very little patching required unlike back in 4.4.

[dkhughes]

I read through the docker file in your repo and you're missing some of the dependencies for zedboards that we had to install into the image manually. Are these board files included in the vivado install now? If not, the build will fail since it's missing pin definitions, timing maps, etc.

See the original docker image definition here:

https://github.com/dkhughes/vivado-docker/blob/master/Jessie-Vivado-2015.4/Dockerfile.in#L20

[the-snowwhite]

Oh I have tested building all bitfiles I guess I pushed that commit directly to the master branch while I was testing dockerhub sync.

[the-snowwhite]

I was already including the Ultra96 bdf's so I just added the missing zedboard ones to the archive.

I had to bump the revision number from 1.1 to 1.2 as 1.1 is not in Avnets current bdf repo

[the-snowwhite]

I saw you said you baked the image into boot.bin. Why did you go that road instead of using the overlay framework like the existing projects? The newest xilinx kernels support it out of the box with very little patching required unlike back in 4.4.

Can you be so kind as to provide links to how ? 1. I have not seen documentation about that. So I was in doubt as to if it were possible to boot linux a desktop with unconfigured fpga. And if its possible to fully reconfigure the whole fpga while linux os is running(old habits from the cyclone v with framebuffer I guess) (Just found out that the cpu config is provided via the hw definition to a config file in one of the other fw files in boot.bin ).

2. There is only 1 PIN configuration in the initial pre-release.

3. What is the correct method for adding pin configs in the Vivado stuff ..?

---

I expect to provide overlay support in the final release right now I'm focusing on the prerelease thats out, the overlay stuff and other (unknown for me) loose ends can wait until the final release (via external( from here) test feedback).

[the-snowwhite]

Why did you go that road ?

Nope ..This is where the stuff delivered with the ultra96 was at upon delivery.

[the-snowwhite]

Lastly as mentioned via the method I was provided I have so far found it mostly impossible to figure out what happened behind the petalinux boot stuff or even get the boot.bin not to program the fpga while booting.

---

That is until I just recently stumbled across a Fosdem lecture:ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip linked to this very interesting article: Booting with U-Boot SPL on ZynqMP, a step forward June 2, 2019 by Luca Ceresoli

[the-snowwhite]

@dkhughes I sense that I left out the main reason for skipping the traditional device-tree overlay fpga program method in the pre release: Reason is that some time and space is needed to figure out how to implement a Partial Reconfiguration framework suitable for Machinekit-Hal ongoing..

It takes a certain amount of collaboration to visualize and manifest correctly and while the vision has had some time to settle in the Ggroup forum the possibility of implementation is very recent (this summer 2019). So far no searchable new Open Source articles reflect this change of licence condition.

This odd fact is perhaps due to Open Source PR access and sharing first has been opened via Xilinx 2019.1 suite this summer.

Right now there is a small opening/waking up going on in the Ggroup (as to there is one besides me responding to this idea).

So far I have not been able to get consent of that inspiration/vision from the other HDL capable MK members,,,you and @cdsteinkuehler ,,, for reasons unknown to me (but I hope is because that technology was non Open Source restricted when the vision was described, and that this roadblock is dissolving now ).

Right now I have to take 1 step at a time until I get this release offloaded from my worktable as it unlike the first mksofpga launch this time also contains/requires adding the new cpu architecture itself (arm64).

For some odd reason the aarch64 release got stuck in line behind getting the Vivado builder resolved,,, until this congestion is cleared I'm stuck....

[dkhughes]

@the-snowwhite I want to help you get this new device running, but I am busy with some things at work and I can't spare too much time at present. That being said, I am taking a look at the older vivado projects with your image now that it has the missing board files for zedboard.

To generate a new Vivado project pin file definition etc, use one of the existing projects as a template. Ask me any questions if your confused on an issue here and I'll help you through. I imagine it's going to need an update with the upgrade to vivado 2019.

"time and space is needed to figure out how to implement a Partial Reconfiguration framework suitable for Machinekit-Hal ongoing":

Is this because of the framebuffer issue mentioned? We fully reconfigure the FPGA on the zynq devices after boot. u-boot and its generated SPL are used to bring the device to life, we skip programming the FPGA at all until linux is up and running. Since the zynq is commonly a headless device this is no big deal for the hardware I've tested. I think the new FPGA overlay framework in the kernel has been modified to support partial reconfig as well, but I haven't looked into it at all.

I have no need for partial reconfiguration, but if the project wants that great. In the past, I disagreed with your method to create a brand new hostmot that essentially tore down the things that keep my hardware running, to support the hardware you were running. That's all. If you can implement partial reconfiguration, and it's backwards compatible (read: doesn't require a HUGE FPGA) then cool, I'll happily test it out.

[the-snowwhite]

@dkhughes Whats urgent is to test if the generated binaries work on the zynq boards.(attached) I have already to my best ability made All the changes needed to build all the binaries, for all projects. in this branch

To save you time I here attach an archive of all the new binaries I have built for you to test on the zynq boards. 2019.1_binaries.tar.gz

I have tested the Ultra96 ones (success there).

[dkhughes]

I was able to download the vivado docker image yesterday. Will test tonight and report back.

[dkhughes]

This commit has problems:

https://github.com/the-snowwhite/mksocfpga/commit/f7bee2cc4e8db24f2d7fbfeab0907aaac0d8bb8d

You can't just use the TCL generator and overwrite those Zynq tcl scripts. They were customized to allow the multi device target building that we use to in the final xilinx packages.

The new scripts for 2019.1 appear to be targeting a specific device if the project can't be opened instead of failing (it's hardcoded now to 7z020 for instance in a script that is called by 7z010 devices).

These changes to the TCL scripts are making the Vivado device generated bit files inconsistent and inoperable on my hardware. I can't agree to merging that commit.

Just curious, but why the case change commit here:

https://github.com/the-snowwhite/mksocfpga/commit/5f3e45822a8cdc6c098076d96a7d3406de575270

VHDL is case insensitive. Did the auto updater do this as well?

[dkhughes]

These tags hard coded to your dev machine should be purged as well, I don't have these archives locally:

https://github.com/the-snowwhite/mksocfpga/commit/5f3e45822a8cdc6c098076d96a7d3406de575270#diff-574f61113b52deed5c64b79bf4c597a7R813

[the-snowwhite]

This commit has problems: the-snowwhite@f7bee2c The new scripts for 2019.1 appear to be targeting a specific device if the project can't be opened instead of failing (it's hardcoded now to 7z020 for instance in a script that is called by 7z010 devices). The remote = 0 variable change ensures that these hard coded project paths are not used (i have edited the commit title to reflect this) I have edited that commit to fail and error if a project is missing, and no hard coding of device/board.

You can't just use the TCL generator and overwrite those Zynq tcl scripts. They were customized to ... Since the default tcl policy is that scripts are locked to Vivado revisions (including the hand edited scripts) and the update is from 2015.4 to 2019.1 It makes more sense to begin with a newly generated tcl script as base and then modify it into the original functionality.

allow the multi device target building that we use to in the final xilinx packages. So I'm somewhat back at this question:

What is the correct method for adding pin configs in the Vivado stuff ..? What is the correct method for adding a new board/device ?

These tags hard coded to your dev machine should be purged as well, I don't have these archives locally:

the-snowwhite@5f3e458#diff-574f61113b52deed5c64b79bf4c597a7R813

I have deleted those tags and found otherones I deleted as well I have put up a new branch with the re-based changes.

---

Docker file down: I'm currently re-building all the bitfiles but had to take down the new docker file as something has gone wrong with adding the microzed board files. So I have to build it again test and then upload a correct version.

[dkhughes]

Great, I will check it out these changes when you upload the updated docker file.

Xilinx tools are really bad about sneaking in references to your specific build system.

What is the correct method for adding pin configs in the Vivado stuff ..? What is the correct method for adding a new board/device ?

These are the notes I wrote back when I made the projects:

https://github.com/the-snowwhite/mksocfpga/tree/2019.1_u96-ed/HW/VivadoProjects

I'll upgrade the zynq projects to the new version manually once you can build them on your end. That will remind me of the details that readme is missing and I'll add more info to it to help in the future.

[the-snowwhite]

Great.

---

I had retagged the dockerfile and the original was uploaded, that I think that was what prevented me from updating the Docker image.

[the-snowwhite]

OK thesnowwhite/bionic-vivado is renewed on dockerhub And I was able to build all the bitfiles (and *.bit.bin).

---

Let me know if there are more issues I need to fix. Looking forward to see the new doc's

[dkhughes]

I can't find the docker image at dockerhub under bionic-vivado or the older vivado-docker. Have a better link? I'll pull in the morning.

[the-snowwhite]

Try this direct link to the repo:

[the-snowwhite]

resolved