search

openXC7 / xc7k325t-blinky-nextpnr

BSD 3-Clause "New" or "Revised" License
42 stars 9 forks source link

Simplify the build process using git submodules #18

Closed rwhitby closed 2 years ago

rwhitby commented 2 years ago

I think the build process could be simplified by using git submodules:

  1. sudo apt install libftdi1-dev libudev-dev git cmake build-essential tclsh clang tcl-dev libreadline-dev flex bison python3-dev libboost-all-dev libqt5-base-dev-tools libeigen3-dev python3 python3-pip python3-yaml pypy3 pkg-config libqt5opengl5-dev
  2. clone/build/install yosys from https://github.com/YosysHQ/yosys or download a release from https://github.com/YosysHQ/oss-cad-suite-build/releases note: test have been performed with Yosys 0.13+28 (git sha1 bf85dfee5, gcc 10.2.1-6 -fPIC -Os)
  3. git clone https://github.com/kintex-chatter/xc7k325t-blinky-nextpnr.git

    This would change to include --recurse-submodules

  4. git clone --recurse-submodules https://github.com/kintex-chatter/nextpnr-xilinx.git

    This could be made a submodule of xc7k325t-blinky-nextpnr.git and then the branch and specific commit could be controlled. that would remove this step. The nextpnr-xilinx repository could also submodule the kintex7-specific prjxray-db repository.

  5. cd nextpnr-xilinx

    This would change to cd xc7k325t-blinky-nextpnr/nextpnr-xilinx

  6. git checkout xilinx-upstream

    This would no longer be required

  7. mkdir build
  8. pushd build
  9. cmake -DARCH=xilinx -DCMAKE_INSTALL_PREFIX=~/opt/nextpnr ..
  10. make -j2 && make install
  11. popd
  12. pushd xilinx/external

    This would no longer be required

  13. rm -rf prjxray-db

    This would no longer be required

  14. git clone -b k325 https://github.com/kintex-chatter/prjxray-db.git

    This would no longer be required

  15. popd

    This would no longer be required

  16. python3 xilinx/python/bbaexport.py --device xc7k325tffg676-1 --bba xilinx/xc7k325tffg676-1.bba
  17. build/bbasm -l xilinx/xc7k325tffg676-1.bba xilinx/xc7k325tffg676-1.bin
  18. mkdir -p ~/opt/nextpnr/xilinx-chipdb
  19. ln -s $PWD/xilinx/external/prjxray-db ~/opt/nextpnr/
  20. cp xilinx/xc7k325tffg676-1.bin ~/opt/nextpnr/xilinx-chipdb/
rwhitby commented 2 years ago

I will test this theory out, and if it works I'll submit a pull request.

I'm testing on Intel Ubuntu 20.04.4 LTS native and Ubuntu 20.04.4 LTS in WSL2 on Windows.

rwhitby commented 2 years ago

First pull request (for adding prjxray.db into nextpnr-xilinx) is at https://github.com/kintex-chatter/nextpnr-xilinx/pull/1 which removes the need for steps 12 through 15.

Once this one is merged, I'll submit a second pull request to add nextpnr-xilinx to the current repo and remove steps 4 through 6.

unbtorsten commented 2 years ago

Thank you, for the suggestions @rwhitby! I agree that this will simplify development.

In the long run, our focus should be an update of upstream repositories, and a build procedure that clones directly from prjxray. This will only take an adjustment of the submodule. I opened https://github.com/kintex-chatter/nextpnr-xilinx/pull/2 to remind us of this, whenever upstream prjxray-db is updated correspondingly.

Edit: Update of upstream is pending, cf. https://github.com/SymbiFlow/prjxray/pull/1794 and https://github.com/SymbiFlow/prjxray/pull/1844

unbtorsten commented 2 years ago

@rwhitby: I have updated the README correspondingly. I'd say we're ready for the next step of adding kintex-chatter/nextpnr-xilinx.git as submodule.

rwhitby commented 2 years ago

@unbtorsten Thanks for the branch fix. Second step is coming soon.

rwhitby commented 2 years ago

Should we add https://github.com/kintex-chatter/prjxray.git as a submodule too?

Then it will be in a known place for the Makefile to reference.

rwhitby commented 2 years ago

I also think that the top-level Makefile could automatically run most of these steps with a "setup" target.

The following seems to work on Intel Ubuntu 20.04.4 LTS and Intel macOS Montery 12.1:

https://github.com/rwhitby/xc7k325t-blinky-nextpnr/pull/1 (this depends on https://github.com/kintex-chatter/xc7k325t-blinky-nextpnr/pull/20 being merged first, so has not yet been submitted here to prevent confusion).

The instructions would then simplify to:

  1. git clone --recurse https://github.com/kintex-chatter/xc7k325t-blinky-nextpnr.git
  2. cd xc7k325t-blinky-nextpnr
  3. make setup
  4. make BOARD=qmtech
unbtorsten commented 2 years ago

Should we add https://github.com/kintex-chatter/prjxray.git as a submodule too?

Then it will be in a known place for the Makefile to reference.

If you feel like automating this, I would opt for making this optional (if declined, tell the user to set path to prjxray manually). What do you think?

unbtorsten commented 2 years ago

I also think that the top-level Makefile could automatically run most of these steps with a "setup" target.

rwhitby#1 (this depends on #20 being merged first, so has not yet been submitted here to prevent confusion).

I added some comments in the above mentioned pull request for your fork to continue the discussion there.

jrrk2 commented 2 years ago

I think yosys should also a a sub-repo since most distributions will not have a sufficiently up-to-date version available as a binary dependency

rwhitby commented 2 years ago

I think yosys should also a a sub-repo since most distributions will not have a sufficiently up-to-date version available as a binary dependency

As no changes are made to yosys, I found it easiest just to install oss-cad-suite binary nightly and put it in the path.

unbtorsten commented 2 years ago

I notice that we are mixing system requirements, make setup, and make blinky, which is where things can get confusing. I propose that we separate the build environment and tools that are the same for all Kintex7 packages from the data that is specific to the Kintex7 packages:

  1. List of system requirements (available from upstream) to be on a given PATH
  2. Adaptation of build environment (independent of target Kintex package)
    • prjxray-db::k325 (update of prjxray pending)
  3. Custom builds that depend on target Kintex package

Feel free to edit this comment to rearrange items between lists so that we can reach consensus.

I think the goal for the 'tools' should be to send pull requests to upstream repositories, so that make setup would automate clone/build/install of standard packages. See https://github.com/SymbiFlow/yosys/issues/96 for an example at prjxray previously depending on experimental changes to yosys. On the other hand, make blinky could contribute to the corresponding documentation in the long run.

jrrk2 commented 2 years ago

If we want to be portable to Mac-M1, I don’t think we can rely on oss-cad-suite.

unbtorsten commented 2 years ago

I think we should focus on and clone/build only those repositories that require changes in comparison to their respective upstream. Creating a universal FOSS build environment is the objective of Symbiflow et al.

That said, I have updated the proposal for a scope of README and Makefile(s) stated in https://github.com/kintex-chatter/xc7k325t-blinky-nextpnr/issues/18#issuecomment-1046304962. My distribution, too, ships a version of yosys that is outdated with regard to our needs. Nevertheless, the required source and binaries are available from upstream. The same applies for prjxray. It is thus sufficient to rely on them being installed (in specific paths) as opposed to cloning them. The current revision of the README process reflects this and calls for installation of specific files in steps 0, 1, and 14.

rwhitby commented 2 years ago

I agree that anything upstream that is unchanged, should just be installed/used from upstream, and pull requests sent as soon as possible to incorporate any changes to upstream (as I have done already for openFPGALoader).

rwhitby commented 2 years ago

Pull request https://github.com/kintex-chatter/xc7k325t-blinky-nextpnr/pull/25 covers the second part of my proposal.