search

sylefeb / fpga-binutils

Compilation framework for FPGA tools
GNU General Public License v3.0
5 stars 0 forks source link

MSYS2 group mingw-w64-*-eda and hdl/MINGW-packages #1

Open umarcor opened 3 years ago

umarcor commented 3 years ago

Hi @sylefeb!

Are you aware of the mingw-w64-*-eda tool groups in upstream MSYS2 repositories? That one is available for all environments (MINGW32, MINGW64, CLANG32, CLANG64 and UCRT64). See the list of tools currently available on MINGW64: https://packages.msys2.org/group/mingw-w64-x86_64-eda

# pacman -S mingw-w64-x86_64-eda
:: There are 22 members in group mingw-w64-x86_64-eda:
:: Repository mingw64
   1) mingw-w64-x86_64-dfu-util  2) mingw-w64-x86_64-ecpprog  3) mingw-w64-x86_64-fritzing
   4) mingw-w64-x86_64-ghdl-llvm  5) mingw-w64-x86_64-gtkwave  6) mingw-w64-x86_64-icesprog
   7) mingw-w64-x86_64-icestorm  8) mingw-w64-x86_64-iverilog  9) mingw-w64-x86_64-kicad
   10) mingw-w64-x86_64-kicad-footprints  11) mingw-w64-x86_64-kicad-meta  12) mingw-w64-x86_64-kicad-packages3D     
   13) mingw-w64-x86_64-kicad-symbols  14) mingw-w64-x86_64-kicad-templates  15) mingw-w64-x86_64-nextpnr
   16) mingw-w64-x86_64-ngspice  17) mingw-w64-x86_64-openFPGALoader  18) mingw-w64-x86_64-prjtrellis
   19) mingw-w64-x86_64-scopehal-apps  20) mingw-w64-x86_64-serial-studio  21) mingw-w64-x86_64-verilator
   22) mingw-w64-x86_64-yosys

Enter a selection (default=all):

The recipes for those tools are maintained in msys2/MINGW-packages. As soon as a PR is merged there, packages are built. Within a few hours/days, maintainers sign them and upload them to the server. Meanwhile, it's possible to use GitHub Releases as a mirror for pacman.

Currently, I take care of maintaining most of those EDA tooling recipes. I created hdl/MINGW-packages for keeping track (find the bigger picture in hdl/packages). So, it's not a fork, but a repo for documentation and for testing/CI. See hdl.github.io/MINGW-packages. Naturally, any help would be really welcome.

To name a few examples, those packages are used in the CI workflows of Fomu workshop (https://github.com/im-tomu/fomu-workshop/actions/runs/1002654445) or NEORV32 (https://github.com/stnolting/neorv32/actions/runs/993317180). That is done through Action setup-msys2:

    - uses: msys2/setup-msys2@v2
      with:
        msystem: MINGW64
        update: true
        install: >
          make
          mingw-w64-x86_64-icestorm
          mingw-w64-x86_64-yosys
          mingw-w64-x86_64-nextpnr

NOTE: GHDL is a dependency of Yosys, so that mixed-language (VHDL and Verilog) designs are supported by default.

You will find that the use case of NEORV32 is quite similar to this one. The riscv toolchain with multilib is available on MSYS2 already: mingw-w64-x86_64-riscv64-unknown-elf-gcc. As a matter of that, MSYS2 is the main environment I use for synthesis and simulation using open source tooling.

Overall, it seems there are three tools missing for your bundle: fujprog, silice and xray. The other 12 are upstreamed in MSYS2 already. Hence, I want to propose joining forces in hdl/MINGW-packages. Let's collaborate for having PKGBUILD recipes written for those three tools.

Even if any of them cannot be upstreamed, we can use the same recipes for generating zst packages, which can then be used for creating an standalone zipfile. That approach is used in gtkwave/gtkwave and ghdl/ghdl. Since you wrote the list of "extra" DLLs in https://github.com/sylefeb/fpga-binutils/blob/master/mingw32/compile_all.sh, it would be straightforward to mimick the solution in https://github.com/gtkwave/gtkwave/blob/master/MSYS2/standalone_pkg.sh. See an example CI run: https://github.com/gtkwave/gtkwave/runs/2985868198?check_suite_focus=true. The corresponding artifacts (zst packages and standalone tarballs) are uploaded to a "nightly" release: https://github.com/gtkwave/gtkwave/releases.

Last, but not least, the PKGBUILD recipes in msys2/MINGW-packages are pinned to a version or an specific commit. However, minimal modifications are required in order to have them build the master branch. That is used for testing Verilator, Icarus Verilog, OpenFPGALoader, Serial-Studio...

sylefeb commented 3 years ago

Hi Unai, I was not aware of that, this is looking great! Thanks for the detailed instructions and suggestions, I will take the time to dive into that in more depth as soon as possible.

I'd be happy to participate in the effort ; making these builds is not the most enjoyable experience, but I strive to make it easy for anyone to get started, which is why I put some effort in preparing these.

There are a few tricky points though that I'll have to consider:

Final note: I started to include xray but this is very preliminary and not required for now.

umarcor commented 3 years ago

Hey there!

During the last couple of weeks I did some contributions related to this issue:

There is now an example workflow and PKGBUILD recipe in msys2/setup-msys2: examples. So, you can copy and paste those; then modify the PKGBUILD. For instance, in openFPGALoader:

I prepared a PKGBUILD recipe for Silice in https://github.com/umarcor/MINGW-packages/blob/silice/mingw-w64-silice/PKGBUILD. As you see, compared to the openFPGALoader recipe, I used a pinned commit. That depends on the location of the workflow (the project repo, or some third-party repo).

Unfortunately, the build is failing: 

   D:/a/_temp/msys/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: libs/LibSL/src/libs/src/zlib/libzlib.a(deflate_quick.c.obj):deflate_quick.c:(.rdata+0x0): multiple definition of `static_ltree'; libs/LibSL/src/libs/src/zlib/libzlib.a(trees.c.obj):trees.c:(.rdata+0x4e0): first defined here
  [100%] Linking CXX shared library libLibSL_sharp_binder.dll
  collect2.exe: error: ld returned 1 exit status
  make[2]: *** [CMakeFiles/silicehe.dir/build.make:158: silicehe.exe] Error 1
  make[1]: *** [CMakeFiles/Makefile2:432: CMakeFiles/silicehe.dir/all] Error 2
  make[1]: *** Waiting for unfinished jobs....
  make: *** [Makefile:136: all] Error 2

Any guess about that?

I am patching yosys to include my DSP fix for the ice40. The patch is necessary for some Silice demos to work correctly.

I believe the best is to have it fixed upstream. However, should it be really necessary (because that takes too long), we can build an alternative yosys-silice, so that users can install them side-to-side. However, I don't think we should distribute that through MSYS2 repositories. Instead, users might download the yosys-silice package and install it through pacman -U *.zst. See the nightly GHDL assets: https://github.com/ghdl/ghdl/releases/tag/nightly.

We can provite nightly assets in https://github.com/hdl/MINGW-packages for the variants of the tools not to be upstreamed to MSYS2. It's possible to use a GitHub Release as a pacman mirror. So, pacman -Ss would show the upstream and the variants.

I've had trouble with python regarding nextpnr, gowin apicula and (upcoming) symbiyosys, and I do a couple tricks with paths between Silice build system and the package I produce. I'd have to check whether this works fine with the current MinGW packages (hope it does! this was a difficult aspect of getting this together).

That really depends on the type of Python packages you use. Basically, wheels are broken for packages with compiled assets. If the wheel corresponds to a Python only project, then there is no problem. Otherwise, the wheels are not compatible. However, that problem and the possible conflicts between system packages and pip are quite known, and they are being addressed.

In hdl/MINGW-packages we avoid the problem by either providing python-* packages, or by testing that installation through pip does work.

Lately, we have been testing the usage on CPython of GHDL packages built on MSYS2: https://github.com/ghdl/ghdl/actions/runs/1044142852

nextpnr, gowin apicula and (upcoming) symbiyosys

I faced a problem with libusb-1.0.dll in latest MinGW not working, which is why I include it in bin/. Maybe this is just bad configuration on my part, but that is another point to double check. Did you have any similar issues? (basically, bitstream loaders such as iceprog segfault with latest DLL, despite being built onto that same environment, I have to ship with a prior version).

I did not use libusb intensively on MSYS2. However, icesprog (for iCESugar) and iceprog are available on MSYS2 and I am not aware of users having problems with them. @juanmard?

Maybe the libusb DLL you were using was not from MSYS2 but from some other variant of mingw?

Final note: I started to include xray but this is very preliminary and not required for now.

I do also want to add other nextpnr architectures to MSYS2 (apicula, prjoxide, prjbureau...). However, I need some user or developer of those projects to step ahead. I cannot make those a priority for me.

sylefeb commented 3 years ago

Hi @umarcor,

Thanks for the detailed info! Sorry I could not look into this yet ; I want to run a few tests ensuring I have a clean msys2 install (thinking of libusb in particular), and try out existing packages (no longer having to compile gcc-riscv would be a big relief). Hopefully I can find the time for that soon!

Best regards -

umarcor commented 3 years ago

Hi @sylefeb! Take it easy, there is no rush at all. I'm busy too, so let's handle this asynchronously. Whenever we have some slot, we try to move one little step ahead :wink:

sylefeb commented 3 years ago

Hi @umarcor, I started to move towards using the MinGW EDA packages and things are looking good! Now my 'fpga-binutils' only ships with my patched yosys, symbiyosys, yices, nextpnr-gowin, nextpnr-xilinx (still missing xray -- will look into that). Everything else comes from your official packages, that is really great!

Silice draft branch is applying these changes (the get_started_mingw64.sh script takes care of everything).

Did not run into major issues (a few glitches here and there but these were easily corrected). Btw, no more issues with Python. libusb keeps crashing with the libusbK driver, however it now works with the WinUSB driver, so that's all fine. I moved to using openFPGAloader in place of fujprog.

Next I'll consolidate (e.g installing symbiyosys from pip as you suggested) ; I'll also look into the Silice PKGBUILD you started, and will try to make PKGBUILD of the other parts (but I am new to this :) )

Thanks - these packages make things much easier!

umarcor commented 3 years ago

Those are really great news! I'm so happy to hear them 😃

With regard to libusb, I am aware that some devices (particularly from FTDI) are problematic on Windows and people use Zadig for overriding the drivers. However, I did not deal with it myself yet. It's in the "I'd like to have a look at it" list but not a priority for me.

Conversely, let me know if you run into any issue with PKGBUILD recipes. Those are essentially shell scripts with some functions. Therefore, it should be easy for you to understand the syntax. The challenge might be some of the variables such as pkgdir or srcdir which are defined by the build tool/script.