Building with build do not work any more with amaranth, use pdm instead

[mcclure]

Tonight I tried to build this project from Windows 10. The build scripts are Bash, so I ran the commands manually one by one. From a CMD.EXE prompt I ran:

python.exe -m pdm install
.venv\Scripts\activate.bat # I would prefer not to install anything globally
python -m ensurepip
python -m pip install --upgrade build pytest twine black coverage
python -m build
python -m pip install --force-reinstall dist\the_cradle_colorlight_i9_ecp5_amaranth_hdl_by_sporniket-0.0.0-py3-none-any.whl

This appeared to work (although the python -m build appeared to have installed a venv inside of the venv, which was a little dizzying) up to the whl step, where it failed with an error I do not understand:

Should I maybe move to a Linux box (or mac, if that would work? mac would be slightly more convenient because my Linux box is my Windows box…) and try again?

It would be great if the build could be driven and virtualized by pdm completely, as I believe that would be more likely to work cross platform, but maybe the build scripts are too complex for that, I don't know.

[sporniket]

This looks like the same problem I had on the github workflow of my "amaranth-stuff" lib, I will update dependencies. Stay tuned.

[sporniket]

@mcclure , please try with branch 7-does-not-appear-to-build-on-windows, to see if it fix your problem.

[mcclure]

@sporniket , thank you, I tried checking out this branch and got the same error.

I then did a second build. On the second try I did a git clean -xffd first, and to create my virtualenv I did python -m venv .venv instead of pdm install. This also got the same error. Here is more context, after this quote it is same as screenshot above.

(.venv) C:\Users\Andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl>python -m pip install --force-reinstall dist\the_cradle_colorlight_i9_ecp5_amaranth_hdl_by_sporniket-0.0.0-py3-none-any.whl
Processing c:\users\andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl\dist\the_cradle_colorlight_i9_ecp5_amaranth_hdl_by_sporniket-0.0.0-py3-none-any.whl
Collecting amaranth-stuff-by-sporniket@ git+https://github.com/sporniket/amaranth-stuff@22130d399e82e146aa5a7d4708ec26b002979bab (from the-cradle-colorlight-i9-ecp5-amaranth-hdl-by-sporniket==0.0.0)
  Cloning https://github.com/sporniket/amaranth-stuff (to revision 22130d399e82e146aa5a7d4708ec26b002979bab) to c:\users\andi\appdata\local\temp\pip-install-p_s44c66\amaranth-stuff-by-sporniket_bfe85ea8525443f3917d4b8a0c8bc9c0
  Running command git clone --filter=blob:none --quiet https://github.com/sporniket/amaranth-stuff 'C:\Users\Andi\AppData\Local\Temp\pip-install-p_s44c66\amaranth-stuff-by-sporniket_bfe85ea8525443f3917d4b8a0c8bc9c0'
  Running command git rev-parse -q --verify 'sha^22130d399e82e146aa5a7d4708ec26b002979bab'
  Running command git fetch -q https://github.com/sporniket/amaranth-stuff 22130d399e82e146aa5a7d4708ec26b002979bab
  Resolved https://github.com/sporniket/amaranth-stuff to commit 22130d399e82e146aa5a7d4708ec26b002979bab
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting importlib-metadata (from the-cradle-colorlight-i9-ecp5-amaranth-hdl-by-sporniket==0.0.0)
  Obtaining dependency information for importlib-metadata from https://files.pythonhosted.org/packages/cc/37/db7ba97e676af155f5fcb1a35466f446eadc9104e25b83366e8088c9c926/importlib_metadata-6.8.0-py3-none-any.whl.metadata
  Using cached importlib_metadata-6.8.0-py3-none-any.whl.metadata (5.1 kB)
Collecting amaranth@ git+https://github.com/amaranth-lang/amaranth@f135226a79fddf5caf894030ac7f49995774c3a7 (from amaranth-stuff-by-sporniket@ git+https://github.com/sporniket/amaranth-stuff@22130d399e82e146aa5a7d4708ec26b002979bab->the-cradle-colorlight-i9-ecp5-amaranth-hdl-by-sporniket==0.0.0)
  Cloning https://github.com/amaranth-lang/amaranth (to revision f135226a79fddf5caf894030ac7f49995774c3a7) to c:\users\andi\appdata\local\temp\pip-install-p_s44c66\amaranth_2fa5c45c69a1466bafff9b4ba65a7904
  Running command git clone --filter=blob:none --quiet https://github.com/amaranth-lang/amaranth 'C:\Users\Andi\AppData\Local\Temp\pip-install-p_s44c66\amaranth_2fa5c45c69a1466bafff9b4ba65a7904'
  Running command git rev-parse -q --verify 'sha^f135226a79fddf5caf894030ac7f49995774c3a7'
  Running command git fetch -q https://github.com/amaranth-lang/amaranth f135226a79fddf5caf894030ac7f49995774c3a7
  Running command git checkout -q f135226a79fddf5caf894030ac7f49995774c3a7
  Resolved https://github.com/amaranth-lang/amaranth to commit f135226a79fddf5caf894030ac7f49995774c3a7
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]

Do you think it is a problem I am running in a venv?

[sporniket]

Ok, I did reproduce the problem, once I wiped any file inside pip cache, forcing it to re-fetch, build and install amaranth. I will try to solve that for me, and hopefully this will give me clues as to what is happening.

[sporniket]

I tried to build amaranth using the commit I referenced, that use wheel and all, and that failed. It seems than latest commit migrated to pdm and it works, I will work on that as during the weekend.

[sporniket]

I have pushed update, TL;DR; now amaranth-stuff and the cradle uses pdm.

Please have a try.

[mcclure]

Updated the 7-does-not-appear-to-build-on-windows branch and ran

git clean xffd
python.exe -m pdm install
python.exe -m pdm build # I think this didn't do anything?
.venv\Scripts\activate.bat 
python -m ensurepip
python -m pip install --upgrade build pytest twine black coverage
python -m build
python -m pip install --force-reinstall dist\the_cradle_colorlight_i9_ecp5_amaranth_hdl_by_sporniket-0.0.0-py3-none-any.whl
python -m the_cradle.deployer

This time I got further! However on the final step I got this error

(the-cradle-colorlight-i9-ecp5-amaranth-hdl-3.11) C:\Users\Andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl>python -m the_cradle.deployer
========================[ START OF Deployment ]============================
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl\.venv\Lib\site-packages\the_cradle\deployer.py", line 48, in <module>
    Deployer(Colorlight_I9_V7_2_Platform(), TheCradle()).deploy()
  File "C:\Users\Andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl\.venv\Lib\site-packages\the_cradle\deployer.py", line 43, in deploy
    self.endpoint.build(self.payload, do_program=True)
  File "C:\Users\Andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl\.venv\Lib\site-packages\amaranth\build\plat.py", line 103, in build
    require_tool(tool)
  File "C:\Users\Andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl\.venv\Lib\site-packages\amaranth\_toolchain\__init__.py", line 33, in require_tool
    raise ToolNotFound("Could not find required tool {} in PATH. Place "
amaranth._toolchain.ToolNotFound: Could not find required tool yosys in PATH. Place it directly in PATH or specify path explicitly via the YOSYS environment variable

I actually do not have yosys installed. So probably this is expected.

[mcclure]

I put some additional thoughts in #7.

[mcclure]

python -m pip install yowasp-yosys
set YOSYS=C:\Users\Andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl\.venv\Scripts\yowasp-yosys.exe
python -m pip install yowasp-nextpnr-ecp5
set NEXTPNR_ECP5=C:\Users\Andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl\.venv\Scripts\yowasp-nextpnr-ecp5.exe
set ECPPACK=C:\Users\Andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl\.venv\Scripts\yowasp-ecppack.exe
set OPENFPGALOADER=C:\Users\Andi\work\f\other\the-cradle-colorlight-i9-ecp5-amaranth-hdl\fake_openFPGALoader.bat
python -m the_cradle.deployer

THIS appears to have worked, and the arguments to my fake openFPGALoader were -c cmsisdap -m C:\Users\Andi\AppData\Local\Temp\amaranth_h21e5e7__top.bit

However, the file C:\Users\Andi\AppData\Local\Temp\amaranth_h21e5e7__top.bit does not exist.

Does the_cradle.deployer delete the file after it is done programming? I would prefer to use ecpdap rather than openFPGALoader.

[mcclure]

Oh, the_cradle.deployer appears to write build\top.bit. And programming that I get a blinking light… exciting, let me try a monitor.

[sporniket]

nice