search

openXC7 / openXC7-snap

snap package for nextpnr PnR FPGA toolchain for Xilinx 7 series FPGAs, with Spartan7, Artix7, Zynq7 and Kintex7 support
BSD 3-Clause "New" or "Revised" License
26 stars 2 forks source link

Possible to update python? #4

Open exitrip opened 5 months ago

exitrip commented 5 months ago

I'm trying to use the snap install with oss-cad-suite + litex environments and I think I'm hitting a 3.8 vs 3.11 issue with a script crashing:

Python path configuration:
  PYTHONHOME = '/home/engin/oss-cad-suite'
  PYTHONPATH = (not set)
  program name = '/snap/openxc7/x1/usr/bin/nextpnr-xilinx'
  isolated = 0
  environment = 1
  user site = 0
  import site = 1
  sys._base_executable = '/snap/openxc7/x1/usr/bin/nextpnr-xilinx'
  sys.base_prefix = '/home/engin/oss-cad-suite'
  sys.base_exec_prefix = '/home/engin/oss-cad-suite'
  sys.executable = '/snap/openxc7/x1/usr/bin/nextpnr-xilinx'
  sys.prefix = '/home/engin/oss-cad-suite'
  sys.exec_prefix = '/home/engin/oss-cad-suite'
  sys.path = [
    '/home/engin/oss-cad-suite/lib/python38.zip',
    '/home/engin/oss-cad-suite/lib/python3.8',
    '/home/engin/oss-cad-suite/lib/python3.8/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f4c50fb1c00 (most recent call first):
<no Python frame>
Traceback (most recent call last):

Can the snap python version be updated to 3.11?

hansfbaier commented 5 months ago

I would have to update the base image of it. Probably about time. I would really recommend using the nix version though.

exitrip commented 5 months ago

Yeah, I was first trying to use the Nix deployment, but wasn't sure how to square the nix dev shell with a python venv, so switched back to the snap install. I'll try it again.

exitrip commented 5 months ago

So naively trying a complex install of a bunch of extra python modules (see litex_setup.py) into the nix dev shell doesn't work. No surprise as it seems Nix's whole aim is to prevent that type of thing... but a quick pass over Nix's python functionality documentation also doesn't look friendly to litex's "clone n build all the whl locally" approach either.

Surprisingly, sourcing the a venv (via a script that also monkeys with PATH) does work on top of the nix dev shell, up to the same "no module" error running write_json colorlight_i9plus.json in the yosys script.

Now this looks a lot less like a python path / versioning issue, as all the nix paths are there... but that also seems to be the only reason python would fail to find a builtin module.

5. Executing JSON backend.

Warnings: 2 unique messages, 2 total
End of script. Logfile hash: 04c9519e14, CPU: user 49.28s system 1.41s, MEM: 479.84 MB peak
Yosys 0.36 (git sha1 8f07a0d8404, gcc 12.3.0 -fPIC -Os)
Time spent: 26% 12x techmap (14 sec), 12% 32x opt_clean (6 sec), ...
Python path configuration:
  PYTHONHOME = '/home/engin/oss-cad-suite'
  PYTHONPATH = '/nix/store/pa9k6hxg9rdd29ncjmyrw4593w4zdqqa-python3.10-pyyaml-6.0.1/lib/python3.10/site-packages:/nix/store/l8d6q4y9cvp2d5qf2sxxj072668bfcjr-python3-3.10.13/lib/python3.10/site-packages:/nix/store/jpkgydqlmczm9z53m75igsxjmj64fsrk-python3.10-textx-3.0.0/lib/python3.10/site-packages:/nix/store/cqd8p76adwqfhlxzi21dsqagadlzdhl7-python3.10-arpeggio-2.0.2/lib/python3.10/site-packages:/nix/store/m1x5p772qknracqs62cn7082v0mii28i-python3.10-click-8.1.7/lib/python3.10/site-packages:/nix/store/hpfbzc5axcg7749p01g7rnbhx6mp18dx-python3.10-future-0.18.3/lib/python3.10/site-packages:/nix/store/68ihda5zzb37dd7azxaby1rlikaimd20-python3.10-setuptools-68.2.2/lib/python3.10/site-packages:/nix/store/1553c5k26qc93jm2kqdc8p3vxx1bhy5s-python3.10-simplejson-3.19.2/lib/python3.10/site-packages:/nix/store/0vjzm74r3v9zv2v0z0y375bllnd1jymm-python3.10-intervaltree-3.1.0/lib/python3.10/site-packages:/nix/store/j8ary4z5pn669w1lwacyfgb17sqdgrhm-python3.10-sortedcontainers-2.4.0/lib/python3.10/site-packages:/nix/store/byc5sqi8ci9y19jq48im5zqfcg86nmzn-prjxray-76401bd93e493fd5ff4c2af4751d12105b0f4f6d/usr/share/python3/:/nix/store/w7607ki06jzhn1wc8901kwpb2q1n1acw-python3.11-fasm/lib/python3.11/site-packages/'
  program name = 'nextpnr-xilinx'
  isolated = 0
  environment = 1
  user site = 0
  import site = 1
  sys._base_executable = '/nix/store/g3rx7i61f781j13jqsjzwb7sliyggcp8-nextpnr-xilinx-0.7.0/bin/nextpnr-xilinx'
  sys.base_prefix = '/home/engin/oss-cad-suite'
  sys.base_exec_prefix = '/home/engin/oss-cad-suite'
  sys.platlibdir = 'lib'
  sys.executable = '/nix/store/g3rx7i61f781j13jqsjzwb7sliyggcp8-nextpnr-xilinx-0.7.0/bin/nextpnr-xilinx'
  sys.prefix = '/home/engin/oss-cad-suite'
  sys.exec_prefix = '/home/engin/oss-cad-suite'
  sys.path = [
    '/nix/store/pa9k6hxg9rdd29ncjmyrw4593w4zdqqa-python3.10-pyyaml-6.0.1/lib/python3.10/site-packages',
    '/nix/store/l8d6q4y9cvp2d5qf2sxxj072668bfcjr-python3-3.10.13/lib/python3.10/site-packages',
    '/nix/store/jpkgydqlmczm9z53m75igsxjmj64fsrk-python3.10-textx-3.0.0/lib/python3.10/site-packages',
    '/nix/store/cqd8p76adwqfhlxzi21dsqagadlzdhl7-python3.10-arpeggio-2.0.2/lib/python3.10/site-packages',
    '/nix/store/m1x5p772qknracqs62cn7082v0mii28i-python3.10-click-8.1.7/lib/python3.10/site-packages',
    '/nix/store/hpfbzc5axcg7749p01g7rnbhx6mp18dx-python3.10-future-0.18.3/lib/python3.10/site-packages',
    '/nix/store/68ihda5zzb37dd7azxaby1rlikaimd20-python3.10-setuptools-68.2.2/lib/python3.10/site-packages',
    '/nix/store/1553c5k26qc93jm2kqdc8p3vxx1bhy5s-python3.10-simplejson-3.19.2/lib/python3.10/site-packages',
    '/nix/store/0vjzm74r3v9zv2v0z0y375bllnd1jymm-python3.10-intervaltree-3.1.0/lib/python3.10/site-packages',
    '/nix/store/j8ary4z5pn669w1lwacyfgb17sqdgrhm-python3.10-sortedcontainers-2.4.0/lib/python3.10/site-packages',
    '/nix/store/byc5sqi8ci9y19jq48im5zqfcg86nmzn-prjxray-76401bd93e493fd5ff4c2af4751d12105b0f4f6d/usr/share/python3/',
    '/nix/store/w7607ki06jzhn1wc8901kwpb2q1n1acw-python3.11-fasm/lib/python3.11/site-packages/',
    '/home/engin/oss-cad-suite/lib/python310.zip',
    '/home/engin/oss-cad-suite/lib/python3.10',
    '/home/engin/oss-cad-suite/lib/python3.10/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007ffff4f9f7c0 (most recent call first):
  <no Python frame>
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/engin/oss-cad-suite/litex/litex-boards/litex_boards/targets/colorlight_i9plus.py", line 190, in <module>
    main()
  File "/home/engin/oss-cad-suite/litex/litex-boards/litex_boards/targets/colorlight_i9plus.py", line 179, in main
    builder.build(**parser.toolchain_argdict)
  File "/home/engin/oss-cad-suite/lib/python3.11/site-packages/litex/soc/integration/builder.py", line 411, in build
    vns = self.soc.build(build_dir=self.gateware_dir, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/engin/oss-cad-suite/lib/python3.11/site-packages/litex/soc/integration/soc.py", line 1461, in build
    return self.platform.build(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/engin/oss-cad-suite/lib/python3.11/site-packages/litex/build/xilinx/platform.py", line 98, in build
    return self.toolchain.build(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/engin/oss-cad-suite/lib/python3.11/site-packages/litex/build/xilinx/yosys_nextpnr.py", line 172, in build
    return YosysNextPNRToolchain.build(self, platform, fragment, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/engin/oss-cad-suite/lib/python3.11/site-packages/litex/build/yosys_nextpnr_toolchain.py", line 127, in build
    return GenericToolchain.build(self, platform, fragment, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/engin/oss-cad-suite/lib/python3.11/site-packages/litex/build/generic_toolchain.py", line 123, in build
    self.run_script(script)
  File "/home/engin/oss-cad-suite/lib/python3.11/site-packages/litex/build/yosys_nextpnr_toolchain.py", line 234, in run_script
    raise OSError("Error occured during Yosys/Nextpnr's script execution.")
OSError: Error occured during Yosys/Nextpnr's script execution.
⦗OSS CAD Suite⦘ [nix(openXC7)]

Anyway, all that's to say, maybe updating the snap core et al wont actually solve my troubles using openxc7 with litex. The toolchains all work with trivial verilog examples. Cheers.

exitrip commented 5 months ago

okay okay, last post. If I do a VERY bad thing and cp ./lib/python3.11 ./lib/python3.10 the build progresses past this issue, and deeper magic number stuff breaks as expected

File "/home/engin/oss-cad-suite/lib/python3.10/re/_compiler.py", line 18, in <module>
    assert _sre.MAGIC == MAGIC, "SRE module mismatch"

Path of least resistance is to find a release of litex that uses 3.10 and use it with the nix-toolchain.

hansfbaier commented 5 months ago

Yeah, I was first trying to use the Nix deployment, but wasn't sure how to square the nix dev shell with a python venv, so switched back to the snap install. I'll try it again.

Using a python venv in a nix devshell should work. If you have problems with that, I can help. I probably might need to add the venv packages to the devshell, if they are missing. Let me know how that works for you.

exitrip commented 5 months ago

@hansfbaier you may already be aware of this, but your snap install is straight up hardcoded into HEAD of Litex:

 if self.is_openxc7:
            prjxray_db_dir = os.environ.get('PRJXRAY_DB_DIR')
            if prjxray_db_dir is None or prjxray_db_dir == "":
                prjxray_db_dir = '/snap/openxc7/current/opt/nextpnr-xilinx/external/prjxray-db/'

in a few places in this file https://github.com/enjoy-digital/litex/blob/master/litex/build/xilinx/yosys_nextpnr.py#L120

hansfbaier commented 5 months ago

Yes, but only if the environment variable is not set, and it is set in the nix toolchain.