python 3.8 support for importlib files api

to restore python 3.8 support in https://github.com/YosysHQ/oss-cad-suite-build/releases

untested; hope to download apycula.egg artifact and test using that

I was able to test this in the context of the oss-cad-suite release distribution and a very simple counter program run on a SiPEED Tang Nano 9k by doing the following:

creating a personal fork of the apicula repo and cloning that https link locally
extracting the .egg file from inside the oss-cad-suite release (oss-cad-suite/lib/python3.8/site-packages/Apycula-0.8.2a2.dev1+g8e31050-py3.8.egg/ to borrow the required pickle files unzip Apycula-0.8.2a2.dev1+g8e31050-py3.8.egg (deleting the above folder and .egg for replacement with the test candidate after copying the .pickle files)
install python 3.8 if necessary and make a virtualenv inside the local apicula fork: brew install python3.8; python3.8 -m venv env_py38; source env_py38/bin/activate
modify the files from the PR that will need testing - apicula/apycula/[gowin_bba.py, gowin_unpack.py, gowin_pack.py]
git add, git commit, git push the changed files apicula/apycula/[gowin_bba.py, gowin_unpack.py, gowin_pack.py]
take the .pickle files from inside the oss-cad-suite release apicula/apycula directory and add them all to the local apicula/apycula fork directory: GW1N-1.pickle, GW1N-9.pickle, GW1NS-2.pickle, GW1NZ-1.pickle, GW1N-4.pickle, GW1N-9C.pickle, GW1NS-4.pickle, GW2A-18.pickle -- these .pickle files (compressed chip DBs) are not distributed as part of the apicula repo but can also be generated as artifacts during PR check.
package the egg to be tested by running python setup.py bdist_egg, locate it inside the dist directory of my local apicula fork apicula/dist/Apicula-[version/sha].egg
place the .egg to be tested and place it in the oss-cad-suite/lib/python3.8/site-packages/ (the downloaded release dir)
modify all references in oss-cad-suite to the old egg and change them to the egg under test - this means modifying the entry scripts oss-cad-suite/libexec/[gowin_bba, gowin_unpack, gowin_pack] and also inside oss-cad-suite/lib/python3.8/site-packages/apicula.pth where in each of the the entry scripts you change line 2 # EASY-INSTALL-ENTRY-SCRIPT: 'Apycula==[version number] and also in the sys.exit() call in main().
build and program as you would normally (in my case using platformio), and confirm that all proceeds without error

Side Note: I did have one issue programming the device (to test the PR) due to an error Error: unable to open ftdi device: -5 (unable to claim usb device. Make sure the default FTDI driver is not in use). It resolved after I deactivated the env_py38 virtualenv above and also restarted my computer. It is possible this is unrelated to the test process above but worth mentioning.

It seems that these last three checks are consistently failing on recent PRs including ones merged to master Chipdb builder / example (master, nextpnr-0.5) (pull_request), Chipdb builder / example (yosys-0.25, master) (pull_request), and Chipdb builder / example (yosys-0.25, nextpnr-0.5) (pull_request) -- being very new here, is that expected?

Yes. While every effort is made to ensure that different versions of apicula and nextpnr at least don't crash, when it comes to examples it's a different story - the examples contain a demonstration of the most recently added features, and which naturally won't compile on older versions painlessly - they are simply obliged to fail and inform about it, well, because on older versions these innovations are not yet supported and they cannot be silently ignored.

The situation may improve when we stop adding new things :)

I realized that my change to gowin_unpack.py was not pushed. I re-tested using same steps listed above. This time I ran the build and programming steps separately, plugging the board in right before programming -- worked without issue.

There is one error in programming that I consistently get Error: Error key checkSum not found but this is likely unrelated to this issue and could be board related (sipeed tang nano 9k)

Building output:

Starting FPGA Toolchain
Starting Yosys CST Checking
     /----------------------------------------------------------------------------\
     |                                                                            |
     |  yosys -- Yosys Open SYnthesis Suite                                       |
     |                                                                            |
     |  Copyright (C) 2012 - 2020  Claire Xenia Wolf <claire@yosyshq.com>         |
     |                                                                            |
     |  Permission to use, copy, modify, and/or distribute this software for any  |
     |  purpose with or without fee is hereby granted, provided that the above    |
     |  copyright notice and this permission notice appear in all copies.         |
     |                                                                            |
     |  THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES  |
     |  WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF          |
     |  MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR   |
     |  ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES    |
     |  WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN     |
     |  ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF   |
     |  OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.            |
     |                                                                            |
     \----------------------------------------------------------------------------/
    Parsing Counter1.v
        - Module top parsed
    Checking if all ports are defined in constraints file
    All Ports are defined
Finished CST Checking
Starting Synthesys with Yosys (Gowin)
    Step 1:     Executing Verilog-2005 frontend: /Users/krakov/Documents/LushayFPGA/Projects/Counter1Project/Counter1.v
    Step 2:     Executing SYNTH_GOWIN pass.
        Step 2.1:   Executing Verilog-2005 frontend: /Users/krakov/Documents/LushayFPGA/oss-cad-suite/libexec/../share/yosys/gowin/cells_sim.v
        Step 2.2:   Executing Verilog-2005 frontend: /Users/krakov/Documents/LushayFPGA/oss-cad-suite/libexec/../share/yosys/gowin/cells_xtra.v
        Step 2.3:   Executing HIERARCHY pass (managing design hierarchy).
        Step 2.4:   Executing PROC pass (convert processes to netlists).
        Step 2.5:   Executing FLATTEN pass (flatten design).
        Step 2.6:   Executing TRIBUF pass.
        Step 2.7:   Executing DEMINOUT pass (demote inout ports to input or output).
        Step 2.8:   Executing SYNTH pass.
        Step 2.9:   Executing MEMORY_LIBMAP pass (mapping memories to cells).
        Step 2.10:  Executing TECHMAP pass (map to technology primitives).
        Step 2.11:  Executing OPT pass (performing simple optimizations).
        Step 2.12:  Executing MEMORY_MAP pass (converting memories to logic and flip-flops).
        Step 2.13:  Executing OPT pass (performing simple optimizations).
        Step 2.14:  Executing TECHMAP pass (map to technology primitives).
        Step 2.15:  Executing OPT pass (performing simple optimizations).
        Step 2.16:  Executing IOPADMAP pass (mapping inputs/outputs to IO-PAD cells).
        Step 2.17:  Executing OPT_CLEAN pass (remove unused cells and wires).
        Step 2.18:  Executing DFFLEGALIZE pass (convert FFs to types supported by the target).
        Step 2.19:  Executing TECHMAP pass (map to technology primitives).
        Step 2.20:  Executing OPT_EXPR pass (perform const folding).
        Step 2.21:  Executing SIMPLEMAP pass (map simple cells to gate primitives).
        Step 2.22:  Executing ABC pass (technology mapping using ABC).
        Step 2.23:  Executing TECHMAP pass (map to technology primitives).
        Step 2.24:  Executing OPT_LUT_INS pass (discard unused LUT inputs).
        Step 2.25:  Executing SETUNDEF pass (replace undef values with defined constants).
        Step 2.26:  Executing HILOMAP pass (mapping to constant drivers).
        Step 2.27:  Executing AUTONAME pass.
        Step 2.28:  Executing HIERARCHY pass (managing design hierarchy).
        Step 2.29:  Printing statistics.
        Step 2.30:  Executing CHECK pass (checking for obvious problems).
        Step 2.31:  Executing JSON backend.

    Summary
        Number of wires:                 201
        Number of wire bits:             561
        Number of public wires:          201
        Number of public wire bits:      561
        Number of memories:                0
        Number of memory bits:             0
        Number of processes:               0
        Number of cells:                 383
            ALU                          105
            DFFE                          33
            DFFRE                         39
            GND                            1
            IBUF                           1
            LUT1                          70
            LUT2                          40
            LUT3                           5
            LUT4                          13
            MUX2_LUT5                     38
            MUX2_LUT6                     19
            MUX2_LUT7                      8
            MUX2_LUT8                      4
            OBUF                           6
            VCC                            1

Finished Synthesys
Starting PnR with NextPnR
    Series:GW1N-9C Device:GW1NR-9C Package:QFN88P Speed:C6/I5
    Packing constants..
    Packing Shadow RAM..
    Packing GSR..
    Packing IOs..
    Packing diff IOs..
    Packing IO logic..
    Packing wide LUTs..
    Packing LUT5s..
    Packing LUT6s..
    Packing LUT7s..
    Packing LUT8s..
    Packing ALUs..
    Packing LUT-FFs..
    Packing non-LUT FFs..
    Packing PLLs..
    Checksum: 0x1395ddc7
    Annotating ports with timing budgets for target frequency 27.00 MHz
    Checksum: 0x1395ddc7
    Placed 7 cells based on constraints.
    Creating initial analytic placement for 132 cells, random placement wirelen = 5984.
    Running main analytical placer, max placement attempts per cell = 18528.
    HeAP Placer Time: 0.07s
    Running simulated annealing placer for refinement.
    SA placement time 0.12s
    Max frequency for clock 'clk_IBUF_I_O': 89.63 MHz (PASS at 27.00 MHz)
    Max delay posedge clk_IBUF_I_O -> <async>: 14.36 ns
    Checksum: 0x26b5bbe7
    Find global nets...
    Routing globals...
    Routing..
    Setting up routing queue.
    Routing 949 arcs.
    Routing complete.
    Router1 time 3.38s
    Checksum: 0x6e293aad
    Max frequency for clock 'clk_IBUF_I_O': 107.84 MHz (PASS at 27.00 MHz)
    Max delay posedge clk_IBUF_I_O -> <async>: 11.93 ns
    Program finished normally.

    Device Utilisation:
        VCC:               1/    1   100%
        SLICE:           306/ 8640     3%
        IOB:               7/  274     2%
        OSER16:            0/   38     0%
        IDES16:            0/   38     0%
        IOLOGIC:           0/  296     0%
        MUX2_LUT5:        38/ 4320     0%
        MUX2_LUT6:        19/ 2160     0%
        MUX2_LUT7:         8/ 1080     0%
        MUX2_LUT8:         4/ 1056     0%
        GND:               1/    1   100%
        RAMW:              0/  270     0%
        GSR:               1/    1   100%
        OSC:               0/    1     0%
        rPLL:              0/    2     0%

Finished PnR
Starting Bitstream Generation with Apicula
Finished Bitstream Generation
Toolchain Completed

Programming output:

Starting FPGA Toolchain
Starting FPGA Programming with OpenFPGALoader
    write to flash
    Jtag frequency : requested 6.00MHz    found 1 devices
    index 0:
        idcode 0x100481b
        manufacturer Gowin
        family GW1N
        model  GW1N(R)-9C
        irlength 8
    File type : fs
    Parse file Parse /Users/krakov/Documents/LushayFPGA/Projects/Counter1Project/Counter1Project.fs: 
    checksum 0x9f25
    Done
    DONE
    bitstream header infos
    CRCCheck: ON
    Compress: OFF
    ConfDataLength: 712
    ProgramDoneBypass: OFF
    SPIAddr: 00fff000
    SecurityBit: ON
    idcode: 1100481b
    loading_rate: 0
    Jtag frequency : requested 2.50MHz    Done
    erase Flash Done
    Flash Written
    Done
    Error: Error key checkSum not found
    CRC check: Success
    displayReadReg 0001f020
        Memory Erase
        Gowin VLD
        Done Final
        Security Final
        Ready
        POR
Finished FPGA Programming
Toolchain Completed

Looks good!

I'm wondering if we can improve CI to prevent this from happening again. But maybe that's for another time.

Cad suite also probably shouldn't be using our alpha releases.

I'll merge this later and make a new release.

On Thu, 3 Aug 2023, 08:28 david65536 @.***> wrote:

I realized that my change to gowin_unpack.py was not pushed. I re-tested using same steps listed above. This time I ran the build and programming steps separately, plugging the board in right before programming -- worked without issue.

There is one error in programming that I consistently get Error: Error key checkSum not found but this is likely unrelated to this issue and could be board related (sipeed tang nano 9k)

Building output:

Starting FPGA Toolchain Starting Yosys CST Checking /----------------------------------------------------------------------------\ yosys -- Yosys Open SYnthesis Suite

Copyright (C) 2012 - 2020 Claire Xenia Wolf @.***>

Permission to use, copy, modify, and/or distribute this software for any

purpose with or without fee is hereby granted, provided that the above

copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES

WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF

MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR

ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES

WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN

ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF

OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 \----------------------------------------------------------------------------/
Parsing Counter1.v
    - Module top parsed
Checking if all ports are defined in constraints file
All Ports are defined
Finished CST Checking Starting Synthesys with Yosys (Gowin) Step 1: Executing Verilog-2005 frontend: /Users/krakov/Documents/LushayFPGA/Projects/Counter1Project/Counter1.v Step 2: Executing SYNTH_GOWIN pass. Step 2.1: Executing Verilog-2005 frontend: /Users/krakov/Documents/LushayFPGA/oss-cad-suite/libexec/../share/yosys/gowin/cells_sim.v Step 2.2: Executing Verilog-2005 frontend: /Users/krakov/Documents/LushayFPGA/oss-cad-suite/libexec/../share/yosys/gowin/cells_xtra.v Step 2.3: Executing HIERARCHY pass (managing design hierarchy). Step 2.4: Executing PROC pass (convert processes to netlists). Step 2.5: Executing FLATTEN pass (flatten design). Step 2.6: Executing TRIBUF pass. Step 2.7: Executing DEMINOUT pass (demote inout ports to input or output). Step 2.8: Executing SYNTH pass. Step 2.9: Executing MEMORY_LIBMAP pass (mapping memories to cells). Step 2.10: Executing TECHMAP pass (map to technology primitives). Step 2.11: Executing OPT pass (performing simple optimizations). Step 2.12: Executing MEMORY_MAP pass (converting memories to logic and flip-flops). Step 2.13: Executing OPT pass (performing simple optimizations). Step 2.14: Executing TECHMAP pass (map to technology primitives). Step 2.15: Executing OPT pass (performing simple optimizations). Step 2.16: Executing IOPADMAP pass (mapping inputs/outputs to IO-PAD cells). Step 2.17: Executing OPT_CLEAN pass (remove unused cells and wires). Step 2.18: Executing DFFLEGALIZE pass (convert FFs to types supported by the target). Step 2.19: Executing TECHMAP pass (map to technology primitives). Step 2.20: Executing OPT_EXPR pass (perform const folding). Step 2.21: Executing SIMPLEMAP pass (map simple cells to gate primitives). Step 2.22: Executing ABC pass (technology mapping using ABC). Step 2.23: Executing TECHMAP pass (map to technology primitives). Step 2.24: Executing OPT_LUT_INS pass (discard unused LUT inputs). Step 2.25: Executing SETUNDEF pass (replace undef values with defined constants). Step 2.26: Executing HILOMAP pass (mapping to constant drivers). Step 2.27: Executing AUTONAME pass. Step 2.28: Executing HIERARCHY pass (managing design hierarchy). Step 2.29: Printing statistics. Step 2.30: Executing CHECK pass (checking for obvious problems). Step 2.31: Executing JSON backend.
Summary
    Number of wires:                 201
    Number of wire bits:             561
    Number of public wires:          201
    Number of public wire bits:      561
    Number of memories:                0
    Number of memory bits:             0
    Number of processes:               0
    Number of cells:                 383
        ALU                          105
        DFFE                          33
        DFFRE                         39
        GND                            1
        IBUF                           1
        LUT1                          70
        LUT2                          40
        LUT3                           5
        LUT4                          13
        MUX2_LUT5                     38
        MUX2_LUT6                     19
        MUX2_LUT7                      8
        MUX2_LUT8                      4
        OBUF                           6
        VCC                            1
Finished Synthesys Starting PnR with NextPnR Series:GW1N-9C Device:GW1NR-9C Package:QFN88P Speed:C6/I5 Packing constants.. Packing Shadow RAM.. Packing GSR.. Packing IOs.. Packing diff IOs.. Packing IO logic.. Packing wide LUTs.. Packing LUT5s.. Packing LUT6s.. Packing LUT7s.. Packing LUT8s.. Packing ALUs.. Packing LUT-FFs.. Packing non-LUT FFs.. Packing PLLs.. Checksum: 0x1395ddc7 Annotating ports with timing budgets for target frequency 27.00 MHz Checksum: 0x1395ddc7 Placed 7 cells based on constraints. Creating initial analytic placement for 132 cells, random placement wirelen = 5984. Running main analytical placer, max placement attempts per cell = 18528. HeAP Placer Time: 0.07s Running simulated annealing placer for refinement. SA placement time 0.12s Max frequency for clock 'clk_IBUF_I_O': 89.63 MHz (PASS at 27.00 MHz) Max delay posedge clk_IBUF_I_O -> : 14.36 ns Checksum: 0x26b5bbe7 Find global nets... Routing globals... Routing.. Setting up routing queue. Routing 949 arcs. Routing complete. Router1 time 3.38s Checksum: 0x6e293aad Max frequency for clock 'clk_IBUF_I_O': 107.84 MHz (PASS at 27.00 MHz) Max delay posedge clk_IBUF_I_O -> : 11.93 ns Program finished normally.
Device Utilisation:
    VCC:               1/    1   100%
    SLICE:           306/ 8640     3%
    IOB:               7/  274     2%
    OSER16:            0/   38     0%
    IDES16:            0/   38     0%
    IOLOGIC:           0/  296     0%
    MUX2_LUT5:        38/ 4320     0%
    MUX2_LUT6:        19/ 2160     0%
    MUX2_LUT7:         8/ 1080     0%
    MUX2_LUT8:         4/ 1056     0%
    GND:               1/    1   100%
    RAMW:              0/  270     0%
    GSR:               1/    1   100%
    OSC:               0/    1     0%
    rPLL:              0/    2     0%
Finished PnR Starting Bitstream Generation with Apicula Finished Bitstream Generation Toolchain Completed

Programming output:

Starting FPGA Toolchain Starting FPGA Programming with OpenFPGALoader write to flash Jtag frequency : requested 6.00MHz found 1 devices index 0: idcode 0x100481b manufacturer Gowin family GW1N model GW1N(R)-9C irlength 8 File type : fs Parse file Parse /Users/krakov/Documents/LushayFPGA/Projects/Counter1Project/Counter1Project.fs: checksum 0x9f25 Done DONE bitstream header infos CRCCheck: ON Compress: OFF ConfDataLength: 712 ProgramDoneBypass: OFF SPIAddr: 00fff000 SecurityBit: ON idcode: 1100481b loading_rate: 0 Jtag frequency : requested 2.50MHz Done erase Flash Done Flash Written Done Error: Error key checkSum not found CRC check: Success displayReadReg 0001f020 Memory Erase Gowin VLD Done Final Security Final Ready POR Finished FPGA Programming Toolchain Completed

— Reply to this email directly, view it on GitHub https://github.com/YosysHQ/apicula/pull/189#issuecomment-1663367033, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABJFIKTVDXBKPLFYJO5VUTXTNAKDANCNFSM6AAAAAA3B4LE5A . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Starting FPGA Toolchain Starting Yosys CST Checking /----------------------------------------------------------------------------\			yosys -- Yosys Open SYnthesis Suite
Copyright (C) 2012 - 2020 Claire Xenia Wolf @.***>

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

@mmicko I'm tagging a new release that will fix oss-cad-suite. Maybe in the future don't package our alpha releases.

YosysHQ / apicula

python 3.8 support for importlib files api #189