Occasional hangs, fails to verify.

[donnie-j]

Thanks for making this programming app open source.

I've managed to get yosys data into a seemingly suitable .hex file (2 characters per byte space separated, 16 8 bit bytes per line).

Now the issue is the programmer randomly hangs while writing. It seems to be timing related, inserting print "addr %x" %current_address into the appropriate place in the code to view the bytes going out ensures that it only writes a few.

This is a MacBook Pro, with the B2 connected directly to one of the USB Type-C connectors though a C-Micro cable.

Let me know if I can do any testing for you to help debug the cause of the stalls.

Cheers, and thanks again.

[tinyfpga]

i've added some updates that should help with reliability, but i need more testing to be sure

[jcw]

Same issue here - occasional hangs, verify failed, latest 1.1.3, uploading the icestorm_template example, on a 2014 MacBook w/ USB3 ports.

[jcw]

And to follow up: it's not reliable, but with an extra USB2 hub inserted, I reached a verified upload and blinking LED. So yes, it looks like timing - but it validates the bitfile build + programmer + connection.

[tinyfpga]

@jcw: try using the commandline programmer and let me know if you have similar issues when plugged directly into the MacBook USB3 port: https://github.com/tinyfpga/TinyFPGA-B-Series/tree/master/programmer

edit: this will help me eliminate the gui as an issue

[jcw]

That one consistently fails to either detect the board or programming stops with:

$ python programmer/tinyfpgab.py -p ~/Desktop/TinyFPGA_B.bin

    TinyFPGA B-series Programmer CLI
    --------------------------------
    Programming /dev/cu.usbmodem14231 with /Users/jcw/Desktop/TinyFPGA_B.bin
    Waking up SPI flash
    135100 bytes to program
    Erasing designated flash pages
Traceback (most recent call last):
  File "programmer/tinyfpgab.py", line 275, in <module>
    fpga.program_bitstream(addr, bitstream)
  File "programmer/tinyfpgab.py", line 214, in program_bitstream
    if self.program(addr, bitstream):
  File "programmer/tinyfpgab.py", line 178, in program
    self.erase(addr, len(data))
  File "programmer/tinyfpgab.py", line 130, in erase
    self._erase(start_addr, 4 * 1024)
  File "programmer/tinyfpgab.py", line 98, in _erase
    self.wait_while_busy()
  File "programmer/tinyfpgab.py", line 79, in wait_while_busy
    while (self.read_sts()[0] & 1):
IndexError: list index out of range

Are the console and gui programmers doing exactly the same thing and same timing?

PS. Version is Python 2.7.10

[tinyfpga]

The GUI programmer uses the tinyfpgab.py module to actually program the board. However, the GUI itself used to cause some issues due to concurrent tasks accessing the same virtual com port, so I thought there might be some lingering issues.

This exception is possibly what is causing your issues. I’ll have to take a closer look and see what’s going on. Also, if it doesn’t detect your board, it could just mean the bootloader isn’t active. Just press the reset button to reactivate it. After programming the board the programmer automatically reboots the FPGA into the user configuration and the bootloader will not be active again until the reset button is pressed.

On Sat, Oct 14, 2017 at 5:28 PM Jean-Claude Wippler < notifications@github.com> wrote:

That one consistently fails to either detect the board or programming stops with:

$ python programmer/tinyfpgab.py -p ~/Desktop/TinyFPGA_B.bin

TinyFPGA B-series Programmer CLI
--------------------------------
Programming /dev/cu.usbmodem14231 with /Users/jcw/Desktop/TinyFPGA_B.bin
Waking up SPI flash
135100 bytes to program
Erasing designated flash pages

Traceback (most recent call last): File "programmer/tinyfpgab.py", line 275, in fpga.program_bitstream(addr, bitstream) File "programmer/tinyfpgab.py", line 214, in program_bitstream if self.program(addr, bitstream): File "programmer/tinyfpgab.py", line 178, in program self.erase(addr, len(data)) File "programmer/tinyfpgab.py", line 130, in erase self._erase(start_addr, 4 * 1024) File "programmer/tinyfpgab.py", line 98, in _erase self.wait_while_busy() File "programmer/tinyfpgab.py", line 79, in wait_while_busy while (self.read_sts()[0] & 1): IndexError: list index out of range

Are the console and gui programmers doing exactly the same thing and same timing?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tinyfpga/TinyFPGA-Programmer-Application/issues/1#issuecomment-336677046, or mute the thread https://github.com/notifications/unsubscribe-auth/AcnVg4TyZ1rNoj9UocVHiB0BQoByEiQeks5ssVGTgaJpZM4PSCi3 .

[jcw]

Ok, thx for the info. What I see so far:

• USB3 + GUI - always fails to connect on first try, connects usually ok on 2nd and later, but always fails in verification and occasionally stops halfway
• USB2 + GUI - works ≈ 1 out of 10 times, also fails to connect on first program after plug-in
• console version never worked here so far, occasional failure to detect, but usually the index traceback you saw

Personally, I'd prefer the command-line tool, because it can then be scripted as part of the build cycle.

[tinyfpga]

What MacOS version are you running? Do you have access to a windows or linux computer? I think it’s an issue with the way MacOS handles serial ports, but I will need a bit more time to try and reproduce the issue myself. I experienced similar strange behavior while developing the TinyFPGA Programmer for the A-series.

On Sat, Oct 14, 2017 at 5:37 PM Jean-Claude Wippler < notifications@github.com> wrote:

Ok, thx for the info. What I see so far:

• USB3 + GUI - always fails to connect on first try, connects usually ok on 2nd and later, but always fails in verification and occasionally stops halfway
• USB2 + GUI - works ≈ 1 out of 10 times, also fails to connect on first program after plug-in
• console version never worked here so far, occasional failure to detect, but usually the index traceback you saw

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tinyfpga/TinyFPGA-Programmer-Application/issues/1#issuecomment-336677360, or mute the thread https://github.com/notifications/unsubscribe-auth/AcnVg701zQxA5XECsb7lhGWvz3bHwYBjks5ssVOugaJpZM4PSCi3 .

[jcw]

Latest MacOS, 10.13 here.

On Ubuntu 17.10-prerelease, the command line version very consistently fails on the first try after a reset and succeeds afterwards, both USB 2 and 3.

[tinyfpga]

i'm able to reproduce the issue, currently debugging to find the root-cause

Edit: to be more precise, the failure occurs every time on MacOS, but so far has not happened with my Windows 10 machine.

[tinyfpga]

I've identified the issue and I have a workaround in progress.

The waveform below shows the USB interface on the TinyFPGA B2 as well as the SPI interface from the bootloader to the SPI configuration flash. At point (A) a USB request is made to the bootloader to write 16 bytes of data to a specific address in SPI flash. The bootloader begins writing the data to the SPI flash when request (B) is sent over USB. The bootloader correctly sends a NAK for request (B) because it is not yet ready for more data. However, due to a bug in the bootloader the last few bytes of the write data is corrupted.

This doesn't occur on Windows machines because Windows appears to wait for positive acknowledgement of USB data before proceeding past the serial.flush() function call in the python code.

The tinyfpgab.py module _write() function will be updated with a high-precision delay loop that will wait for a number of microseconds after each write so that the bug will not be hit.

[tinyfpga]

The following commit has a fix which inserts a 60us delay between the SPI flash write and the SPI flash status read. In my testing it resolved the verification failures on MacOS.

https://github.com/tinyfpga/TinyFPGA-Programmer-Application/commit/77b783bbdcd807fbbca63dfc38f64fe72c7c77dd

If this solves the issue for you then please let me know and I'll close this out.

[jcw]

Aha - definitely has an effect: 1st flash fails on verification, 2nd one succeeds, 100% repeatably. This is with direct connect to USB3. With a USB2 hub in between, the index error remains, most of the time.

My take on it: if the bug is that the serial flush does not wait for the lowest levels in the USB stack, then a delay seems to improve it - but (somewhat predictably, IMO) with an extra comms delay in the chain through the hub, the timing again causes issues.

PS. This is with the cmd-line version - the gui version never passes verification.

[jcw]

Found something: with your 60 µS spin loop and opening the serial port at 115200 baud, the cmd-line programmer works every time on USB3, also right after reset (doesn't fix for USB2, alas).

[tinyfpga]

I'm installing MacOS 10.13 on my laptop to make sure I can reproduce the other issues you are seeing. When it's done in an hour I'll be able to try out a fix for another issue I noticed that should clean things up for you. Going forward I'll be developing a testbench and test-suite for the bootloader to improve it's quality for subsequent boards I ship: http://cocotb.readthedocs.io/en/latest/introduction.html. I'll be working on a procedure for safely updating the bootloader as well.

Thanks for filing these issues and being so responsive @jcw, I really appreciate the opportunity to improve the TinyFPGA boards.

[tinyfpga]

With TinyFPGA Programmer Application v1.1.4 and MacOS 10.13 installed on my machine, I was able to get both the GUI and CLI to work on the first time as long as the bootloader is active. I did not experience any verification failures.

However, I did experience a small percentage cases where MacOS would not properly enumerate the device and the bootloader would then timeout and load the user configuration after 5 seconds. This was rare and was resolved by either resetting the TinyFPGA board again or unplugging the TinyFPGA board and reconnecting it a few seconds later. Additionally, it can take a second or two before MacOS recognizes the TinyFPGA board is plugged in.

Please give this latest update a try and let me know if this solves your problem or if there are more issues: https://github.com/tinyfpga/TinyFPGA-Programmer-Application/tree/v1.1.4

[jcw]

Confirmed: latest commit works ok on USB3 with very occasional enumeration failures. With a (cheap) USB2 hub inserted, it fails every time. Not a show stopper.

[tinyfpga]

Thanks @jcw! I'm going to close this out and open a seperate issue for the TinyFPGA B-series bootloader testbench.