Writing to flash. FAILURE and SUCCESS

[rtreffer]

I went down the rabbit hole of datasheets, random flash chips and flashrom recovery with an not yet supported T25S40.

But on the bright sight I've now got a working uPD720201 card and would like to share the experience with everyone.

I've bricked the device a few times and in the end recovered it with an inexpensive ch341 programmer (paid ~10€ because I wanted it fast).

Vendor Specific Configuration

When writing data to flash the FW needs to be prefixed with "vendor specific configuration". The easiest is to have a backup of your rom (use the download / -r option).

Here is a documented version of my vendor specific configuration for a 4 port card.

0x28 0x00                               ; length prefix, the next 40 bytes are all vendor specific configuration + CRC
0x00 0xff 0x01 0xff 0x02 0xff 0x03 0xff ; set pci subsystem vendor / subsystem to 0xff ff ff ff
0x0c 0x00 0x0d 0x00 0x0e 0x00 0x0f 0x00 ; set PHY control 0 to 0x00 - undocumented HW init
0x10 0x00 0x11 0x00 0x12 0x00 0x13 0x00 ; set PHY control 1 to 0x00 - undocumented HW init - may break uPD720202
0x14 0x00 0x15 0x00                     ; PHY control 2, battery charging SDP only
0x16 0x55                               ; PHY control 2, Tr/Tf fine control, default value - uPD720202 has a different default
0x17 0x00                               ; PHY control 2, HW init
0x18 0x00 0x19 0x00 0x1a 0x01 0x1b 0x05 ; host configuration register, default values
0xe7 0xef                               ; CRC16

so this contains nothing but a sane default for PCI and the port registers. It also sets the secret HWInit bits to 0. The card won't boot the firmware without a valid / good vendor specific configuration and good CRC16. Oh and a FW that works....

The CRC16 can be recomputed with the following c code:

#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>

const uint8_t test[42] = {
        0x28, 0x00, 0x00, 0xff, 0x01, 0xff, 0x02, 0xff, 0x03, 0xff, 0x0c, 0x00, 0x0d, 0x00, 0x0e, 0x00,
        0x0f, 0x00, 0x10, 0x00, 0x11, 0x00, 0x12, 0x00, 0x13, 0x00, 0x14, 0x00, 0x15, 0x00, 0x16, 0x55,
        0x17, 0x00, 0x18, 0x00, 0x19, 0x00, 0x1a, 0x01, 0x1b, 0x05
};

const uint16_t crc_table[16] = {
        0x0000,   0x1081,   0x2102,   0x3183,   0x4204,   0x5285,   0x6306,   0x7387,
        0x8408,   0x9489,   0xA50A,   0xB58B,   0xC60C,   0xD68D,  0xE70E,   0xF78F,
};

uint16_t crc_update(uint16_t crc, uint8_t *data, int len)
{
        uint16_t c = crc ^ 0xffff;
        for(int n = 0; n < len; n++) {
                uint8_t m = data[n];
                c = crc_table[(c ^ m) & 0xf] ^ (c >> 4);
                c = crc_table[(c ^ (m >> 4)) & 0xf] ^ (c >> 4);
        }
        return c ^ 0xffff;
}

int main(int argc, char **argv) {
  uint16_t crc = crc_update(0, &test, 42);
  printf("0x%02x 0x%02x\n", (crc & 0xff), (crc >> 8));
  exit(0);
}

NOTE: be double careful about byte order.

Building a rom image

The ROM image has 2 copies of the vendor specific configuration + firmware, aligned to the block size as noted in the datasheet. Unused space in the blocks is filled with 0xff.

My flash was undocumented but the dump contained 2 copies starting at 0x0000 and 0x4000 which translates 16kb blocks (the datasheet says 32kb and 64kb block erases are supported by the chip :facepalm:)

I generated a 512kb bytes (rom size) 0xff file (for i in $(seq 512) ; do for j in $(seq 1024) ; do echo -ne '\xff' ; done ; done > ff). Generating a working ROM block is then a matter of

(
  echo -ne '\x28\x00\x00\xff\x01\xff\x02\xff\x03\xff\x0c\x00\x0d\x00\x0e\x00\x0f\x00\x10\x00\x11\x00\x12\x00\x13\x00\x14\x00\x15\x00\x16\x55\x17\x00\x18\x00\x19\x00\x1a\x01\x1b\x05\xe7\xef'
  cat K2013080.mem
  cat ff
) | head -c $((16*1024)) > fw_block

I needed a double version of that to flash the whole ROM with flashrom:

cat fw_block fw_block ff | head -c $((512*1024)) > rom

Uploading an image

upd72020x-load is able to upload a firmware block, this should overwrite only the second copy of the firmware.

# ./upd72020x-load -b 4 -d 0 -f 0 -w -i fw_block
Doing the writing
bus = 4 
dev = 0 
fct = 0 
fname = /home/ubuntu/block 
Found an UPD720201 chipset
got firmware version: 201308
EEPROM installed
got rom_info: 5e2013
got rom_config: 700
setting rom_config: 700
STATUS: enabling EEPROM write
STATUS: performing EEPROM write
STATUS: finishing EEPROM write
STATUS: confirming EEPROM write
 ======> PASSED

I have not verified the result, but I would expect it to work.

Flashing the whole ROM file with flashrom (offline / card pulled):

flashrom -p ch341a_spi -w rom

I hope this helps others to recover / flash their card to a sane firmware version.

TODO: verification of the upload should be possible by mutating the order of PCI writes in the vendor specific configuration, then uploading that through upd72020x and dumping/verifying it with flashrom

[rtreffer]

With all this knowledge I thought it would be time to try an actual upgrade. I took the UPDATE.mem 2.0.2.6 from https://github.com/denisandroid/uPD72020x-Firmware

(
  echo -ne '\x28\x00\x00\xff\x01\xff\x02\xff\x03\xff\x0c\x00\x0d\x00\x0e\x00\x0f\x00\x10\x00\x11\x00\x12\x00\x13\x00\x14\x00\x15\x00\x16\x55\x17\x00\x18\x00\x19\x00\x1a\x01\x1b\x05\xe7\xef'
  cat uPD72020x-Firmware/UPDATE.mem\ 2.0.2.6
  cat ff
) | head -c $((16*1024)) > fw_block
# ./upd72020x-load -b 4 -d 0 -f 0 -w -i fw_block
Doing the writing
bus = 4 
dev = 0 
fct = 0 
fname = fw_block 
Found an UPD720201 chipset
got firmware version: 201308
EEPROM installed
got rom_info: 5e2013
got rom_config: 0
setting rom_config: 700
STATUS: enabling EEPROM write
STATUS: performing EEPROM write
STATUS: finishing EEPROM write
STATUS: confirming EEPROM write
 ======> PASSED

And after a full power off and power on I checked a backup of the FW

# ./upd72020x-load -b 4 -d 0 -f 0 -r -o backup2
Doing the reading
bus = 4 
dev = 0 
fct = 0 
fname = backup2 
Found an UPD720201 chipset
got firmware version: 202609
EEPROM installed
got rom_info: 5e2013
got rom_config: 0
setting rom_config: 700
 ======> PASSED

so here we go, got firmware version: 202609 which means 2.0.2.6

Flashing with this tool works if you know the block size and have a valid vendor specific configuration :smile:

[markusj]

Thank you a lot for your research and this write-up. If you agree, I would like to put this either into a wiki page or some documentation file in the repository.

The block alignment you discovered really puzzles me, because it clearly conflicts with the datasheet which says that firmware blocks are aligned to erase blocks.

[rtreffer]

Yes, it would be great to have that in a wiki or markdown file.

I was puzzled by the erase block size too. But I by now assume that the hardware does not fully understand these (which is why we need the rom config for writing) and probes different offsets (or alike).

So as long as smaller blocks work it might be fine. It could also be that unknown chips use 16kb.

Hard to tell without someone dumping the contents of another chip.

Btw, the linked issue for working CM4 PCIe devices has pictures and a bit more documentation like content, feel free to pull that in, too!

[rtreffer]

That datasheet also says it should be possible to update the first copy by issuing a chip erase (not implemented, but would be easy) followed by an upload.

And @pclinuxer uploaded a less-thab-blocksize image. That seems to work, too.

The ROM config register might be the equivalent to the flashrom chip table.

[markusj]

Well, I guess that fact that those flash memories are smarter than old-fashioned EEPROM makes this possible. They automatically perform a readout-erase-program cycle for the whole erase block if I understand the datasheet right.

[markusj]

That datasheet also says it should be possible to update the first copy by issuing a chip erase (not implemented, but would be easy) followed by an upload.

Yes, but I do not see me implement such a feature. Either a card is not bricked (yet), then one should probably not do something like this, or it is and then one needs an external interface. I would argue that if someone really wants to fiddle with the factory image, that person then should directly interface the flash because this way, a recovery path is also already available.

The ROM config register might be the equivalent to the flashrom chip table.

In which terms? I had a look into flashchips.c but that structure is contains much more information than the single register can hold. Or do you think of some kind of index to an internal database? The SST ROMs do not really fit there, I guess. Some flags which signal certain features? That appears more plausible to me, guess one could find some correlation between flash ROM properties and the set bits.

[rtreffer]

You can get away with 1. block erase / full erase function, 2. block size and 3. write function.

This might be doable with a few bytes of config. Just a wild guess though, but it could reduce the complexity of the chip, which sounds likely to me.... It would also explain the strange numbering.

[lvmtime]

Hi. First, thanks for the great work!

I ordered a 4 port USB 3 card from AliExpress and it works thanks to this tool. I am currently on Ubuntu 21.10 kernel 5.13.0-23-generic

But, I would like to flash the frimware permanently instead of loading it via systemd. Problem is, this card seems to have some kind of new EEPROM chip:

Found an UPD720201 chipset
got firmware version: 202609
EEPROM installed
got rom_info: 5e3213
got rom_config: 0
unknown EEPROM, no parameters found
ERROR: can not perform action

I would love to help add support for it, and I don't mind risking bricking the card in the process, but I have no idea where to start. So if you could guide me what actions to take / information to provide, I will do my best and share the results.

Should I open a separate issue for this discussion?

[rtreffer]

@lvmtime I've done a write up with pictures of the flash at https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/103#issuecomment-804034815

It shows you what a EEPROM chip looks like, how to flash it out of band etc. A simple clamp can save your card (backup/restore the EEPROM if the card is dead).

What you need to find out (before flashing) is

1. What's the flash chip? (Usually a SOIC-8, check the card)
2. What configuration does the flash chip need? (Search for the datasheet of the EEPROM)
3. What does flashrom need to handle it?
4. Backup! Check the backup.
5. Check the header, flash a new version.

[lvmtime]

@rtreffer an update:

My board looks almost identical to the one in the writeup, except they added a heatsink over the controller chip. The writing on the flash chip says:

T25S40
PB2018
EA5(S?)089

I tried adding the ID 5e3213 to the lookup_rompar function to return 0x700, but I got ERROR: GET_DATAx never go to zero. I tried unbinding the xhci_hcd as per the readme, but that did not solve the problem.

I will try out other addresses, and probably order a programmer with a clip.

I wonder if there is any way other than guessing to find out the rom config? If I dump the firmware with a reader, will it be somewhere in there?

[rtreffer]

@lvmtime ok, T25S40 is the same I got (datasheet), at it worked for me in 0x700 mode, so this is probably a bug and may warrant a dedicated issue.

Anyway, it would be interesting if you could do it out of band and add the IDs to flashrom, see https://github.com/flashrom/flashrom/pull/200/files

I assume that I might have picked the wrong config with 0x700 or that the chip has special write protection enabled in your case. The config parameter is tied to the upd chips (see the dumped table). So anything the chip does not expect should result in an error, especially for chips the vendor did not foresee.

But flashrom is way more flexible and forgiving and can try multiple ways to flash. Plus it supports unlocking of the flash chip. So it would be interesting if that works. Plus you could dump the ROM first and check if you have the same vendor specific configuration :-)

[markusj]

I wonder if there is any way other than guessing to find out the rom config? If I dump the firmware with a reader, will it be somewhere in there?

Non that I am aware of. And I hesitate to add ROM configurations with some random parameters without an official source. The magic numbers in the Renesas datasheet are likely based on some rules for which I have not seen any documentation and which I do not know. So there is not much I can do, sadly.

[lvmtime]

Just a quick update: my programmer and clip arrived today. I will get the card out of my PC and try to dump the flash chip in a few days and let you know if I find out anything.

[lvmtime]

Update: success!

I finally got to pull the card out of my PC, removed the heat sink, and used a ch341a programmer from AliExpress to dump the firmware and flash a new firmware.

Here is what I learned, hope it will help someone else along the way:

1. The flash chip model ID on my card was 0x3213 not 0x2013 as in @rtreffer write up. I had to patch flashrom but the rest of the configuration from @rtreffer patch worked fine.

2. The "vendor specific configuration" was identical to what is shown in the writeup.

3. There was no second copy of the firmware on the chip, nor a second copy of the "vendor specific configuration". Just the first block, then all 0xff. When building the new ROM image I followed @rtreffer instructions but skipped adding the block for the second time. This worked for my card.

4. The chip was not write protected. I do not know why flashing via PCIe commands did not work, but flashing with the programmer worked right away, and the card is now working without the need to load firmware on boot.

[dreamlayers]

I've successfully recovered my card using https://github.com/markusj/upd72020x-load/issues/15#issuecomment-798889799 instructions. Thank you!

After writing the flash, I read it and saw corruption. I think that was because the kernel driver was also using the same USB controller chip and messing with the reads. After echo -n 0000:$device > /sys/bus/pci/drivers/xhci_hcd/unbind reads matched what I wrote.

BTW. The card had been working for a long time. The flash got corrupted when I supplied the 2026 firmware file from k2026fwup1.exe as /lib/firmware/renesas_usb_fw.mem in /boot/initrd.img in Ubuntu 23.10. Recent kernels complain about lack of that file, but apparently, supplying it is dangerous. I could make the card work using that same firmware file and upd72020x-check-and-init from here, but I wanted firmware in flash, so it doesn't have to be loaded like that.

The k2026fwup1.exe and K2024FWUP1 Windows firmware updaters were useless. They failed when the device was non-functional, and said firmware was already up to date when upd72020x-check-and-init temporarily made the device work and I rebooted into Windows with the device working because of that.