search

sinara-hw / sinara

Sayma AMC/RTM issue tracker
Other
42 stars 7 forks source link

Sayma Unknown flash device (ID 0x00ffffff) #562

Open jbqubit opened 6 years ago

jbqubit commented 6 years ago

Sayma s/n -8 worked on bench but in crate reports error when flashing. 

$ artiq_flash -t sayma --srcbuild ~/artiq-builds/artiq_sayma_20180612_f8627952c8
Design: top;COMPRESS=TRUE;UserID=0XFFFFFFFF;Version=2018.1
Part name: xcku040-ffva1156-1-c
Date: 2018/06/12
Time: 16:10:51
Bitstream payload length: 0xc4a5a8
Design: top;UserID=0XFFFFFFFF;COMPRESS=TRUE;Version=2018.1
Part name: 7a15tcsg325
Date: 2018/06/12
Time: 15:26:32
Bitstream payload length: 0xbe910
Open On-Chip Debugger 0.10.0-00013-gbb7beda (2018-02-13-15:56)
Licensed under GNU GPL v2
For bug reports, read
    http://openocd.org/doc/doxygen/bugs.html
none separate
adapter speed: 5000 kHz
Info : clock speed 5000 kHz
Info : JTAG tap: xc7.tap tap/device found: 0x0362e093 (mfg: 0x049 (Xilinx), part: 0x362e, ver: 0x0)
Info : JTAG tap: xcu.tap tap/device found: 0xffffffff (mfg: 0x7ff (<invalid>), part: 0xffff, ver: 0xf)
Warn : JTAG tap: xcu.tap       UNEXPECTED: 0xffffffff (mfg: 0x7ff (<invalid>), part: 0xffff, ver: 0xf)
Error: JTAG tap: xcu.tap  expected 1 of 1: 0x03822093 (mfg: 0x049 (Xilinx), part: 0x3822, ver: 0x0)
Error: Trying to use configured scan chain anyway...
Error: xcu.tap: IR capture error; saw 0x3f not 0x01
Warn : Bypassing JTAG setup events due to errors
Info : gdb server disabled
RTM FPGA XADC:
TEMP 33028232.43 C
VCCINT 196608.000 V
VCCAUX 196608.000 V
VCCBRAM 196608.000 V
VPVN 196608.000 V
VREFP 196608.000 V
VREFN 196608.000 V
VCCPINT 196608.000 V
VCCPAUX 196608.000 V
VCCODDR 196608.000 V
AMC FPGA XADC:
TEMP 33028232.44 C
VCCINT 196608.000 V
VCCAUX 196608.000 V
VCCBRAM 196608.000 V
VPVN 196608.000 V
VREFP 196608.000 V
VREFN 196608.000 V
VCCPINT 196608.000 V
VCCPAUX 196608.000 V
VCCODDR 196608.000 V
loaded file /home/britton/miniconda3/envs/artiq-dev/share/bscan-spi-bitstreams/bscan_spi_xcku040-sayma.bit to pld device 1 in 3s 847509us
Error: Unknown flash device (ID 0x00ffffff)

Traceback (most recent call last):
  File "/home/britton/miniconda3/envs/artiq-dev/bin/artiq_flash", line 11, in <module>
    load_entry_point('artiq', 'console_scripts', 'artiq_flash')()
  File "/home/britton/artiq-dev/artiq/artiq/frontend/artiq_flash.py", line 371, in main
    programmer.run()
  File "/home/britton/artiq-dev/artiq/artiq/frontend/artiq_flash.py", line 191, in run
    self._client.run_command(cmdline)
  File "/home/britton/artiq-dev/artiq/artiq/remoting.py", line 56, in run_command
    subprocess.check_call([arg.format(tmp=self._tmp, **kws) for arg in cmd])
  File "/home/britton/miniconda3/envs/artiq-dev/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['openocd', '-s', '/home/britton/miniconda3/envs/artiq-dev/share/openocd/scripts', '-c', 'source [find fpga/xilinx-xadc.cfg]; interface ftdi; ftdi_device_desc "Quad RS232-HS"; ftdi_vid_pid 0x0403 0x6011; ftdi_channel 0; ftdi_layout_init 0x0098 0x008b; reset_config none; adapter_khz 5000; transport select jtag; source [find cpld/xilinx-xc7.cfg]; set CHIP XCKU040; source [find cpld/xilinx-xcu.cfg]; target create xcu.spi0.proxy testee -chain-position xcu.tap; flash bank spi0 jtagspi 0 0 0 0 xcu.spi0.proxy 0x2; target create xcu.spi1.proxy testee -chain-position xcu.tap; flash bank spi1 jtagspi 0 0 0 0 xcu.spi1.proxy 0x3; gdb_port disabled; tcl_port disabled; telnet_port disabled; init; echo "RTM FPGA XADC:"; xadc_report xc7.tap; echo "AMC FPGA XADC:"; xadc_report xcu.tap; pld load 1 {/home/britton/miniconda3/envs/artiq-dev/share/bscan-spi-bitstreams/bscan_spi_xcku040-sayma.bit}; flash probe spi0; flash erase_sector spi0 0 196; flash write_bank spi0 {/tmp/artiq_4xssp83j_top.bit} 0x0; flash verify_bank spi0 {/tmp/artiq_4xssp83j_top.bit} 0x0; flash probe spi1; flash erase_sector spi1 32 43; flash write_bank spi1 {/tmp/artiq_wtar64ct_rtm.bit} 0x200000; flash verify_bank spi1 {/tmp/artiq_wtar64ct_rtm.bit} 0x200000; flash probe spi1; flash erase_sector spi1 0 1; flash write_bank spi1 {/home/britton/artiq-builds/artiq_sayma_20180612_f8627952c8/standalone/software/bootloader/bootloader.bin} 0x0; flash verify_bank spi1 {/home/britton/artiq-builds/artiq_sayma_20180612_f8627952c8/standalone/software/bootloader/bootloader.bin} 0x0; flash probe spi1; flash erase_sector spi1 5 14; flash write_bank spi1 {/home/britton/artiq-builds/artiq_sayma_20180612_f8627952c8/standalone/software/runtime/runtime.fbi} 0x50000; flash verify_bank spi1 {/home/britton/artiq-builds/artiq_sayma_20180612_f8627952c8/standalone/software/runtime/runtime.fbi} 0x50000; xcu_program xcu.tap; exit']' returned non-zero exit status 1
gkasprow commented 6 years ago

do you observe such behaviour with another board in the crate?

marmeladapk commented 6 years ago

@gkasprow Sometimes I also got those errors on our Saymas (on bench top).

marmeladapk commented 6 years ago

Recreated on -8 board (jtag works on bench top and doesn't work in crate with exactly same error message).

jbqubit commented 6 years ago

This is now happening on AMC s/n -2. Both in crate and on bench top. I've tried several known-good USB cables. It was fine when the board first arrived from Warsaw.

hartytp commented 6 years ago

Did you add the jumper? That fixes the issue normally.

gkasprow commented 6 years ago

And make sure that FPGA is detected correctly. And the jumper must fit tightly.

jbqubit commented 6 years ago

This started happening on both my Sayma boards. I removed the jumper and slightly deformed its sockets to squeezer harder. Reinstalled jumper at same location. Now it works again. Will also try this on second Sayma soon.

hartytp commented 6 years ago

I just soldered across it. Never had issues after that...

marmeladapk commented 6 years ago

@jbqubit Is this resolved?

vmsch commented 5 years ago

I see the same error on the Sayma board (.1.91) @jbqubit sent us, even after soldering together pins 11&13 on the JTAG socket in case the jumper wasn't tight enough. I also tried different USB cables, but none worked.

This might be (un)related, but we also see problems with the RTM only starting up correctly intermittently, see issue https://github.com/sinara-hw/sinara/issues/610 .

Error message:

(artiq-dev) ion@dungeonlinux:~/steaneShared/artiqResults/binaries/5.0.dev-1085-gf60b300b/sayma/sayma_satellite$ artiq_flash -t sayma -d .
Design: top;COMPRESS=TRUE;UserID=0XFFFFFFFF;Version=2018.1
Part name: xcku040-ffva1156-1-c
Date: 2019/02/18
Time: 14:47:08
Bitstream payload length: 0xbc60f0
Design: rtm;UserID=0XFFFFFFFF;COMPRESS=TRUE;Version=2018.1
Part name: 7a15tcsg325
Date: 2019/02/18
Time: 14:13:23
Bitstream payload length: 0xba49c
Open On-Chip Debugger 0.10.0-00013-gbb7beda (2018-02-13-15:56)
Licensed under GNU GPL v2
For bug reports, read
    http://openocd.org/doc/doxygen/bugs.html
none separate
adapter speed: 5000 kHz
Info : clock speed 5000 kHz
Error: JTAG scan chain interrogation failed: all zeroes
Error: Check JTAG interface, timings, target power, etc.
Error: Trying to use configured scan chain anyway...
Error: xc7.tap: IR capture error; saw 0x00 not 0x01
Warn : Bypassing JTAG setup events due to errors
Info : gdb server disabled
RTM FPGA XADC:
TEMP -273.15 C
VCCINT 0.000 V
VCCAUX 0.000 V
VCCBRAM 0.000 V
VPVN 0.000 V
VREFP 0.000 V
VREFN 0.000 V
VCCPINT 0.000 V
VCCPAUX 0.000 V
VCCODDR 0.000 V
AMC FPGA XADC:
TEMP -273.15 C
VCCINT 0.000 V
VCCAUX 0.000 V
VCCBRAM 0.000 V
VPVN 0.000 V
VREFP 0.000 V
VREFN 0.000 V
VCCPINT 0.000 V
VCCPAUX 0.000 V
VCCODDR 0.000 V
loaded file /home/ion/anaconda3/envs/artiq-dev/share/bscan-spi-bitstreams/bscan_spi_xcku040-sayma.bit to pld device 1 in 3s 856527us
Error: Unknown flash device (ID 0x00000000)

Traceback (most recent call last):
  File "/home/ion/anaconda3/envs/artiq-dev/bin/artiq_flash", line 11, in <module>
    load_entry_point('artiq', 'console_scripts', 'artiq_flash')()
  File "/home/ion/scratch/tph/artiq/artiq/frontend/artiq_flash.py", line 385, in main
    programmer.run()
  File "/home/ion/scratch/tph/artiq/artiq/frontend/artiq_flash.py", line 199, in run
    self._client.run_command(cmdline)
  File "/home/ion/scratch/tph/artiq/artiq/remoting.py", line 56, in run_command
    subprocess.check_call([arg.format(tmp=self._tmp, **kws) for arg in cmd])
  File "/home/ion/anaconda3/envs/artiq-dev/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['openocd', '-s', '/home/ion/anaconda3/envs/artiq-dev/share/openocd/scripts', '-c', 'source [find fpga/xilinx-xadc.cfg]; interface ftdi; ftdi_device_desc "Quad RS232-HS"; ftdi_vid_pid 0x0403 0x6011; ftdi_channel 0; ftdi_layout_init 0x0098 0x008b; reset_config none; adapter_khz 5000; transport select jtag; source [find cpld/xilinx-xc7.cfg]; set CHIP XCKU040; source [find cpld/xilinx-xcu.cfg]; target create xcu.spi0.proxy testee -chain-position xcu.tap; flash bank spi0 jtagspi 0 0 0 0 xcu.spi0.proxy 0x2; target create xcu.spi1.proxy testee -chain-position xcu.tap; flash bank spi1 jtagspi 0 0 0 0 xcu.spi1.proxy 0x3; gdb_port disabled; tcl_port disabled; telnet_port disabled; init; echo "RTM FPGA XADC:"; xadc_report xc7.tap; echo "AMC FPGA XADC:"; xadc_report xcu.tap; pld load 1 {/home/ion/anaconda3/envs/artiq-dev/share/bscan-spi-bitstreams/bscan_spi_xcku040-sayma.bit}; flash probe spi0; flash erase_sector spi0 0 188; flash write_bank spi0 {/tmp/artiq_y4vr0ngx_top.bit} 0x0; flash verify_bank spi0 {/tmp/artiq_y4vr0ngx_top.bit} 0x0; flash probe spi1; flash erase_sector spi1 32 43; flash write_bank spi1 {/tmp/artiq_g8jd74bb_rtm.bit} 0x200000; flash verify_bank spi1 {/tmp/artiq_g8jd74bb_rtm.bit} 0x200000; flash probe spi1; flash erase_sector spi1 0 0; flash write_bank spi1 {./bootloader.bin} 0x0; flash verify_bank spi1 {./bootloader.bin} 0x0; flash probe spi1; flash erase_sector spi1 5 6; flash write_bank spi1 {./runtime.fbi} 0x50000; flash verify_bank spi1 {./runtime.fbi} 0x50000; xcu_program xcu.tap; exit']' returned non-zero exit status 1
sbourdeauducq commented 5 years ago

Check all power LEDs on the AMC.

gkasprow commented 5 years ago

looks like missing RTM power.

vmsch commented 5 years ago

On the AMC the bottom LED is blinking green, and on the board LD1 is continuously green, all other LEDs are off from what I can see.

On the RTM no LEDs on the back are on, but on the board LD9, LD8 and LD6, LD5, LD12, LD3 and LD14 on the back are continuously green.

vmsch commented 5 years ago

Actually there is also at least one LED on the LD10-LD22 block on the AMC on, but it is hard to see which.

vmsch commented 5 years ago

The 12V and 0V9 LEDs are the only AMC power LEDs that are on, independently of the RTM red LED being on or off.

sbourdeauducq commented 5 years ago

I sent @gkasprow a board with the same symptom, hopefully that's the same bug.

vmsch commented 5 years ago

Okay, thank you. This is independent of the RTM btw: if only the AMC is plugged in, these two power LEDs are also the only ones switched on on that AMC board.

hartytp commented 5 years ago

@gkasprow did you get a chance to look at the board @sbourdeauducq sent you? Is there anything we can do about this before for finalising v2.0?

hartytp commented 5 years ago

Ping

gkasprow commented 5 years ago

I didn't have time to play with it. I only connected it to bench supply and all supplies are OK.