search

sinara-hw / sinara

Sayma AMC/RTM issue tracker
Other
42 stars 7 forks source link

bad CODEGRPSYNCFLG #585

Closed hartytp closed 5 years ago

hartytp commented 5 years ago 
 __  __ _ ____         ____ 
|  \/  (_) ___|  ___  / ___|
| |\/| | \___ \ / _ \| |    
| |  | | |___) | (_) | |___ 
|_|  |_|_|____/ \___/ \____|

MiSoC Bootloader
Copyright (c) 2017-2018 M-Labs Limited

Bootloader CRC passed
Gateware ident 4.0.dev+1400.g69d060b6;standalone.without-sawg
Initializing SDRAM...
DQS initial delay: 110 taps
Write leveling scan:
Module 3:
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111101100000000000000000000000000000000000000000
Module 2:
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111101101000000000000000000000000000
Module 1:
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000011111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111011011000000000000000000000000000000000000000000000000000000000000000
Module 0:
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111011010000000000000000000000000000000000000000000000000000000000000000
DQS initial delay: 110 taps
Write leveling: 102 112 139 128 done
Read leveling scan:
Module 3:
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001001111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111110010010100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Module 2:
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000011111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111101010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Module 1:
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001101111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111100010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Module 0:
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000010111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111110100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Read leveling: 223+-85 211+-91 195+-85 180+-91 done
SDRAM initialized
Memory test passed

Booting from flash...
Starting firmware.
[     0.000006s]  INFO(runtime): ARTIQ runtime starting...
[     0.003867s]  INFO(runtime): software ident 4.0.dev+1400.g69d060b6;standalone.without-sawg
[     0.012130s]  INFO(runtime): gateware ident 4.0.dev+1400.g69d060b6;standalone.without-sawg
[     0.020771s]  INFO(runtime): log level set to INFO by default
[     0.026297s]  INFO(runtime): UART log level set to INFO by default
[     0.032260s]  INFO(board_artiq::slave_fpga): Loading slave FPGA gateware...
[     0.039210s]  INFO(board_artiq::slave_fpga):   magic: 0x5352544d, length: 0x000b5ce8
[     0.046937s]  INFO(board_artiq::slave_fpga):   DONE before loading
[     0.999431s]  INFO(board_artiq::slave_fpga):   ...done
[     1.003243s]  INFO(board_artiq::serwb): waiting for AMC/RTM serwb bridge to be ready...
[     1.017001s]  INFO(board_artiq::serwb):  ...done.
[     1.020374s]  INFO(board_artiq::serwb): RTM to AMC link test...
[     2.502676s]  INFO(board_artiq::serwb):   ...passed
[     2.506222s]  INFO(board_artiq::serwb): AMC to RTM link test...
[     3.988532s]  INFO(board_artiq::serwb):   ...passed
[     3.992088s]  INFO(board_artiq::serwb): Wishbone test...
[     5.966622s]  INFO(board_artiq::serwb):   ...passed
[     5.970425s]  INFO(board_artiq::serwb): RTM gateware version 4.0.dev+1400.g69d060b6
[     5.977811s]  INFO(runtime): press 'e' to erase startup and idle kernels...
[     6.977006s]  INFO(runtime): continuing boot
[     7.239248s]  INFO(board_artiq::si5324): waiting for Si5324 lock...
[    13.538077s]  INFO(board_artiq::si5324):   ...locked
[    13.541849s]  INFO(board_artiq::hmc830_7043::hmc830): loading HMC830 configuration...
[    13.549790s]  INFO(board_artiq::hmc830_7043::hmc830):   ...done
[    13.555435s]  INFO(board_artiq::hmc830_7043::hmc830): setting HMC830 dividers...
[    13.562902s]  INFO(board_artiq::hmc830_7043::hmc830):   ...done
[    13.568720s]  INFO(board_artiq::hmc830_7043::hmc830): waiting for HMC830 lock...
[    13.576141s]  INFO(board_artiq::hmc830_7043::hmc830):   ...locked
[    13.582374s]  INFO(board_artiq::hmc830_7043::hmc7043): enabling HMC7043
[    13.599078s]  INFO(board_artiq::hmc830_7043::hmc7043): loading configuration...
[    13.616617s]  INFO(board_artiq::hmc830_7043::hmc7043):   ...done
[    13.621301s]  INFO(board_artiq::hmc830_7043::hmc7043): testing GPO...
[    13.628440s]  INFO(board_artiq::hmc830_7043::hmc7043):   ...passed
[    13.644636s]  INFO(board_artiq::ad9154): AD9154-0 initializing...
[    13.656609s]  INFO(board_artiq::ad9154):   ...done
[    13.736521s]  WARN(board_artiq::ad9154): AD9154-0 config attempt #1 failed (bad SYNC)
[    13.753340s]  INFO(board_artiq::ad9154): AD9154-0 initializing...
[    13.765270s]  INFO(board_artiq::ad9154):   ...done
[    13.840015s]  WARN(board_artiq::ad9154): AD9154-0 config attempt #2 failed (bad CODEGRPSYNCFLG)
[    13.857703s]  INFO(board_artiq::ad9154): AD9154-0 initializing...
[    13.869632s]  INFO(board_artiq::ad9154):   ...done
[    13.945427s]  WARN(board_artiq::ad9154): AD9154-0 config attempt #3 failed (bad CODEGRPSYNCFLG)
[    13.963114s]  INFO(board_artiq::ad9154): AD9154-0 initializing...
[    13.975044s]  INFO(board_artiq::ad9154):   ...done
[    14.058459s]  WARN(board_artiq::ad9154): AD9154-0 config attempt #4 failed (bad CODEGRPSYNCFLG)
[    14.076147s]  INFO(board_artiq::ad9154): AD9154-0 initializing...
[    14.088080s]  INFO(board_artiq::ad9154):   ...done
[    14.165520s]  WARN(board_artiq::ad9154): AD9154-0 config attempt #5 failed (bad CODEGRPSYNCFLG)
[    14.183208s]  INFO(board_artiq::ad9154): AD9154-0 initializing...
[    14.195137s]  INFO(board_artiq::ad9154):   ...done
[    14.272262s]  WARN(board_artiq::ad9154): AD9154-0 config attempt #6 failed (bad SYNC)
[    14.289076s]  INFO(board_artiq::ad9154): AD9154-0 initializing...
[    14.301006s]  INFO(board_artiq::ad9154):   ...done
[    14.376637s]  WARN(board_artiq::ad9154): AD9154-0 config attempt #7 failed (bad CODEGRPSYNCFLG)
[    14.394324s]  INFO(board_artiq::ad9154): AD9154-0 initializing...
[    14.406254s]  INFO(board_artiq::ad9154):   ...done
[    14.488073s]  WARN(board_artiq::ad9154): AD9154-0 config attempt #8 failed (bad CODEGRPSYNCFLG)
[    14.505760s]  INFO(board_artiq::ad9154): AD9154-0 initializing...
[    14.517688s]  INFO(board_artiq::ad9154):   ...done
[    14.593830s]  WARN(board_artiq::ad9154): AD9154-0 config attempt #9 failed (bad SYNC)
[    14.610644s]  INFO(board_artiq::ad9154): AD9154-0 initializing...
[    14.622574s]  INFO(board_artiq::ad9154):   ...done
[    14.703689s]  WARN(board_artiq::ad9154): AD9154-0 config attempt #10 failed (bad SYNC)
[    14.710307s] ERROR(board_artiq::ad9154): failed to initialize AD9154-0: bad SYNC
[    14.728327s]  INFO(board_artiq::ad9154): AD9154-1 initializing...
[    14.740143s]  INFO(board_artiq::ad9154):   ...done
[    14.816398s]  WARN(board_artiq::ad9154): AD9154-1 config attempt #1 failed (bad SYNC)
[    14.833213s]  INFO(board_artiq::ad9154): AD9154-1 initializing...
[    14.845029s]  INFO(board_artiq::ad9154):   ...done
[    14.922379s]  WARN(board_artiq::ad9154): AD9154-1 config attempt #2 failed (bad CODEGRPSYNCFLG)
[    14.940066s]  INFO(board_artiq::ad9154): AD9154-1 initializing...
[    14.951883s]  INFO(board_artiq::ad9154):   ...done
[    15.027843s]  WARN(board_artiq::ad9154): AD9154-1 config attempt #3 failed (bad SYNC)
[    15.044659s]  INFO(board_artiq::ad9154): AD9154-1 initializing...
[    15.056478s]  INFO(board_artiq::ad9154):   ...done
[    15.138966s]  WARN(board_artiq::ad9154): AD9154-1 config attempt #4 failed (bad SYNC)
[    15.155782s]  INFO(board_artiq::ad9154): AD9154-1 initializing...
[    15.167598s]  INFO(board_artiq::ad9154):   ...done
[    15.249528s]  WARN(board_artiq::ad9154): AD9154-1 config attempt #5 failed (bad SYNC)
[    15.266343s]  INFO(board_artiq::ad9154): AD9154-1 initializing...
[    15.278160s]  INFO(board_artiq::ad9154):   ...done
[    15.352957s]  WARN(board_artiq::ad9154): AD9154-1 config attempt #6 failed (bad SYNC)
[    15.369772s]  INFO(board_artiq::ad9154): AD9154-1 initializing...
[    15.381590s]  INFO(board_artiq::ad9154):   ...done
[    15.455351s]  WARN(board_artiq::ad9154): AD9154-1 config attempt #7 failed (bad CODEGRPSYNCFLG)
[    15.473038s]  INFO(board_artiq::ad9154): AD9154-1 initializing...
[    15.484855s]  INFO(board_artiq::ad9154):   ...done
[    15.559776s]  WARN(board_artiq::ad9154): AD9154-1 config attempt #8 failed (bad SYNC)
[    15.576591s]  INFO(board_artiq::ad9154): AD9154-1 initializing...
[    15.588407s]  INFO(board_artiq::ad9154):   ...done
[    15.663810s]  WARN(board_artiq::ad9154): AD9154-1 config attempt #9 failed (bad CODEGRPSYNCFLG)
[    15.681497s]  INFO(board_artiq::ad9154): AD9154-1 initializing...
[    15.693315s]  INFO(board_artiq::ad9154):   ...done
[    15.780453s]  WARN(board_artiq::ad9154): AD9154-1 config attempt #10 failed (bad CODEGRPSYNCFLG)
[    15.787941s] ERROR(board_artiq::ad9154): failed to initialize AD9154-1: bad CODEGRPSYNCFLG
[    15.796536s]  INFO(board_artiq::jesd204sync): calibrating SYSREF phase offset at FPGA...
[    15.813857s]  INFO(board_artiq::jesd204sync):   ...done, phase offset: 137
[    15.824821s]  INFO(board_artiq::jesd204sync): aligning SYSREF with RTIO...
[    15.851043s]  INFO(board_artiq::jesd204sync):   ...done (64/64 slips)
[    15.859210s]  INFO(board_artiq::jesd204sync):   margins at FPGA: -17 +17
[    15.864981s]  INFO(board_artiq::jesd204sync): verifying SYSREF margins at DAC-0...
[    15.962524s]  INFO(board_artiq::jesd204sync):   margins: -0 +69
[    15.967125s] ERROR(runtime): failed to align SYSREF at DAC: SYSREF margins at DAC are too small, board needs recalibration
[    15.978156s]  INFO(board_artiq::hmc542): card 0 channel 0 set to 4 dB
[    15.986675s]  INFO(board_artiq::hmc542): card 0 channel 1 set to 4 dB
[    15.993899s]  INFO(board_artiq::hmc542): card 1 channel 0 set to 4 dB
[    16.001122s]  INFO(board_artiq::hmc542): card 1 channel 1 set to 4 dB
[    16.008346s]  INFO(board_artiq::hmc542): card 2 channel 0 set to 4 dB
[    16.015569s]  INFO(board_artiq::hmc542): card 2 channel 1 set to 4 dB
[    16.022796s]  INFO(board_artiq::hmc542): card 3 channel 0 set to 4 dB
[    16.030022s]  INFO(board_artiq::hmc542): card 3 channel 1 set to 4 dB
[    16.037479s]  INFO(runtime): using MAC address 34-45-32-12-ff-22
[    16.042437s]  INFO(runtime): using IP address 192.168.95.176
[    16.049459s]  INFO(runtime::mgmt): management interface active
[    16.062605s]  INFO(runtime::session): accepting network sessions
[    16.076550s]  INFO(runtime::session): running startup kernel
[    16.081287s]  INFO(runtime::session): no startup kernel found
[    16.086724s]  INFO(runtime::session): no connection, starting idle kernel
[    16.093870s]  INFO(runtime::session): no idle kernel found
hartytp commented 5 years ago

@marmeladapk I looked at the output of the HMC830 on my board on a fast oscilloscope. It seems locked, but sometimes it jumps to a different frequency. I'll post a video in a second. But, if that is happening on other boards it could explain some of the issues we've seen!

NB I'm running my HMC830 output at 2.4GHz not 1.2GHz

hartytp commented 5 years ago

Probing on the UFL connectors.

It was at 2.4GHz at first, but then it jumped and is now wandering between 2.9GHz and 3GHz.

https://drive.google.com/open?id=1OgUiDcEbpp-GNcApitAWPpE_TnNefFv1

The signal at the reference input looks a bit weaker than I expected (only about 400mVpp)

hartytp commented 5 years ago

Okay, another observation: the UART reports that the HMC830 is locked but when I trigger a scope from the HMC830 reference input and look at the RF output on J59 it's pretty clear that it isn't locking even transiently.

I don't have a fast enough probe to look at the HMC830 output directly so it's possible that the 3GHz RF I'm seeing is actually just a clock buffer free-oscillating.

hartytp commented 5 years ago

@marmeladapk is there any way you can track down the board I sent you and post it to me? That board didn't have any of these issues, and this is just killing me

gkasprow commented 5 years ago

@hartytp I observed output with spectrum analyser and to make sure it locks, I changed the input frequency by a few tens of Hz to observe if output follows it

hartytp commented 5 years ago

@gkasprow

hartytp commented 5 years ago

Cool. Poked around a bit to find what was oscillating and it started working! Whoo!

hartytp commented 5 years ago

Anyway, @gkasprow one thing I still find concerning is that it is clearly possible for the HMC830 to indicate lock when it's not in fact locked. That's a pain and important to know