Synchronization of two STEMlab 125-14 via SATA daisy chain

[popovkon]

Description of the setup:

• Device: STEMlab 125-14
• SD card image: Debian Linux
• Application: generation of 10MHz sinewave at DAC outputs
• Other relevant information:

Description of the problem:

Dear Pavel,

I am a Ph.D. student in SOKENDAI university, Japan. I have developed Low-Level RF feedback to stabilize phase and amplitude for LUCX accelerator RF-Gun. LUCX accelerator is a linear electron accelerator. I attach URL of the paper with the facility description.

https://accelconf.web.cern.ch/ipac2022/papers/tupopt066.pdf

I used your projects and FPGA images for the feedback development.

So far, I must synchronize DAC outputs of two STEMlab 125-14 boards via SATA connection. To the best of my knowledge, it is called "SATA daisy chain".

1) How to synchronize Clocking Wizard (PLL) with clock signal sent via SATA? Please, see picture below

2) Could you advice me how to synchronize two FPGA boards' DAC outputs using these outputs and inputs?

3) How do these inputs and outputs work?

Thank you in advance!

Sincerely, Popov Konstantin

[pavel-demin]

Hello Konstantin,

If I understand correctly the idea of the Red Pitaya developers, then they currently propose the following clocking configuration:

• master board outputs clock and trigger signals to both SATA connectors
• slave boards receive clock and trigger signals from master and output this clock from FPGA to ADC and DAC
• slave boards should have the resistors R25 and R26 removed and resistors R27 and R28 installed as shown on page 3 of the schematics

I have just added to this repository the following two projects that I used to test this clocking configuration:

https://github.com/pavel-demin/red-pitaya-notes/tree/master/projects/led_blinker_master https://github.com/pavel-demin/red-pitaya-notes/tree/master/projects/led_blinker_slave

[popovkon]

Dear Pavel, Thank you very much for your response!

Just to be sure, If I change the resistors' positions and apply "led_blinker_master" and "led_blinker_slave" clocking configuration, slave STEMlab 125-14 FPGA board will be locked with master board clock. I mean DAC outputs and ADC inputs. Is it so?

Sincerely, Konstantin

[pavel-demin]

Yes, it is more or less so.

There are, however, a few problems:

• the distribution of clocks via FPGA brings additional jitter
• sending 125 MHz clock through SATA cables does not seem very reliable
• the phase of the clock on the slave board must be adjusted to compensate for phase shift due to SATA cables and other elements in the path of the clock

If you only need to use DAC outputs, this can be done without changing the resistors. Resistors configure the clock input of the ADC chip. The DAC chip is always clocked from the FPGA.

[popovkon]

Thank you!

OK. I understand. So far, I need only DAC outputs at both STEMlab 125-14 boards. I will try to phase-lock DAC outputs using SATA today.

I will update the information about this issue today.

Sincerely, Konstantin

[popovkon]

Good evening, Pavel!

I have tried the following configuration for the slave board. Please, check the picture bellow.

I supplied "daisy_clock_n_i" and "daisy_clock_p_i" into Clocking wizard.

Then, I supplied 3 clocks with "locked" signal to the DAC.

It does not work. Vivado shows error. I attach error screenshot.

I do not understand what is wrong. Could you advice me how to solve this problem?

Sincerely, Konstantin

[popovkon]

I have some ideas. I will try to figure what is wrong.

[pavel-demin]

I can build bitstream for the following configuration:

# SATA connectors

delete_bd_objs [get_bd_ports adc_enc_p_o]
delete_bd_objs [get_bd_ports adc_enc_n_o]

create_bd_port -dir I daisy_trg_p_i
create_bd_port -dir I daisy_trg_n_i

create_bd_port -dir I daisy_clk_p_i
create_bd_port -dir I daisy_clk_n_i

create_bd_port -dir O daisy_trg_p_o
create_bd_port -dir O daisy_trg_n_o

create_bd_port -dir O daisy_clk_p_o
create_bd_port -dir O daisy_clk_n_o

create_bd_port -dir O adc_clk_p_o
create_bd_port -dir O adc_clk_n_o

# Create clk_wiz
cell xilinx.com:ip:clk_wiz pll_0 {
  PRIMITIVE PLL
  PRIM_IN_FREQ.VALUE_SRC USER
  PRIM_IN_FREQ 125.0
  PRIM_SOURCE Differential_clock_capable_pin
  CLKOUT1_USED true
  CLKOUT1_REQUESTED_OUT_FREQ 125.0
  USE_RESET false
} {
  clk_in1_p daisy_clk_p_i
  clk_in1_n daisy_clk_n_i
}

# Create processing_system7
cell xilinx.com:ip:processing_system7 ps_0 {
  PCW_IMPORT_BOARD_PRESET cfg/red_pitaya.xml
} {
  M_AXI_GP0_ACLK pll_0/clk_out1
}

# Create all required interconnections
apply_bd_automation -rule xilinx.com:bd_rule:processing_system7 -config {
  make_external {FIXED_IO, DDR}
  Master Disable
  Slave Disable
} [get_bd_cells ps_0]

# Create xlconstant
cell xilinx.com:ip:xlconstant const_0

# Create util_ds_buf
cell xilinx.com:ip:util_ds_buf:2.1 trg_buf_0 {
  C_SIZE 1
  C_BUF_TYPE OBUFDS
} {
  OBUF_DS_P daisy_trg_p_o
  OBUF_DS_N daisy_trg_n_o
  OBUF_IN const_0/dout
}

# Create util_ds_buf
cell xilinx.com:ip:util_ds_buf:2.1 trg_buf_1 {
  C_SIZE 1
  C_BUF_TYPE IBUFDS
} {
  IBUF_DS_P daisy_trg_p_i
  IBUF_DS_N daisy_trg_n_i
}

# Create util_ds_buf
cell xilinx.com:ip:util_ds_buf:2.1 clk_buf_0 {
  C_SIZE 1
  C_BUF_TYPE OBUFDS
} {
  OBUF_DS_P daisy_clk_p_o
  OBUF_DS_N daisy_clk_n_o
  OBUF_IN const_0/dout
}

# Create util_ds_buf
cell xilinx.com:ip:util_ds_buf:2.1 clk_buf_1 {
  C_SIZE 1
  C_BUF_TYPE OBUFDS
} {
  OBUF_DS_P adc_clk_p_o
  OBUF_DS_N adc_clk_n_o
  OBUF_IN const_0/dout
}

# LED

# Create c_counter_binary
cell xilinx.com:ip:c_counter_binary cntr_0 {
  Output_Width 32
} {
  CLK pll_0/clk_out1
}

# Create port_slicer
cell pavel-demin:user:port_slicer slice_0 {
  DIN_WIDTH 32 DIN_FROM 26 DIN_TO 26
} {
  din cntr_0/Q
  dout led_o
}

[popovkon]

Hello, Pavel!

Thank you! I understand the configuration. I am trying this configuration right now. I will tell the results soon.

Sincerely, Konstantin

[popovkon]

Dear Pavel,

Good afternoon! I tried your configuration. It works well. Here is the configuration of master board.

Here is slave board.

Here is picture of FPGA boards connection via SATA.

DAC outputs of both FPGA boards are phase-locked. There is not phase drift. So, I attach screenshots from oscilloscope. 10MHz signals screenshot is bellow.

40MHz signal is bellow.

Thank you very much for your help! Spasibo!

Sincerely, Popov Konstantin