SYNC_IN jitter

[jordens]

The jitter on the SYNC_IN signal from Kasli to the AD9910 (throught the LVDS buffers and the fanout) is very high in some caes (the tester setup connected to the buildbot).

At validation delay 1 (hold and setup margin 1 tap) the window is just 2 taps wide (a tap is about 75 ps). http://buildbot.m-labs.hk/builders/artiq/builds/2669/steps/python_unittest_2/logs/stdio

This is the SMP_ERR matrix on tester, rows are increasing validation delay, columns are SYNC_IN delay on the AD9910.:

[1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

There also seems to be some bounce around the edges (top row).

On the systems I have here I get about 5-6 tap wide windows at validation delay 1. That's not stellar but OK. When using the SYNC signal on board from the first DDS, the window at validation delay 1 is 8 taps wide on tester, 8-9 taps here. Assuming equal tap delay for the validation delays and the SYNC_IN delays, the theoretically best case is validation delay 4 and a window width of ~4 or a validation delay of 1 and a window width of ~10 (i.e. SYNC_IN delay periodicity minus twice the validation delay).

[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0]
[1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
[1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
[1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
[1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

In both cases Kasli/v1.1 and Urukul-AD9910/v1.3, connected via MMCX to Kasli-J1.

The jitter seems to come in part from Urukul and in part (the larger part) from Kasli. And it varies between setups. I changed a couple things (see the artiq changelog) to optimize jitter on the RTIO clock but there was little effect. From Vivado the max peak-peak jitter on the clock driving the SYNC output buffer in the FPGA is ~90ps. I tried running the SYNC fanout from the supposedly quieter P1V8A rail but that doesn't seem to work at all.

@gkasprow @marmeladapk could you have a look at the jitter on SYNC_IN (EEM0:7, before and after the sync fanout, compared to the Kasli MMCX clock)?

@cjbe @klickverbot When you were playing with SYNC, did you look at SMP_ERR? How did you select the SYNC_IN delay and the validation delay? Did you scan them?

c.f. m-labs/artiq#1143

[marmeladapk]

On DDS sync_in pads:

On EEM 7 connector:

[jordens]

60ps would be awesome. Maybe the level is just too high and we overdrive it (DC or AC). OTOH, according to the datasheet, SYNC_IN is supposed to be LVDS with internal 100 R termination and the Si53304 output schematic looks correct.

[marmeladapk]

@jordens 60 ps from 10 Hz to 20 MHz (grey area was not integrated)

[jordens]

Yes. From what I have seen the problem is not on longer timescales (already observed within a few µs). I.e. either FM radio is really getting into the SYNC signal or it's somewhere/something else.

[hartytp]

@marmeladapk did you make any more progress on this? I'd really like to get the next revision of Urukul out soon.

Also, did you try running the test script on your board to see what window size it reported? We should check you can reproduce the issue (narrow windows reported on the ARTIQ scan) before spending too much time measuring noise.

[marmeladapk]

@hartytp I ran the test script from ARTIQ and measured similar windows as Robert did. However I couldn't find this jitter anywhere on the sync_in path.

[hartytp]

Interesting. So to check I understand the situation:

• you measured the differential AD9910 SYNC_IN signal for one of the DDSs (DDS0?) as close to the DDS chip as possible, with both Kasli and DDS0 as the SYNC source? I assume the signal levels etc are the same for both cases since they're driven from the same clock source. Is the frequency of both signals the same, or is there any other difference?
• the measured jitter was the same for both cases? and the jitter was very low compared with the 1ns sampling window. Was your measurement sensitive enough to spot infrequent glitches (which wouldn't necessarily show up on a power spectrum)? Did you look at the signals on a scope on persist mode?
• the DDS sync validation scan reports different windows for the two cases.

If you checked all that then I'm baffled as to what the source of this issue could be!

One other thing that would be interesting would be to look at the differential SYNC_IN signal right by the DDS when it's driven by the FPGA, and also look at the DDS SYNC_OUT. Put both on a scope on persist and leave for a few minutes. That might show something up.

What do you think?

[marmeladapk]

@hartytp I ran test from artiq and waited for signal on SYNC_IN. Then I measured with scope and with SSA signals on sync_in, after clock fanout (just to check) and on EEM connector. Screenshots are here https://github.com/sinara-hw/Urukul/issues/16#issuecomment-483627251

I wanted to measure the signal that was problematic during the test.I didn't look with the persist mode.

[hartytp]

@marmeladapk thanks. Well, I'm a bit baffled as to what this could be. My only thought is that maybe there are infrequent glitches on the EEM SYNC, which wouldn't have shown up on the tests you did.

Please can you do this one test:

• On a scope simultaneously measure the SYNC_OUT, SYNC_IN (differential), and DDS clock
• Leave that on persist for a while
• do this for each SYNC source

As I said, my only thought is that maybe the FPGA has infrequent glitches or something that kill the SYNC window without contributing much noise power. That test should show it up. If not, I have no idea what the issue is (DDS configuration?).

any other ideas @jordens ?

[jordens]

As at the top it's not infrequent. A few dozen µs are already sufficient. I would also compare with using SYNC_OUT and I would triple check common mode and differential signal levels as it looked like double windows sometimes (see above as well). @hartytp could you check the configuration?

[hartytp]

@marmeladapk can you have a go at the measurements? I'll check the AD9910 software configuration when I have a minute.

[hartytp]

@marmeladapk do you think you’ll have time to do that measurement this week? I’d like to finalise the next version of Urukul soon, but would like to do this first

[hartytp]

@marmelada if/when you do look at this please try to get as many interesting signals on the scope at once as possible (e.g. FPGA SYNC, DDS SYNC_OUT, REF CLK, etc) also please get a ms or so on persist to check for glitches.

[marmeladapk]

@hartytp I've got 4 channels on my scope, 3 are taken by SYNC_OUT, SYNC_IN, and DDS clock. Sync out seems to be disabled. I use test_ad9910 script. How can I change SYNC source in it?

[jordens]

The datasheet describes which bits to set to enable SYNC_OUT. Is the ringing real? There was none in https://github.com/sinara-hw/Urukul/issues/16#issuecomment-483627251.

[marmeladapk]

@jordens No, it's not.

[dnadlinger]

@marmeladapk: Also see https://github.com/m-labs/artiq/blob/master/artiq/coredevice/ad9910.py#L684.

[marmeladapk]

Thanks @dnadlinger!

[hartytp]

Thanks @marmeladapk! To check:

• SYNC_IN is measured at the DDS and is only one half of the LVDS pair, right?
• SYNC_OUT is measured at the DDS as well?

[hartytp]

@marmeladapk one final thing: that was with the DDS SYNC driven by the FPGA. What does it look like if you take the same measurement but with the DDS using the internal SYNC (so we can see both DDS sync out and the SYNC that's looped back through the buffer)?

[hartytp]

I'll find some time this week to go over the code again and compare it to the way we used to do this in our setup.

[marmeladapk]

@hartytp SYNC signals are measured at the DDS. Differential signals go through baluns. Do I need to change something in registers to use internal SYNC (is this SYNC internal to DDS or to Urukul?)

[hartytp]

Thanks for clarifying. You need to change this https://github.com/m-labs/artiq/blob/704b5c63054fe6d62b2afd88f91708af0e88eea6/artiq/coredevice/ad9910.py#L684

[dnadlinger]

(You'll also need to set the sync_sel bit on the CPLD – there is an API – to change inputs on the clock buffer.)

[hartytp]

@marmeladapk please can you post the results of the ARTIQ eye scan for your board as well, so that we have that. And, if you haven't done that, stick a scope on all the Urukul supplies to check for excessive noise/wrong DC levels/etc.

[marmeladapk]

Previous measurement with sync_sel 0 (screenshot of measurement here https://github.com/sinara-hw/Urukul/issues/16#issuecomment-496986388) :

.F.1 [100, 100, 92, 16] [100, 89, 0, 25]
.........6.10768e-05
.8.3208e-06
..[0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0]
[0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
12 0 [0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0]

Measurement with sync_sel set to 1:

..[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0]
[1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
20 2 [1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1]

One time sync_in returned to normal level, without me touching the setup. ed: Nvm, cable broke off, will post new images in a minute

[hartytp]

@marmeladapk let me check I understand this data:

Previous measurement:

So, that was using the ARTIQ test routine, with the FPGA generating the DDS SYNC_IN, right?

Measurement with sync_sel set to 1:

This is the scope trace that goes along with the previous eye scan, right? Why is the SYNC_IN amplitude so much lower now?

Am I misunderstanding things, or is the observation here that the FPGA SYNC_IN amplitude at the DDS is much lower during the eye scans than it was during your previous measurements?

[marmeladapk]

@hartytp see new images, cable broke off.

[hartytp]

Okay, so to be clear, the first eye scan there is with SYNC_SEL set to 0, which corresponds to the FPGA generating SYNC_IN. This looks a lot like the eye scan @jordens posted above, including ~the fact that the eye width is not constant as the delay is scanned over multiple periods~ the SYNC errors in the middle of the eyes.

The scope trace and the second eye scan were taken using SYNC_SEL set to 1, which corresponds to DDS0 generating the SYNC_IN. The only difference between those two traces was using the CPLD to change SYNC_SEL (e.g. no different registers programmed into the AD9910 by ARTIQ; the DDS was still generating a SYNC_OUT signal; the FPGA was still generating a SYNC_IN (although it wasn't routed to the DDS by the mux) signal, etc). This shows a good eye.

However, the scope measurements also show that the SYNC_IN signal reaching the DDS is equally good in both cases, so there should be no difference. Is that all correct?

[hartytp]

And, one final question, but you looked at the FPGA SYNC trace on the scope in persist mode during the eye scans to check there were no glitches on it? If so, I'm completely baffled.

I checked the ARTIQ AD9910 configuration and can't see anything wrong with it yet.

[marmeladapk]

@hartytp All yes (although it's unclear for me why the second eye is good). Sync in polarity changed with respect to DDS_CLKB.

Which signal do you mean by FPGA SYNC?

[hartytp]

Which signal do you mean by FPGA SYNC?

I mean the SYNC_IN signal generated by the FPGA on EEM0[7] that goes into CLK0. I'm just trying to check that the only difference between the two configurations is the mux select line (e.g. to rule out some grounding/EMI issue when the FPGA SYNC_IN signal is present).

[hartytp]

(NB by default ARTIQ disables the DDS SYNC_OUT when the FPGA SYNC is present. THat's not necessary but should do no harm, but seemed best to avoid doing it to try to keep the two configurations as similar as possible).

[hartytp]

One other question, in the bottom eye scan (DDS0 SYNC used) do you know why we don't see any eyes after the first two rows? That looks like odd behaviour. Is that repeatable?

[hartytp]

One final thought on this: can you repeat the two eye scans on a different channel on the same card, please?

It looks as thought the DDS responds differently to the externally/internally generated SYNC_IN signals, despite them having nearly identical signal + noise levels. That's very odd. On DDS1, both SYNC_IN signals are "external" to the DDS chip (either from DDS0 or from the FPGA), so it would be interesting to see if that DDS also treats the two signals differently.

[hartytp]

Sync in polarity changed with respect to DDS_CLKB.

I guess that's expected since it depends on the length of the cables used. The AD9910 SYNC_IN delay should allow this to be taken care of anyway assuming it's working correctly.

[hartytp]

Other than those questions (and assuming you've quickly stuck a scope on all power lines and important signal lines like SYNC_SEL etc to sanity check them), I'm out of ideas. So, unless anyone else has something to suggest I think we should press on with the next hw rev. There are other important things i want to see fixed.

[marmeladapk]

Diff:

+++ b/artiq/coredevice/ad9910.py
@@ -681,7 +681,7 @@ class AD9910:
         self.write32(_AD9910_REG_SYNC,
                      (window << 28) |  # SYNC S/H validation delay
                      (1 << 27) |  # SYNC receiver enable
-                     (0 << 26) |  # SYNC generator disable
+                     (1 << 26) |  # SYNC generator enable !!!changed

...

@@ -114,7 +114,8 @@ device_db.update(
             "io_update_device": "ttl_urukul0_io_update",
             "sync_device": "ttl_urukul0_sync",
             "refclk": 125e6,
-            "clk_sel": 2
+            "clk_sel": 2,
+            "sync_sel": 1

When I measured signal on EEM connector here https://github.com/sinara-hw/Urukul/issues/16#issuecomment-483627251 I didn't use persistence.

When I power cycle Urukul I get this when I run tests with sync_sel 1:

..[0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]
[0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
10 3 [1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

Second run:

..[0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0]
[0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
23 3 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1]

So previous scan could have been affected by not power cycling. Also sinc in phase shifted.

[hartytp]

When I measured signal on EEM connector here #16 (comment) I didn't use persistence.

Please can you do that? I'd like to check that there aren't occasional glitches on that line.

[marmeladapk]

@hartytp This will probably break cables I use for measurement, so I'd rather not right now.

[marmeladapk]

Power supplies seem clean, sync sel is 1.

Another measurement, power cycled, sync sel still 1:

..[0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]
[0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
14 0 [0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]

[hartytp]

It's really odd that you don't see any eyes beyond the largest two windows. Contrast that to the measurements that @jordens posted at the top of this issue. Something is definitely wrong there!

[hartytp]

@hartytp This will probably break cables I use for measurement, so I'd rather not right now.

What will break the cables. To be clear, what I'm suggesting is taking the same measurement you're already taking, but with the FPGA driving SYNC_IN, so no need to touch any cables.

I think it's very important to get the FPGA-driven SYNC_IN measurement with the scope on persist since about the only idea I have for this is that there may be an issue with cross-talk in the EEM cable that leads to increased jitter on the FPGA SYNC_IN signal during SPI transactions.

[hartytp]

Beyond that, I can't see any more hw tests to perform.

[marmeladapk]

can you repeat the two eye scans on a different channel on the same card, please?

@hartytp Sorry, I don't know how.

To be clear, what I'm suggesting is taking the same measurement you're already taking, but with the FPGA driving SYNC_IN, so no need to touch any cables.

Right, so sync_sel 0 measurement was taken with persistence. However I measure all signals on DDS pads not on EEM connector.

Another eye with sync_sel = 1. Sync In phase is different:

..[0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1]
[0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1]
[0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
11 3 [1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

[hartytp]

However I measure all signals on DDS pads not on EEM connector.

AAh, okay, that's fine then. If is no glitch at the DDS pad then that's all that matters.

[hartytp]

Another eye with sync_sel = 1. Sync In phase is different:

Is there a correlation between the initial phase of SYNC_IN and the quality of the eye scans you see? That could explain a lot.

[hartytp]

Anyway, other than that, I think you've made all the measurements I can think of now, so I suggest we move on to the next hw rev.

[marmeladapk]

So SYNC_IN phase changes with each power cycle, and with each power cycle eyes are different. Not sure if there's any correlation, I would have to do more power cycles and screenshots.

[hartytp]

and with each power cycle eyes are different

As in, the size of the eyes are different (not just their offsets)?

This is starting to feel more like an issue with the AD9910 sync itself than with Urukul.

FWIW, last time we used AD9910 sync reliably we clocked our FPGAs from the DDS SYNC_CLK out, so some of these phase relationships were fixed (i.e. the way it's done in the app notes). I didn't think that should make a difference, but it's the only obvious difference I see between the setups.