Kasli v1.0 tests & errata

[marmeladapk]

To be changed for rev1.1

• Misc:
  • [x] R57 must be 300R
  • [x] Improve signal integrity for backplane LVDS lines (remove reference plane cut) (one cut between 2v5 and 12V0 seems unavoidable)
  • [x] Add two more USER_LEDs
  • [x] (v1.1/rj) consider adding a PCB-mount heatsink (one of the FPGAs died from overheating during testing)
  • [ ] Document power supply design parameters (#317)
  • [x] update
  • [x] and double check power budget after changes.
  • [x] add a PN for that clock balun on the schematic
  • [ ] Change revision to 1.1 on schematics
• USB:
  • [x] (v1.1/rj) Swap I2C with UART on FTDI chip (channel C doesn't have MPSSE)
  • [x] (v1.1/rj) fix direction of the IC19 (both channels)
• I2C:
  • [x] (v1.1/rj) add TVS on EEM I2C
  • [x] (v1.1/rj) add ID chip 24AA02E48T-I/OT, same I2C port as the Si5324
  • [x] (v1.1/rj) Connect reset of the I2C routers IC14 IC15 to the FPGA
  • [x] Add TCA9517 to Si5324 I2C
  • [x] Connect IC10:A1 (I2C_SW_RESET) to one of the unused BDBUS lines (BDBUS7 or BDBUS5 if you can) and not to CDBUS4.
• Backplane:
  • [x] (v1.1/rj) Inconsistent power connector PN 712RA vs L712RA (712RA is the correct one) (it was just wrong footprint name, PN was correct; PK)
  • [x] (v1.1/rj) Remove second power connector J17.
  • [x] (v1.1/rj) Populate the backplane connector J20.
• SFP:
  • [x] Move SFP LEDs closer to panel (same as was done with USB)
  • [x] Free up six pins on the FPGA by having MOD_DEF1 and MOD_DEF2 routed through the I2C switch only and not additionally directly to the FPGA.
  • [x] Remove the I2C level converters IC10 on those SFP I2C busses.
  • [x] (v1.1/rj) Connect RATE_SELECT1 (pin 9 of the SFP) to the FPGA
• Clock distribution
  • [x] (v1.1/rj) Remove the 50 MHz oscillator OSC1. Not needed as the 125 MHz GTP input can be routed to fabric.
  • [x] (v1.1/rj) Remove IC3
  • [x] Remove the CLK_SEL logic of IC3 (free up FPGA pin)
  • [x] (v1.1/rj) Remove J19 and related components
  • [x] (v1.1/rj) Remove J5-J7 and related components
  • [x] Remove the R98 path of CLK_OUT (IC1:Q2 to the front panel SMA)
  • [x] (v1.1/rj) Add an LDO for the power supply of IC1 and IC2 (also worth adding a ferrite/inductor between the SMPS and the LDO to kill high-frequency spurs).
  • [x] (v1.1/rj) Connect four MMCX (J1-J4) to the Q2, Q2B, Q3, Q3B outputs of IC1
  • [x] (v1.1/rj) Mark J2 and J4 (on the negative Q2B and Q3B outputs) clearly on the PCB
  • [x] (v1.1/rj) Connect backplane clock CLK_EXT_P/N to CKOUT2+- of IC2
  • [x] (tph) change 0R resistor to 100k to float SMA clock input
  • [x] (tph) consider floating MMCX connector grounds in same way as SMAs (only if you have time!)
  • [x] Change the transformer TR1 to TC2-1TX+. No changes to layout. Footprint and package are compatible. But pinout is rotated 180 degrees.
  • [x] DNP C13, C9
• Layout:
  • [x] Either teardrops everywhere or nowhere. Right now there are teardrops on roughly half the pins.
  • [x] Set up consistent rules
  • [x] Properly clean up polygons
  • [ ] Don't do arcs where they are not needed.
  • [x] Remove weird teardrop on FTDI
  • [x] Connect PCB edge GND stitching vias to top and bottom GND pour. Stick with @gkasprow recommendation on via density.
  • [x] Check XADC connection to components
  • [x] Replace C49 with a 0R (connect VREFN to XADC_GND) to use internal reference (connected without jumper, PK)

[gkasprow]

It does not matter if traces are routed in the middle or not. The open edge of the board forms slot antenna and irradiates EMI. That's why we place vias at the board edges that short all ground planes and form some kind of Faraday cage for internal layers.

[marmeladapk]

@jordens

Rotating SATA connector is harder than I thought. It's both too close to the edge and to the JTAG connector. Moving JTAG down would require rerouting GTP pairs (and many more signals) and is not optimal. Rotating it would create problems with P12V0 distribution and also requires rerouting. Since you said to pass if it's troublesome, I'm going to defer it to next revision.

[marmeladapk]

I found better heatsink

http://www.fischerelektronik.de/web_fischer/en_GB/heatsinks/B02/Heatsinks%20for%20BGAs/PR/ICKBGA23x23x10_/$productCard/parameters/index.xhtml

It has dimension of FPGA and is self-adhesive (previous one was smaller than FPGA).

[hartytp]

So long as the glue is tough enough to survive lab usage (e.g. being nocked by all those ribbon cables) that looks perfect!

[jordens]

@marmeladapk all ACK. Then we are good to go.

[marmeladapk]

I pushed RC2. Please check if shielding vias aren't too close to board edge.

[hartytp]

@marmeladapk Thanks for uploading the R2C release along with the change notes. That's really helpful and exactly how I think this should be done!

All looks great on the schematic now, and I'd be happy to close this and send it off for manufacture. Good job.

One thing though, I'd really like to close all outstanding Kasli issues, including #317. So, could you either add that info to the schematic and close the issue, or let's agree that this info isn't important and close the issue.

[jordens]

• For the stitching vias to have any effect, they need to connect the top and bottom ground pours (as Greg explained) and they need to be closely spaced (~four times as dense as you have them now). I don't think you are doing that. But @gkasprow knows best what they should look like and whether they are needed.
• I would not short pin 4 (B2) of IC10 to ground. Even though it's low, it's still an output.
• (my fault, sorry, but this is a bug) Connect IC10:A1 to one of the unused BDBUS lines (BDBUS7 if you can) and not to CDBUS4. I2C is on the B bus now and that's where the reset should be as well. I looked at the wrong schematic.
• I am not overly excited to see the I2C bus and the UART scattered over the banks on the FPGA. The various SFP control lines would have been better candidates for scattering. But I think this is OK for now (@sbourdeauducq could you confirm?)
• IIRC we also said that if we use a balun as input, we want a DC-isolated one. @hartytp What about the TC2-1TX+? Almost the exact same footprint, just rotated and with one pin less. Nice for pick-and place (same top hat). Works for all frequencies we might want to put into the Si5324 that I can think of right now (3-300 MHz). There are others with lower and higher 3dB frequencies but this one matches the 5-100 MHz (plus 3rd harmonic for steeper edges from square sources) and footprint best. Maybe we want to DNP C13, C9 as well then.

[hartytp]

IIRC we also said that if we use a balun as input, we want a DC-isolated one

Well, I'm still not keen on this. As discussed before, I don't think that the high isolation voltage afforded by the DC-isolated balun is useful, and it will mangle the rising edges of any clocks we feed through, as it only passes the fundamental of a 150MHz clock. This will make the clock's noise immunity much worse, as we can't feed in signals with fast rising edges.

I also object to using a different balun/isolation method here to everywhere else in Sinara. It's one more datasheet for users to have to look at, and it gives the user the impression that there is somethign special about this input. IMHO, unless there is a real need to do something different here, it would be much better to just stick with the solution that we've used everywhere else and tested carefully. (Not to mention that sticking with the current solution avoids having to do extra work).

But, that said, it's your board and your call...

---

Assuming the TC2-1TX+ is pin-compatible with the current balun, it looks okay to me (apart from the crap frequency cut-off).

[hartytp]

@jordens Also, if you want to change the balun to improve the isolation voltage then other changes are needed as well:

• DNP the 100k resistor and capacitors on the input (if the caps are still there then they limit the breakdown voltage)
• Add a ground island around the balun. This ground island must not be connected to the main pcb ground, even with capacitors (I don't really like from an EMI perspective, given that there are high-frequency signals around that are far outside the BW of your balun)
• generally, review all components around the balun and input to make sure it does what you want and works

Do we really want to go to that effort at this late stage in the design? It's not so hard, but there are a few ways of screwing things up, so I'd prefer to stick with our current balun situation, which we have tested elsewhere (e.g. clocker) and know should work...

[jordens]

@hartytp (1) The 5, 10 or 100 MHz outputs from reference source we would be using is sinusoidal. There is no point in accommodating for what we won't use. (2) In cases where Kasli is DRTIO master, I don't see anyone inputting a clock other than sinusoidal or other than 5/10/100 MHz. As you say, 150 MHz and 125 MHz are rare. (3) The previous balun has a crap lower frequency cut of as well. 5 MHz is not such an uncommon reference frequency. And loosing SNR there is so much worse than at 100 MHz because of the lower frequency. (4) I feel zero obligation to follow the Sayma clock input designs. It's an illusion that we are doing that (no squaring up chip, different PLL, different bandwidth, different application). The Urukul output stages already use a different Balun (I am fine with copying that design. It goes to higher frequencies but it's a different footprint). The Sayma design is neither known to be a good choice -- certainly no testing at all at low frequencies or with a narrow band PLL -- nor do we have the data to support the claim that the TLT is better noise-wise. (5) Remember that the typical PLL bandwidth of the Si5324 is 5-10 Hz. I have my doubts that you will see any relevant phase noise difference when using 150 MHz. At 5-10 Hz bandwidth there will be other effects dominating.

[jordens]

Clocker is tested already?

[hartytp]

Clocker is tested already?

I haven't tested the version @gkasprow built yet, but we have something essentially identical, which we've been using for quite a while and have tested very carefully (this is what was used for a lot of the phase noise plots I've posted so far).

Anyway, I don't feel strongly enough to argue with you about this. Since you're the one pushing for it, just make sure you review all changes carefully it doesn't break anything...

[hartytp]

It goes to higher frequencies but it's a different footprint

Oh, and I don't care too much about changing the footprint. I do not want to support multiple population versions for this, so let's just pick a balun and get it working.

[jordens]

On the breakdown I would keep the capacitors on the shield as they are now and also don't make an island. I am not trying to protect against discharge into the SMA shield, the case, the panel, SFP cage, 12V outer barrel. I am more worried about the signal path into the Si5324.

[jordens]

@hartytp You can also populate the TC2-72T+. With 700 MHz that's more than the required 4V/ns bandwidth the ADCLK would want. But since the is not the ADCLK948 and we don't have any data about how the Si5324 reacts to diffferent slew rates and how its 5 Hz PLL sees that, I don't see the point.

Ok. @marmeladapk Then please change the transformer TR1 to TC2-1TX+ for my boards. No changes to the layout required. Just that this transformer is rotated 180 degrees.

[marmeladapk]

SN74AVC2T245 datasheet says to "Tie any unused input and output ports directly to ground."

@gkasprow recommended me 10 mm spacing. I'll move vias closer to the middle to also connect with top and bottom ground polygons.

Other changes will have to wait till Sunday at least. I need to catch up with few things on my uni. Try not to change the specification after Sunday, since I feel like I'm trying to hit a moving target. ;)

@jordens Could you update top post with these last things you and @hartytp discussed? So it's clear which changes do you want to be implemented.

[marmeladapk]

I'd really like to close all outstanding Kasli issues, including #317

@hartytp I'll complete it, but after board is finalised.

[jordens]

@marmeladapk all ACK, done and top post updated.

[hartytp]

TR1 to TC2-1TX+ for my boards

Let's do this for all boards then. I want to minimize the number of population variants floating around for this hardware.

[jordens]

@marmeladapk I saw your note on the XADC issues. Are you using vivado with JTAG to access the XADC directly or do you have gateware that you are using to access it?

[gkasprow]

we use Vivado, I also observed such effect. But after some time it started working.

[sbourdeauducq]

I am not overly excited to see the I2C bus and the UART scattered over the banks on the FPGA. The various SFP control lines would have been better candidates for scattering. But I think this is OK for now (@sbourdeauducq could you confirm?)

I2C and UART are low-speed signals, and they are using the most regular FPGA IO (no clock-capable pins etc.). So there shouldn't be an issue with scattering them.

[jordens]

@marmeladapk "The XADC also has an on-chip reference option which is selected by connecting VREFP and VREFN to ADCGND": replace C49 with a 0R. (added to top post).

[marmeladapk]

@jordens I connected it without a jumper. C49 was in 0201 case, our jumpers are 0402, there's no way I could fit that under FPGA.

[jordens]

@marmeladapk thanks.

[marmeladapk]

@jordens If I just flip TC2-1TX+ by 180 deg. then output will be swapped (_P line with _N line). Do we care? I can connect CLK_IN to pin 4 if we want to avoid that.

(up - current balun, down, TC2-1TX+ rotaded 180 deg.)

[jordens]

@marmeladapk Correct (well observed!). The swapped polarity doesn't hurt me but if you want (and for the perfectionists inside us) swapping the connection on the primary (6-4) is nicer.

[marmeladapk]

I finished, but I'll upload documentation from my home computer, as here I've got some issues with generating it (new Altium... ).

[marmeladapk]

@jordens @hartytp I released rc3. Unless you have any objection I'd like to send it to the manufacturer.

[jordens]

@marmeladapk thanks! Give me an hour to check it.

[hartytp]

@marmeladapk looks beautiful. If I had to nit pick, I'd probably suggest adding a PN for that clock balun on the schematic. Otherwise, no comments here apart from send it off! Great work!

[jordens]

@marmeladapk Good to go! Send it!

[hartytp]

Whooo!

[marmeladapk]

@hartytp I'll fix schematics in the final version.

[marmeladapk]

Note to self: I unmasked thermal vias.

[marmeladapk]

As a part of one of my projects on uni I wrote code that controls all parameters of Si5324. It abstracts all the registers stuff, allows to set desired input clock and output frequency (but only on one output). It finds right divider values so you don't need to use DSPLLsim for this but it does so using brute force method. In all cases I tested it found right values in <0.5 second (even with odd frequencies like 192.3 MHz). Do you want me to upload it for later use?

[gkasprow]

@marmeladapk ask @jordens

[jordens]

Integrate it into the code I posted before.

[marmeladapk]

@jordens V1.1 arrived yesterday. Unfortunately pyftdi resets all other outputs when using I2C, so we have to fall back to bitbang. :/

[jordens]

@marmeladapk You mean it messes with the other channels? Yes. I have seen that too. Or do you mean it resets the other pins on the second channel? That should and can be fixed in pyftdi as well. In both cases, could you file a bug with pyftdi if there isn't one yet?

[jordens]

Anyway, bitbanging I2C is just fine.

[marmeladapk]

@jordens @hartytp Unless anyone has any objections to schematics, I'd like to publish v1.1 and upload board files.

[hartytp]

Go for it.

[jordens]

:+1:

[marmeladapk]

v1.1 is released, so I'll go ahead and close this.