search

sinara-hw / sinara

Sayma AMC/RTM issue tracker
Other
42 stars 7 forks source link

Kasli v1.0 tests & errata #349

Closed marmeladapk closed 6 years ago

marmeladapk commented 6 years ago

To be changed for rev1.1

jordens commented 6 years ago

@marmeladapk @gkasprow Is Kasli back from the board house?

gkasprow commented 6 years ago

Components are delivered, the PCBs were shipped yesterday. They will start assembly on Monday. It takes roughly 2 days.

hartytp commented 6 years ago

Wonderful! Really looking forward to seeing these!

gkasprow commented 6 years ago

Kasli photo: 2017-12-15 14 41 27

gkasprow commented 6 years ago

Power supplies work well. It consumes 0.2A@12V USB works as well. At least I see 4 serial ports under Windows. 2017-12-15 16 52 00 That's all what I can do at home.

hartytp commented 6 years ago

Great!

jordens commented 6 years ago

Awesome!

gkasprow commented 6 years ago

Progress; JTAG works, FLASH programming and booting works. There is litle bug, R57must be 300R I loaded design which causes 2.5V rail to switch off. With little help of external PSU it wakes up but the board takes 600mA and FPGA gets really hot. Then we realised that it must be split termination that is switched on on all EEM LVDS lines because there are just ordinary registers attached so far. And indeed, the construct below instantiates uncallibrated split impedance


-- LVDS bidir buffers
    eem0 : for I in 0 to 7 generate
        lvds_buf : IOBUFDS_INTERMDISABLE
            GENERIC MAP (
                DIFF_TERM         => "true",
                IBUF_LOW_PWR      => "true",
                IOSTANDARD        => "LVDS_25",
                USE_IBUFDISABLE   => "false")
            PORT MAP (
                O                 => eem0_I(I),
                IO                => eem0_p(I),
                IOB               => eem0_n(I),
                INTERMDISABLE     => eem0_term_dis(I),
                I                 => eem0_O(I),
                IBUFDISABLE       => '0',
                T                 => eem0_S(I) -- 3-state enable input, high=input, low=output
            );
gkasprow commented 6 years ago

clock distribution works as well

jordens commented 6 years ago

Great job @gkasprow and @marmeladapk.

gkasprow commented 6 years ago

@jordens The only remaining thing we need to test is Silabs, I2C distribution and EEMs itself. Hopefully we will ship it in Wednesday. We can ship it as it is tomorrow and play with remaining boards that will be delivered on Wednesday.

jordens commented 6 years ago

No need to rush it then. I'll be gone Friday until the 28th. You can time it so that it arrives on the 28th.

gkasprow commented 6 years ago

I2C and silabs chip also work.

gkasprow commented 6 years ago

UART works as well. I talk with FORTH CPU

marmeladapk commented 6 years ago

@jordens I removed arcs remark, because they were added on purpose - reducing spikes of current density. Same as with you last remark about polygons (believe me, Greg bugged me about this).

We need to switch UART with I2C on FTDI chip, because channel C doesn't have MPSSE. I made bitbang I2C using pyftdi which works (I can program Si5324 and I2C switches), but its timings are horrible.

SFP0 and SFP2 seem to work on 6,25 Gbps. There is a problem with SFP1 TX line. GTP PLL locks both on 125 MHz source (OSC2) and on 125 MHz signal from Si5324 (which had 125 MHz input on SMA). sf0-sfp2-ber6g sf1-sfp2-ber6g

marmeladapk commented 6 years ago

@jordens Also FPGA tends to get quite hot with termination enabled so please cool it with fans to avoid damage. Also, there is some kind of a problem with XADC, sometimes it shows all voltages = 0 V and temperature = 0 K.

hartytp commented 6 years ago

@jordens Also FPGA tends to get quite hot with termination enabled so please cool it with fans to avoid damage. Also, there is some kind of a problem with XADC, sometimes it shows all voltages = 0 V and temperature = 0 K.

Should we add a heat sink to the FPGA to help to avoid this? Maybe a nice PCB-mount one?

jordens commented 6 years ago

@marmeladapk Current density at a bend diverges with sharpness/mesh size in many Maxwell approximations. If you think you need an arc, that's most likely a simulation/algorithm artefact which does not correspond to reality. I'd be very surprised if there was a case on Kasli where an arc solves a problem that a pair of 45 degree bends does not solve. Both on polygons and on traces.

Ack the UART/I2C switch. Could you post the bitbang i2c code somewhere?

Don't worry about the XADC. Did you test the DDR RAM?

On the GTP pairs you need to be a bit more careful with your design rules. You have some lines coupled closer to unrelated signals than to their differential pair partner!

jordens commented 6 years ago

@marmeladapk can't get jtag over usb to work. are DIR1 and DIR2 of IC19 wrong?

jordens commented 6 years ago

Either teardrops everywhere or nowhere. Right now there are teardrops on roughly half the pins. image Set up consistent rules image Properly clean up polygons image Don't do arcs where they are not needed.image\

image 3.5 mil gap near IC18. Is that ok? Did they fix that in the fab?

sbourdeauducq commented 6 years ago

@marmeladapk @gkasprow ping

gkasprow commented 6 years ago

@jordens in IC17, the DIR is H so it means A -> B direction which is correct But on IC19 channel 1 direction is B->A which is wrong. Channel 2 direction is A -> B which is also wrong. This explains why it does not work You can simply short pins 8, 5 and 9,4, we will fix it the right way on remaining boards. obraz

jordens commented 6 years ago

Yes, that's what I said. Could you implement your fix and show me (photo) what you mean? Just shorting still has the same contention issue. You know better where to cut and scratch the lines.

jordens commented 6 years ago

I have MiSoC running (spi flash XIP, boot from spi flash, SDRAM working, uart working). Uses 4W.

gkasprow commented 6 years ago

Shorting of the input with output via 220R resistors should do the job. Resistor is to limit current that will flow from FTDI to FPGA. Spartan devices are known to react with latchup when 2.5V JTAG signal is treated with 3.3V logic output :)

jordens commented 6 years ago

Still unclear what you are suggesting. Desoldering the level converter? Cutting, scratching the traces on both sides and bridging over the converter?

jordens commented 6 years ago

I desoldered the level converter and added a short on TDO and 1k on TMS. JTAG over FTDI also works with 25 MHz. That seems to have saved nearly 500 mW of contention current. 3.5 W now.

jordens commented 6 years ago

No. That was the SFP transceiver. 4 W with, 3.5 W without.

gkasprow commented 6 years ago

This is exactly what I meant :)

jordens commented 6 years ago

On my board (number 2, thanks for putting the sticker on there) port 5 of IC14 has SDA (EEM1:SDA) shorted to ground.

marmeladapk commented 6 years ago

On board 3 EEM1 SDA is pulled to 3V3 and I can see signal. Could you check if R157 is 2k2 and is connected to 3V3?

img_20180108_144411

@jordens Sorry for the delay, do you still need my bitbang I2C?

jordens commented 6 years ago

The 2k2 are there but they don't matter. The resistance to ground is 300 Ohm (310 in one polarity and 270 in the other). No EEM connected. Either the chip is broken or there is some conducting dust ;).

marmeladapk commented 6 years ago

@jordens Could you share your openocd setup?

jordens commented 6 years ago

kasli.cfg: 

interface ftdi
ftdi_device_desc "Quad RS232-HS"
ftdi_vid_pid 0x0403 0x6011
ftdi_channel 0
ftdi_layout_init 0x0008 0x000b
ftdi_location 1:8

reset_config none
transport select jtag
adapter_khz 25000

source [find cpld/xilinx-xc7.cfg]
source [find cpld/jtagspi.cfg]

The spi proxy bitstreams are either in our openocd conda packages or at https://github.com/jordens/bscan_spi_bitstreams

And if you want to play I2C, I pushed my toy code to https://github.com/quartiq/kasli-i2c

jordens commented 6 years ago

As @sbourdeauducq reported elsewhere (https://github.com/m-labs/artiq/issues/789), Kasli Ethernet is working. On the board here ("#2") SFP1 also works fine.

marmeladapk commented 6 years ago

@jordens To build Kasli target I pulled latest Artiq repo and I wanted to update conda environment to it using conda env update -f conda/artiq-dev.yaml. However this didn't pull latest migen and misoc with changes for Kasli, so I had to do this manually to be able to build gateware. Is this expected behaviour?

Also, build failed with errors: Rule violation (IOSTDTYPE-1) IOStandard Type - I/O port eem0_n is Single-Ended but has an IOStandard of LVDS_25 which can only support Differential. I used Vivado 2016.4. Is this known or is there some mistake on my side?

jordens commented 6 years ago

@marmeladapk I didn't know about conda update but -f is force and I don't think you can use it in this case. Just install the latest with conda install -c m-labs/label/dev misoc migen openocd. I don't know about the rule violation. I don't see those. We use 2017.2 or 2017.4. The buildbot will generate binary packages for Kasli soon.

marmeladapk commented 6 years ago

I'll check with 2017.4, but generally I got those errors when in constraints eem lines were set as LVDS, but they didn't have IOBUFDS instantiated in code.

jordens commented 6 years ago

@marmeladapk Make sure that the misoc/migen that are being used are actually the ones installed through conda and not some packages that you git clone-ed yourself and setup.py install or pip install-ed. But I can see unused inputs causing that warning. Pretty sure it's benign.

marmeladapk commented 6 years ago

Huh on board 4 and 6 SFP1 also works. Seems like I didn't do the test properly.

I tested last link using @gkasprow SATA-SFP converter. Below is a screenshot of all links working simultaneously at 6,25 Gbps.

wszystkie-6g
jordens commented 6 years ago

Si5324 clock distribution noise and spurs running from on-board XO.

Looking at J1, noise density mostly limited by the SA. The kHz should be Hz.

The one spur at ~449 kHz seems to be the SMPS. It wanders if I touch the SMPS components.

image image

Edit: proper measurement, instrument limit around -132 dBc/Hz, all spurs except the 450 kHz and multiples are artefacts. This is differential and one-sided (i.e. take -6 dB when comparing phase noise with datasheets). Matches datasheet to within 2-3dB.

image

image

hartytp commented 6 years ago

So, something like -100dBc/Hz at 100kHz offset? We expect something like -130dBc/Hz based on the datasheet. Wonder what limits that.

cf data sheet figure 6

jordens commented 6 years ago

The SA is the limit on noise density everywhere but not on spurs. I corrected the post above.

gkasprow commented 6 years ago

The Silabs supply was not designed to be very low noise. We can add LDO.

jordens commented 6 years ago
jordens commented 6 years ago

But let's get v1.1 into the fab first. We can't wait. Or is it already there? @gkasprow

gkasprow commented 6 years ago

@jordens do you need these FPGA pins for something else? Or just test point? The SFP+ modules have I2C, while some SFP ones use these pins for bandwidth control.

gkasprow commented 6 years ago

I've just prepared new revisions for: Urukul, BNC, SMA, TTL tester, Sampler. Now I'm working on Zotino, @marmeladapk will update Kasli ASAP.

hartytp commented 6 years ago

The Silabs supply was not designed to be very low noise. We can add LDO.

If we can do this without adding too much additional complexity then I think it would be a good idea. Would it need an extra SMPS on Kasli though to produce the voltage that drives the LDO (or, can we use a 2V5 LDO driven from 3V3 for the Si chip?).

I'd do that but also remove the clock fanout in favor of a single MMCX output form the Si5324 and then using Clocker or/and the backplane. Kasli is too dense and overloaded. The additional space would help with figuring out a good heatsinking solution, better SI, cleaner routing, higher yield, and lower manufacturing risk.

If it really does help then I'd be okay with this (although, it's definitely not something I'd like to see happen). However, before we do it I'd like @gkasprow to confirm that removing that clock buffer will really make a significant (at least noticeable) difference to any of the things you've said. e.g. I'm actually not convinced that removing that buffer will improve Kasli's SI or yield. Also, don't we still need a clock buffer to drive the BP clock lines as well as that MMCX?

jordens commented 6 years ago

@gkasprow AFAICT all modules use the rate select pins (RS0 and RS1) for bandwidth control. The MSA mandates I2C on MOD_DEF1/2.