Metlino front panel

[gkasprow]

The Metlino schematics are done in 90%, to finish I need more info about connectivity. On the front panel I can fit one of many possible connector configurations:

1. 2 x VHDCI + 3x SFP
2. 2x VHDCI + 2x QSFP + 1x SMA
3. 2x VHDCI + 2x SFP + 2x SMA
4. 1x VHDCI + 4x SFP + 3 x SMA
5. 3x VHDCI + 1 x QSFP + 3x SMA
6. 2x VHDCI + 1 x QSFP + 6x SMA
7. 1x VHDCI + 2x QSFP +1x SFP + 2x SMA

• micro USB, LEDs and some MMCX connectors will fit easily in each configuration

VHDCI is used for IO box connetivity. 2 VHDCI let us connect 2 IO boxes to Metlino. Each VHDCI has 32 differential LVDS pairs (1Gbit capable) + 3V supply and I2C. Standard cables ranging from 30cm to 10m are easily available (SCSI). SFP+ cages are compatible with White Rabbit WDM modules QSFP are dense, 4 channel 10Gbit modules. QSFP+ to 4xSFP+ copper cables and optical assemblies are available easily.

Another thing is FMC modue. There is no space on MCH front panel so the FMC bezel can be faced RTM side. This is one possible configuration with 4 SFPs, 1 VHDCI, 3 SMAs and example FMC module from the RTM side.

[jmizrahi]

I would vote for at least 2x VHDCI

[sbourdeauducq]

Note one advantage of VHDCI TTLs: they can be very low latency (especially when combined with to-be-developed custom gateware) as they don't have to cross any transceivers.

[jordens]

Shouldn't there be the connection to the rtm? I don't think an fmc is needed or particularly useful there. QSFP alone is tricky because of lane alignment. QSFP pigtails would worry me because I don't see how they handle the other signals per SFP.

If go for configuration 4 or 1.

[gkasprow]

with 2 VHDCI we may still fit SFP and QSFP or 2x SFP or or

[gkasprow]

The FMC connector can be used in two ways.

1. We can plug standard FMC board with bezel, i.e. for additional computing resources
2. Or we can plug FMC board with RTM connector on it. The SEARAY connectors can have stacking height up to 17mm so we will easily fit. I have some doubts about passing 10Gbit signalling via 2 connectors - signal regeneration would be needed. 5Gbit should work for Metlino RTM SFPs.

[gkasprow]

Yes, SFP cages can be installed on FMC as well. But rear access could be a little difficult.

But if we install RTM connector on FMC, then we can use existing RTM SFP module or modify it to gain access to the uRF backplane

[gkasprow]

QSFP are configured only by I2C. There are no per-lane signals.

The only control signals are module select, module reset, low power mode

Look at the datasheet here. There is per lane configuration by registers.

ftp://ftp.seagate.com/pub/sff/SFF-8436.PDF ftp://ftp.seagate.com/pub/sff/SFF-8436.PDF

[jordens]

What I meant is that where you have the FMC I would have expected the zone 3 connector between the RTM and the MCH. We need a way to get the 100 MHz from the RF backplane to the Metlino. Or do you want to use the FMC to bridge the gap to the RTM?

I would ignore QSFP and go for SFPs only. As many SFP as possible on the MCH panel and then fitting plenty on the MCH-RTM as well.

[gkasprow]

Yes, I want to use FMC to serve 3 purposes:

• bridge the gap between Metlino and RTM
• use FMC-SFP for stand-alone applications without RF backplane
• optional IO or computing module

If we ignore QSFP then with 2x VHDCI we can fit 3 SFPs on a front panel or 2 x SFPs + 2 x SMA.

look at drawing below with FMC-RTM use case

[jordens]

Looks good to me if that works out mechanically.

[gkasprow]

Yes, it fits. The remaining issue is front panel - do we need SMAs there for clock input for stand-alone application? Is it important to have 3 SFPs rather than 2 ?

[jordens]

One SMA for RTIO clock injection without an RF backplane is useful. My guess is that a consensus could be 2xVHDCI, 1xSMA and as many SFP as possible. If it's just two SFP that's fine assuming that there are additional SFPs on the RTM panel.

[gkasprow]

This SMA should go to Si chip? It has 2 inputs so one can be connected to SMA, another to RTM

[gkasprow]

OK, I managed to fit 3 SFP+ cages, 2 VHDCI connectors and 2 SMAs.

[sbourdeauducq]

Why are there two Ethernet connectors?

[gkasprow]

They sit on MCH part from NAT, one is Ethernet, second is console

[sbourdeauducq]

Do we really need that MCH part from NAT?

[gkasprow]

yes

On 14 November 2016 at 09:52, Sébastien Bourdeauducq < notifications@github.com> wrote:

Do we really need that MCH part from NAT?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/m-labs/sinara/issues/73#issuecomment-260280000, or mute the thread https://github.com/notifications/unsubscribe-auth/AEH-vh96dUpgAhWC6uW26GrO2ueFra37ks5q-CE1gaJpZM4KwiGJ .

[sbourdeauducq]

What is it for exactly? Can we build the Metlino in a way that makes it optional?

[jmizrahi]

@jordens Is communication to Kasli boxes the purpose you have in mind for the SFP connectors? Or is there some other role they play?

General question -- if, in the future, we decide we want more general purpose I/Os out of Metlino (as SFP or VHDCI or anything else) couldn't we build a simple routing card that would go into an empty slot in the chassis, and route that slot's control signals to the front panel?

[gkasprow]

We need Tongue 1 to deliver power supply to other tongues and to have simplest crate management. In the future we can design simple mechanical adapter that would do the same job. But so far we have enough work to do and will relay on existing and working MCH.

On 14 November 2016 at 13:13, Sébastien Bourdeauducq < notifications@github.com> wrote:

What is it for exactly? Can we build the Metlino in a way that makes it optional?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/m-labs/sinara/issues/73#issuecomment-260320230, or mute the thread https://github.com/notifications/unsubscribe-auth/AEH-vtZo0k2M_8qFXyfwhG6pYTxOYCNsks5q-FB1gaJpZM4KwiGJ .

[gkasprow]

We already have 2 VHDCI connectors so there are no FPGA pins left. We also have FMC connector that can be routed using simple adapter to front panel as another set of SFPs or VHDCi connector

—

[jmizrahi]

Aren't there dedicated FPGA pins routed to each slot in the chassis?

[jordens]

@jmizrahi SFPs to other crates (i.e. other Metlinos), other Kaslis, and in general other devices. See the whitepaper. Generically, there would be one uplink and the rest as downlinks. For more TTLs we will have the HPC FMC that's on each Sayma. Currently, we do not plan to use any signals between the Saymas and the Metlino other than the DRTIO link.

There needs to be significant consolidation and maybe a bit of runtime discovery/autoconfiguration of the possible configurations for Sayma and Metlino. We can't herd dozens of bitstreams, let alone document or test them. And we can't offload that task to the user either.

[gkasprow]

At the moment FPGA pins are connected to: 64 pairs, 3 FPGA banks - 2x VHDCI 3 FPGA banks - DDR3 x 64 2 FPGA banks, 34 pairs - FMC and I have one HR bank and half HP banks left to connect to the other slots. So I can fully connect one additional fabric (x12 Rx, Tx)

[jordens]

@gkasprow with "additional fabric" you mean something on the AMC backplane? Go for it. Given that the front panel is full and the RTM connector otherwise occupied with the SFP transciever links (I assume) I can't see a better use right now.

[jmizrahi]

@gkasprow what is the purpose of the -CLK-PHYS option you specified on the MCH? If as you say it's just doing crate management and power distribution, I don't see why we need that option.

[gkasprow]

We need it to have a place where Metlino will be installed

Metlino is connected to CLK PHYS board and this clock option is the cheapest available.

It also can deliver 100MHz reference to all AMCs and Metlino. Some time we discussed that this board will be used for clock synchronization, then we changed conception because we switched to RTM and uRFB

So the idea was that CLK PHYS gets clock from SMA input on MCH panel and then passes it via low jitter clock mux to all AMCs.

In parallel it generates low jitter 100MHz fabric clock ( FCLK) to all AMCs and Metlino.

We can get rid of it but otherwise still another PCB would need to be designed that has Tongue 2 connector and passes supply to Metlino and also forms MCH connector stack.

Some day we will design our own, dedicated MCH Tongue 1 with more SFPs and VHDCI connectors and simple pcb that holds Tongue 2 connector and supply for Metlino.

Now we need it to limit number of PCB that need to be developed.

[sbourdeauducq]

The AMCs do not need 100MHz clock distribution, they will recover it from the data links. Only the root Metlino needs the 100MHz to clock those data links - and I thought it was going to receive it via RTM? And generally only the RTM side has clocks connected, we do not need any clock inputs on AMC side.

[gkasprow]

Yes, Metlino will get it via RTM or from front panel SMA for stand-alone applications. For the moment we won't design RTM for Metlino since we have more important things to do. In addition we can provide an adapter that lets you connect RTM backplane with Metlino FMC connector just to check clock distribution in the crate. https://github.com/notifications/unsubscribe-auth/AEH-vsXTpjFGsI1cMj-Ofr1rvkO6yfuvks5q-6pTgaJpZM4KwiGJ

[gkasprow]

Can we close this issue?

[jordens]

@gkasprow just ping whoever needs/should sign off on this: @jmizrahi @jbqubit ping.

[jmizrahi]

@gkasprow so the final configuration is as shown here (namely 3 SFP+ cages, 2 VHDCI connectors and 2 SMAs) ? I am happy with that.

[gkasprow]

Yes, + FMC connector for RTM extension.

[jbqubit]

Yes, sounds good. :)