search

butterstick-fpga / butterstick-hardware

Basic ECP5 based GigE to SYZYGY interface.
Other
195 stars 19 forks source link

Hardware r0.3: Features/Suggestions #5

Closed gregdavill closed 3 years ago

gregdavill commented 4 years ago

I'm planing on refreshing this design based on experience with the current version.

These are the main features I'd personally like to cover:

I'll entertain crowd-sourcing ideas because there are likely things I've not thought of. But understand that I'm clearly not able to include 100% of the ideas proposed. I'd still like to make this a general purpose development board, that can appeal to reasonably wide audience.

As with the previous designs the KiCad source files will be open-source so that anyone can study/remix this design if it doesn't quite fit their needs.

So, what else would you like to see on this board? :)

tommythorn commented 4 years ago

You had me at DDR3L, especially if this would enable two 8Gib parts. I presume that ~HDMI~ GPDI could be done over Transceiver SYZYGY? Otherwise it would be useful to have a plan for digital video.

gregdavill commented 4 years ago

Yes. Infact you could output over GPIO pins on any of the SYZYGY ports, upto 1080p30. I'd like to experiment with the GPDI output at higher bitrates using the serdes, I think that should work, maybe even get to 4K? ;)

I'm not sure that I'd have enough extra pins/space to include a video out port on the board itself.

oskirby commented 4 years ago

I think there's a lot of cool stuff you could do with the USB-C connector. It has high-speed lines that can be configured either as USB-SS, but also for HDMI or DisplayPort if you needed a video output port, though they will need a SERDES or two to use effectively.

USBC and power delivery also work well for your input power.

ewenmcneill commented 4 years ago

Of possible relevance, there's been interest from, eg, some Debian developers in having a FPGA board with open source tools (eg, ECP5) which can run a soft CPU and have a decent amount of RAM connected to it to (a) be able to run Linux on the soft CPU, and (b) self host building a substantial portion of the (eg, Debian) Linux distribution (One rapidly needs to be able to build Java, Python, Perl, Haskell, etc, to be able to build some of the fairly core libraries, like embedded databases, with all their language integrations, so the "base" RAM requirements for building a Linux distro are non-trivial.) The idea is to have something that is open/inspectable from the bitstream up with no "closed" blobs.

The TrellisBoard 1GB (8 Gigabits) was heading in the right direction for them, but still too tight for building some of the larger language toolchains. Having a way to get to 4x-8x that (so 4-8GB, ie 32-64 Gigabits), might make it quite useful to them, but even 2x (so 2GB, ie 16 Gigabits) could help. That's possibly more than you'd want to/could put on an 80x49mm board. But maybe there's a way to allow adding more via expansion ports? (Ie, like the HyperRAM PMODs, but for larger RAM modules.)

Ewen

gregdavill commented 4 years ago

@oskirby Thanks for the suggestions, the ECP5 only provides 4 serdes lanes in the package I'm looking at. I'd want all of these to go to the SZYZGY high-speed connector. Adding high-speed muxes is an option, but adds complications + costs. I do want to try experimenting with USBC USB3 / Alt modes on this. So I'll be creating some SZYZGY pods that connect the serdes parts up to a USBC connector.

@ewenmcneill Interesting thanks for this insight. Like the OrangeCrab the plan is to enable the hardware to support upto 8Gbit DDR3L chips (Although they're not great on the dollars/bits curve, compared with 1/2/4Gbit parts). Due to the I/O constraints imposed by the ECP5 on DDR3 interfaces and each port only having 32 I/O pins, it may be tricky to add extra memory via a SZYZGY port. An interesting idea might be to pair up 2 boards, and have an external bridge between them that can pull the extra memory from the second board into the address space of the first board with a linux capable soft processor.

ewenmcneill commented 4 years ago

An interesting idea might be to pair up 2 boards, and have an external bridge between them that can pull the extra memory from the second board into the address space of the first board with a linux capable soft processor.

@gregdavill That might be a really good way to approach it, especially if more than 2 boards could be daisy chained somehow and maybe multiple soft CPUs could be run on the set of boards. That'd effectively give you SMP (Symmetric Multi-Processing) with each board having local CPU cores and local RAM, but also having a bus method to be able to access "far RAM" that's connected to another board. (AMD used that kind of architecture for some of their CPU generations a few years ago IIRC, and the Linux kernel has built in support to try to optimise memory layout nearer the CPU core running the code to keep "far RAM" accesses less frequent.)

Ewen

mithro commented 4 years ago

Let me leave this here --> https://github.com/enjoy-digital/liteiclink

mithro commented 4 years ago

FYI -- @antmicro (like @kgugala) and @SpinalHDL are also working on bringing up SMP support in LiteX and VexRISCV.

mithro commented 4 years ago

What you are talking about is called NUMA in x86 -- https://en.wikipedia.org/wiki/Non-uniform_memory_access

ewenmcneill commented 4 years ago

What you are talking about is called NUMA in x86 [...] https://github.com/enjoy-digital/liteiclink

@mithro Thanks for the reminder of the NUMA term; that what was I was trying to think of when describing a "two board" SMP (which actually wouldn't be symmetric if they have distinct local RAM). And yes, Enjoy Digital's link-via-PCIe-connector idea, that you link to, was the sort of thing I had in mind for a board interconnect.

@gregdavill, possibly relevant to your design would be how easily a ECP5 5G part could be an option to place on the PCB, with the 5G enhanced pins facing, eg, one of the SYZYGY interfaces. (I'm not sure off the top of my head how pin compatible the various ECP5 variants are, but it's at least worth considering if it's possible to arrange for the 5G enhanced pins to be used for a higher speed board to board interconnect for multi-board NUMA as a build option.)

Ewen

gregdavill commented 4 years ago

A design goal will be to support the 5G parts.

One of the SYZYGY ports is a TXR-4 Transceiver type, which can support 4 TX/RX lanes. This is exactly how many lanes we have on the 381 package ECP5, and should enable some fun experiments. (SDI, display port, high res HDMI?, PCIe, SGMII, etc)

mithro commented 4 years ago

@gregdavill - Would you be able to do a SYZYGY to TOFE adapter board? It should be also usable for SYZYGY to PCIe...

thejpster commented 3 years ago

The 8P8C Ethernet Jack looks like it would get in the way of putting this module down onto a larger board (as you might fit a Raspberry Pi CM4 on to the Raspberry Pi IO board). Is it possible to get the Ethernet jack on the other side, or as a no-fit?

gregdavill commented 3 years ago

@thejpster moving the Ethernet connector is not an option at this stage as the design work for this version has been completed now.

The board as designed is more configured to operate like a RPI+Hat rather than a CM4+base plate.