open-power / snap

CAPI SNAP Framework Hardware and Software
Apache License 2.0
108 stars 61 forks source link

Enable the 2nd DRAM controller for ADKU3 and S121B cards #619

Open jsvogt opened 6 years ago

jsvogt commented 6 years ago

These cards have two separate DRAM channels. Therefore they need two MIG cores to drive the I/Os, otherwise the action can only access 1/2 of the available memory. In the past this effort had been prevented by the PSL DCP blocking DRAM I/Os. There are two options to support DRAM on these cards (TBD):

a) Combine the two AXI buses with AXI interconnect. => No change to action wrapper, as the action still just sees one AXI with 2x the memory size => May be able to clock the merged AXI faster to get the max. bandwidth from the DRAM controllers. b) Connect both MIG AXI buses to the action directly => ~2x bandwidth possible, the two DRAM controllers work independently. => Requires changing the action wrapper and all example actions!

bmesnet commented 6 years ago

Is it feasible to let the user select one of these 2 options depending on his needs : more space or more bandwidth?

luyong6 commented 6 years ago

My opinion: To make a more flexible DDR mapping, allow the user to select: 1) Use which DDR port. (KU3/S121B/8K5 has two DDR ports. Next generation CAPI2.0 cards may have up to 4 DDR ports.) We need to allow the user to select. 2) Use how many AXI interfaces for more than 1 DDR ports. Some suggested variables are:

Someone is going to be mad. We need to set down the naming rules, for example, not allow S121B to use the name of "c1, c2" while KU3 uses the name of "b0, b1", we regulate the names all start from 0. Then allow the script to manipulate the code in snap/hdl/core....

Implement the basic DDR mapping and leave the complex combination to the user who has a real requirement.

jsvogt commented 6 years ago

I don't think we need all combinations (driving someone mad). IMHO the most important option is a) : Provide the full DRAM size, more bandwidth as well. Let the AXI interconnect handle the AXI routing to one of the (2-4) MIGs. Option b) doesn't really give more benefits, just a little more control. Testing all "little complex" combinations will be a nightmare without much added customer value.

=> Support 1 DRAM (no AXI interconnect needed), or all DRAM MIGs (AXI interconnect needed) Users who really need fewer MIGs can then manually remove unused MIGs from the block design, or propose changes to automate this.

I agree with the naming. All names should start with 0.

jsvogt commented 6 years ago

So far, no card implements two DRAM MIGs (with two separate DRAM interfaces in hardware/hdl/core/psl_fpga_<cardname>.vhd_source