direct mw size issue ?

[dusane]

I am trying to increase size of direct mw over bar2/3 (64 bit). (to 32M). I see that switchtec_ntb_mw_direct_get_range(), limits the size to offset or max_mw_size. If i have 32 LUTs this size is limited to 2MB (64K * 32), I dont understand why is direct mw size dependent on #LUTs ? (I understand #LUTs, will change the base offset of the direct mw). If I make LUT=0 (in config file, to use direct m wonly configs), the vnic ntb ping fails. I also see that, with LUT=0, there is no link up event generated for one of the b2b hosts(which loads & configures) the ntb_hw driver later).

[lsgunth]

The translation of the memory windows in hardware is done simply by replacing the top N bits of the address. So the offset of the window must be aligned to at least the size of the window. So if the base offset is 1MB then the maximum size is 1MB.

The ntb_hw driver requires at least one LUT to function correctly.

You can either increase the size/number of LUTs to increase the direct window offset. Or you can enable BAR4 with no LUTs so it can use any size.

[dusane]

@lsgunth Thanks for the explanation. I tried increasing lut sz, but it affects spad offsets somehow causing ntb_transport to fail.

[lsgunth]

That doesn't sound right... how did you increase the lut size?

[dusane]

@lsgunth I was able to increase the bar23 mw size by increasing lut sz. (LUT_SIZE param.). I tried to disable luts altogether but that didn’t help. If I set num of luts to 0 in config file, what else do I need to do ?

[lsgunth]

Increasing the LUT_SIZE parameter should work as far as I know. Pretty sure other people have done that. It should not affect spad offsets in any way.

Again, you can not have zero luts... The ntb_hw driver requires at least one LUT.

[sjrom]

Hi, @lsgunth: We increased the LUT_SIZE from 64K to 256K. However, we found the init and mapping of NTB MW was incorrect after offset 2M. (LUT# 32) Ex. Value at offset 0x1FFFF8 (0x7f37497baff8): 0x00000000 (Correctly initialized.) Value at offset 0x1FFFFC (0x7f37497baffc): 0x00000000 Value at offset 0x200000 (0x7f37497bb000): 0xFFFFFFFF (Not initialized.) Value at offset 0x200004 (0x7f37497bb004): 0xFFFFFFFF

Base on the latest github code. Did any tips we lost to enlarge the shared memory window?

Any guide or suggestion was appreciated.