Z97 board can't allocate above 4G even with above 4g decoding enable in BIOS

val3nt33n commented 1 year ago

System

Motherboard: Asus Z97i-Plus
BIOS Version: 2704
GPU: Intel Arc A750 LE (supports up to 8GB BAR)
Above 4G decoding is enabled in BIOS
CSM is disabled
Applied PciBus | Remove <16GB BAR size limit patch
replaced NvramSmi module from an older BIOS(version 2103) to allow new uefi variable creation

Description I modded the BIOS to enable rebar using your instructions but the maximum working BAR size is 1GB. At 2GB the system gets stuck during post, at 4GB and 8GB it boots into the OS(Windows 10 and Ubuntu 20.04) but video output works only from the iGPU.

This is dmesg after booting with 8GB

[    0.194347] pci_bus 0000:00: root bus resource [bus 00-3e]
[    0.194349] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.194350] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.194352] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff window]
[    0.194353] pci_bus 0000:00: root bus resource [mem 0xdf200000-0xfeafffff window]
[    0.194354] pci_bus 0000:00: root bus resource [mem 0x800000000-0xfbfffffff window]
[    0.197052] pci 0000:00:01.0: PCI bridge to [bus 01-04]
[    0.197055] pci 0000:00:01.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.197058] pci 0000:00:01.0:   bridge window [mem 0xf0000000-0xf07fffff 64bit pref]
[    0.197408] pci 0000:01:00.0: PCI bridge to [bus 02-04]
[    0.197413] pci 0000:01:00.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.197453] pci 0000:03:00.0: [8086:56a1] type 00 class 0x030000
[    0.197468] pci 0000:03:00.0: reg 0x10: [mem 0x00000000-0x00ffffff 64bit]
[    0.197477] pci 0000:03:00.0: reg 0x18: [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.197491] pci 0000:03:00.0: reg 0x30: [mem 0x00000000-0x001fffff pref]
[    0.197563] pci 0000:03:00.0: PME# supported from D0 D3hot
[    0.209169] pnp 00:00: disabling [mem 0xfed40000-0xfed44fff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209785] pnp 00:06: disabling [mem 0xfed1c000-0xfed1ffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209788] pnp 00:06: disabling [mem 0xfed10000-0xfed17fff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209790] pnp 00:06: disabling [mem 0xfed18000-0xfed18fff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209791] pnp 00:06: disabling [mem 0xfed19000-0xfed19fff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209793] pnp 00:06: disabling [mem 0xf8000000-0xfbffffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209794] pnp 00:06: disabling [mem 0xfed20000-0xfed3ffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209796] pnp 00:06: disabling [mem 0xfed90000-0xfed93fff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209797] pnp 00:06: disabling [mem 0xfed45000-0xfed8ffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209798] pnp 00:06: disabling [mem 0xff000000-0xffffffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209800] pnp 00:06: disabling [mem 0xfee00000-0xfeefffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209801] pnp 00:06: disabling [mem 0xf7fe0000-0xf7feffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.209803] pnp 00:06: disabling [mem 0xf7ff0000-0xf7ffffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.217718] pci 0000:01:00.0: BAR 15: no space for [mem size 0x200000000 64bit pref]
[    0.217720] pci 0000:01:00.0: BAR 15: failed to assign [mem size 0x200000000 64bit pref]
[    0.217723] pci 0000:02:01.0: BAR 15: no space for [mem size 0x200000000 64bit pref]
[    0.217724] pci 0000:02:01.0: BAR 15: failed to assign [mem size 0x200000000 64bit pref]
[    0.217725] pci 0000:02:01.0: BAR 14: no space for [mem size 0x01800000]
[    0.217726] pci 0000:02:01.0: BAR 14: failed to assign [mem size 0x01800000]
[    0.217727] pci 0000:03:00.0: BAR 2: no space for [mem size 0x200000000 64bit pref]
[    0.217729] pci 0000:03:00.0: BAR 2: failed to assign [mem size 0x200000000 64bit pref]
[    0.217730] pci 0000:03:00.0: BAR 0: no space for [mem size 0x01000000 64bit]
[    0.217731] pci 0000:03:00.0: BAR 0: failed to assign [mem size 0x01000000 64bit]
[    0.217732] pci 0000:03:00.0: BAR 6: no space for [mem size 0x00200000 pref]
[    0.217733] pci 0000:03:00.0: BAR 6: failed to assign [mem size 0x00200000 pref]
[    0.217734] pci 0000:02:01.0: PCI bridge to [bus 03]
[    0.217744] pci 0000:02:04.0: PCI bridge to [bus 04]
[    0.217748] pci 0000:02:04.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.217756] pci 0000:01:00.0: PCI bridge to [bus 02-04]
[    0.217759] pci 0000:01:00.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.217764] pci 0000:00:01.0: PCI bridge to [bus 01-04]
[    0.217766] pci 0000:00:01.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.217768] pci 0000:00:01.0:   bridge window [mem 0xf0000000-0xf07fffff 64bit pref]
[    0.217771] pci 0000:00:1c.0: PCI bridge to [bus 05]
[    0.217779] pci_bus 0000:00: Some PCI device resources are unassigned, try booting with pci=realloc
[    0.217780] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    0.217782] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    0.217783] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff window]
[    0.217784] pci_bus 0000:00: resource 7 [mem 0xdf200000-0xfeafffff window]
[    0.217785] pci_bus 0000:00: resource 8 [mem 0x800000000-0xfbfffffff window]
[    0.217786] pci_bus 0000:01: resource 1 [mem 0xf7c00000-0xf7cfffff]
[    0.217787] pci_bus 0000:01: resource 2 [mem 0xf0000000-0xf07fffff 64bit pref]
[    0.217788] pci_bus 0000:02: resource 1 [mem 0xf7c00000-0xf7cfffff]
[    0.217789] pci_bus 0000:04: resource 1 [mem 0xf7c00000-0xf7cfffff]

00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
03:00.0 VGA compatible controller: Intel Corporation Device 56a1 (rev 08)
06:00.0 Network controller: Broadcom Inc. and subsidiaries BCM4352 802.11ac Wireless Network Adapter (rev 03)

and /proc/iomem

f0000000-f07fffff : PCI Bus 0000:01
  f0000000-f07fffff : 0000:01:00.0
f7400000-f77fffff : 0000:00:02.0
f7800000-f7afffff : PCI Bus 0000:06
  f7800000-f79fffff : 0000:06:00.0
  f7a00000-f7a07fff : 0000:06:00.0
f7b00000-f7bfffff : PCI Bus 0000:01
  f7b00000-f7bfffff : PCI Bus 0000:02
    f7b00000-f7bfffff : PCI Bus 0000:04
      f7b00000-f7b03fff : 0000:04:00.0
...
100000000-41fdfffff : System RAM
  179e00000-17ae025c7 : Kernel code
  17b000000-17ba87fff : Kernel rodata
  17bc00000-17c0455ff : Kernel data
  17c393000-17c9fffff : Kernel bss
41fe00000-41fffffff : RAM buffer

device manger

val3nt33n commented 1 year ago

I tried looking at the memory range in DSDT but it's using some external symbols that I couldn't find anywhere in the other ACPI tables.

        CreateQWordField (BUF0, \_SB.PCI0._Y1E._LEN, M2LN)  // _LEN: Length          
        CreateQWordField (BUF0, \_SB.PCI0._Y1E._MIN, M2MN)  // _MIN: Minimum Base Address         
        CreateQWordField (BUF0, \_SB.PCI0._Y1E._MAX, M2MX)  // _MAX: Maximum Base Address             
        M2LN = M64L /* External reference */
        M2MN = M64B /* External reference */                                         
        M2MX = ((M2MN + M2LN) - One)

val3nt33n commented 1 year ago

After setting pci=realloc at boot it looks like it manages to assign memory to the GPU. I'm still not getting video output, though.

[    0.213321] pnp 00:00: disabling [mem 0xfed40000-0xfed44fff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213934] pnp 00:06: disabling [mem 0xfed1c000-0xfed1ffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213937] pnp 00:06: disabling [mem 0xfed10000-0xfed17fff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213939] pnp 00:06: disabling [mem 0xfed18000-0xfed18fff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213940] pnp 00:06: disabling [mem 0xfed19000-0xfed19fff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213941] pnp 00:06: disabling [mem 0xf8000000-0xfbffffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213943] pnp 00:06: disabling [mem 0xfed20000-0xfed3ffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213944] pnp 00:06: disabling [mem 0xfed90000-0xfed93fff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213945] pnp 00:06: disabling [mem 0xfed45000-0xfed8ffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213947] pnp 00:06: disabling [mem 0xff000000-0xffffffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213948] pnp 00:06: disabling [mem 0xfee00000-0xfeefffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213950] pnp 00:06: disabling [mem 0xf7fe0000-0xf7feffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.213951] pnp 00:06: disabling [mem 0xf7ff0000-0xf7ffffff] because it overlaps 0000:03:00.0 BAR 2 [mem 0x00000000-0x1ffffffff 64bit pref]
[    0.221845] pci_bus 0000:00: max bus depth: 3 pci_try_num: 4
[    0.221854] pci 0000:01:00.0: BAR 15: no space for [mem size 0x200000000 64bit pref]
[    0.221856] pci 0000:01:00.0: BAR 15: failed to assign [mem size 0x200000000 64bit pref]
[    0.221858] pci 0000:02:01.0: BAR 15: no space for [mem size 0x200000000 64bit pref]
[    0.221859] pci 0000:02:01.0: BAR 15: failed to assign [mem size 0x200000000 64bit pref]
[    0.221860] pci 0000:02:01.0: BAR 14: no space for [mem size 0x01800000]
[    0.221861] pci 0000:02:01.0: BAR 14: failed to assign [mem size 0x01800000]
[    0.221863] pci 0000:03:00.0: BAR 2: no space for [mem size 0x200000000 64bit pref]
[    0.221864] pci 0000:03:00.0: BAR 2: failed to assign [mem size 0x200000000 64bit pref]
[    0.221865] pci 0000:03:00.0: BAR 0: no space for [mem size 0x01000000 64bit]
[    0.221866] pci 0000:03:00.0: BAR 0: failed to assign [mem size 0x01000000 64bit]
[    0.221868] pci 0000:03:00.0: BAR 6: no space for [mem size 0x00200000 pref]
[    0.221869] pci 0000:03:00.0: BAR 6: failed to assign [mem size 0x00200000 pref]
[    0.221870] pci 0000:02:01.0: PCI bridge to [bus 03]
[    0.221879] pci 0000:02:04.0: PCI bridge to [bus 04]
[    0.221884] pci 0000:02:04.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.221892] pci 0000:01:00.0: PCI bridge to [bus 02-04]
[    0.221895] pci 0000:01:00.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.221900] pci 0000:00:01.0: PCI bridge to [bus 01-04]
[    0.221902] pci 0000:00:01.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.221903] pci 0000:00:01.0:   bridge window [mem 0xf0000000-0xf07fffff 64bit pref]
[    0.221907] pci 0000:00:1c.0: PCI bridge to [bus 05]
[    0.221914] pci_bus 0000:00: No. 2 try to assign unassigned res
[    0.221918] pci 0000:01:00.0: BAR 15: no space for [mem size 0x200000000 64bit pref]
[    0.221919] pci 0000:01:00.0: BAR 15: failed to assign [mem size 0x200000000 64bit pref]
[    0.221921] pci 0000:02:01.0: BAR 15: no space for [mem size 0x200000000 64bit pref]
[    0.221922] pci 0000:02:01.0: BAR 15: failed to assign [mem size 0x200000000 64bit pref]
[    0.221923] pci 0000:02:01.0: BAR 14: no space for [mem size 0x02000000]
[    0.221924] pci 0000:02:01.0: BAR 14: failed to assign [mem size 0x02000000]
[    0.221926] pci 0000:03:00.0: BAR 2: no space for [mem size 0x200000000 64bit pref]
[    0.221927] pci 0000:03:00.0: BAR 2: failed to assign [mem size 0x200000000 64bit pref]
[    0.221928] pci 0000:03:00.0: BAR 0: no space for [mem size 0x01000000 64bit]
[    0.221929] pci 0000:03:00.0: BAR 0: failed to assign [mem size 0x01000000 64bit]
[    0.221930] pci 0000:02:01.0: PCI bridge to [bus 03]
[    0.221939] pci 0000:02:04.0: PCI bridge to [bus 04]
[    0.221943] pci 0000:02:04.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.221951] pci 0000:01:00.0: PCI bridge to [bus 02-04]
[    0.221954] pci 0000:01:00.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.221959] pci 0000:00:01.0: PCI bridge to [bus 01-04]
[    0.221961] pci 0000:00:01.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.221962] pci 0000:00:01.0:   bridge window [mem 0xf0000000-0xf07fffff 64bit pref]
[    0.221965] pci 0000:00:1c.0: PCI bridge to [bus 05]
[    0.221973] pci_bus 0000:00: No. 3 try to assign unassigned res
[    0.221975] pci 0000:02:04.0: resource 14 [mem 0xf7c00000-0xf7cfffff] released
[    0.221976] pci 0000:02:04.0: PCI bridge to [bus 04]
[    0.221979] pci 0000:01:00.0: resource 14 [mem 0xf7c00000-0xf7cfffff] released
[    0.221980] pci 0000:01:00.0: PCI bridge to [bus 02-04]
[    0.221982] release child resource [mem 0xf0000000-0xf07fffff 64bit pref]
[    0.221983] pci 0000:00:01.0: resource 15 [mem 0xf0000000-0xf07fffff 64bit pref] released
[    0.221984] pci 0000:00:01.0: PCI bridge to [bus 01-04]
[    0.221991] pci 0000:00:01.0: BAR 15: assigned [mem 0x800000000-0xaffffffff 64bit pref]
[    0.221993] pci 0000:01:00.0: BAR 15: assigned [mem 0x800000000-0x9ffffffff 64bit pref]
[    0.221995] pci 0000:01:00.0: BAR 0: assigned [mem 0xa00000000-0xa007fffff 64bit pref]
[    0.222001] pci 0000:01:00.0: BAR 14: no space for [mem size 0x02800000]
[    0.222002] pci 0000:01:00.0: BAR 14: failed to assign [mem size 0x02800000]
[    0.222004] pci 0000:02:01.0: BAR 15: assigned [mem 0x800000000-0x9ffffffff 64bit pref]
[    0.222005] pci 0000:02:01.0: BAR 14: no space for [mem size 0x02000000]
[    0.222006] pci 0000:02:01.0: BAR 14: failed to assign [mem size 0x02000000]
[    0.222007] pci 0000:02:04.0: BAR 14: no space for [mem size 0x00100000]
[    0.222008] pci 0000:02:04.0: BAR 14: failed to assign [mem size 0x00100000]
[    0.222009] pci 0000:03:00.0: BAR 2: assigned [mem 0x800000000-0x9ffffffff 64bit pref]
[    0.222017] pci 0000:03:00.0: BAR 0: no space for [mem size 0x01000000 64bit]
[    0.222018] pci 0000:03:00.0: BAR 0: failed to assign [mem size 0x01000000 64bit]
[    0.222019] pci 0000:02:01.0: PCI bridge to [bus 03]
[    0.222024] pci 0000:02:01.0:   bridge window [mem 0x800000000-0x9ffffffff 64bit pref]
[    0.222029] pci 0000:04:00.0: BAR 0: no space for [mem size 0x00004000 64bit]
[    0.222030] pci 0000:04:00.0: BAR 0: failed to assign [mem size 0x00004000 64bit]
[    0.222031] pci 0000:02:04.0: PCI bridge to [bus 04]
[    0.222042] pci 0000:01:00.0: PCI bridge to [bus 02-04]
[    0.222047] pci 0000:01:00.0:   bridge window [mem 0x800000000-0x9ffffffff 64bit pref]
[    0.222050] pci 0000:00:01.0: PCI bridge to [bus 01-04]
[    0.222052] pci 0000:00:01.0:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.222054] pci 0000:00:01.0:   bridge window [mem 0x800000000-0xaffffffff 64bit pref]
[    0.222057] pci 0000:00:1c.0: PCI bridge to [bus 05]
[    0.222064] pci_bus 0000:00: No. 4 try to assign unassigned res
[    0.222065] pci 0000:00:01.0: resource 14 [mem 0xf7c00000-0xf7cfffff] released
[    0.222066] pci 0000:00:01.0: PCI bridge to [bus 01-04]
[    0.222070] pci 0000:00:01.0: BAR 14: assigned [mem 0xf0000000-0xf27fffff]
[    0.222072] pci 0000:01:00.0: BAR 14: assigned [mem 0xf0000000-0xf27fffff]
[    0.222074] pci 0000:02:01.0: BAR 14: assigned [mem 0xf0000000-0xf1ffffff]
[    0.222075] pci 0000:02:04.0: BAR 14: assigned [mem 0xf2000000-0xf20fffff]
[    0.222077] pci 0000:03:00.0: BAR 0: assigned [mem 0xf0000000-0xf0ffffff 64bit]
[    0.222084] pci 0000:02:01.0: PCI bridge to [bus 03]
[    0.222088] pci 0000:02:01.0:   bridge window [mem 0xf0000000-0xf1ffffff]
[    0.222090] pci 0000:02:01.0:   bridge window [mem 0x800000000-0x9ffffffff 64bit pref]
[    0.222095] pci 0000:04:00.0: BAR 0: assigned [mem 0xf2000000-0xf2003fff 64bit]
[    0.222107] pci 0000:02:04.0: PCI bridge to [bus 04]
[    0.222111] pci 0000:02:04.0:   bridge window [mem 0xf2000000-0xf20fffff]
[    0.222119] pci 0000:01:00.0: PCI bridge to [bus 02-04]
[    0.222122] pci 0000:01:00.0:   bridge window [mem 0xf0000000-0xf27fffff]
[    0.222125] pci 0000:01:00.0:   bridge window [mem 0x800000000-0x9ffffffff 64bit pref]
[    0.222128] pci 0000:00:01.0: PCI bridge to [bus 01-04]
[    0.222130] pci 0000:00:01.0:   bridge window [mem 0xf0000000-0xf27fffff]
[    0.222132] pci 0000:00:01.0:   bridge window [mem 0x800000000-0xaffffffff 64bit pref]
[    0.222134] pci 0000:00:1c.0: PCI bridge to [bus 05]
[    0.222142] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    0.222143] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    0.222144] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff window]
[    0.222146] pci_bus 0000:00: resource 7 [mem 0xdf200000-0xfeafffff window]
[    0.222147] pci_bus 0000:00: resource 8 [mem 0x800000000-0xfbfffffff window]
[    0.222148] pci_bus 0000:01: resource 1 [mem 0xf0000000-0xf27fffff]
[    0.222149] pci_bus 0000:01: resource 2 [mem 0x800000000-0xaffffffff 64bit pref]
[    0.222150] pci_bus 0000:02: resource 1 [mem 0xf0000000-0xf27fffff]
[    0.222151] pci_bus 0000:02: resource 2 [mem 0x800000000-0x9ffffffff 64bit pref]
[    0.222152] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf1ffffff]
[    0.222153] pci_bus 0000:03: resource 2 [mem 0x800000000-0x9ffffffff 64bit pref]
[    0.222154] pci_bus 0000:04: resource 1 [mem 0xf2000000-0xf20fffff]

03:00.0 VGA compatible controller: Intel Corporation Device 56a1 (rev 08) (prog-if 00 [VGA controller])
    Subsystem: Intel Corporation Device 1021
    Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Region 0: Memory at f0000000 (64-bit, non-prefetchable) [disabled] [size=16M]
    Region 2: Memory at 800000000 (64-bit, prefetchable) [disabled] [size=8G]
    Capabilities: [40] Vendor Specific Information: Len=0c <?>
    Capabilities: [70] Express (v2) Endpoint, MSI 00

val3nt33n commented 1 year ago

Browsing through AMIBCP I found some PCIe settings that don't show up in BIOS. Not sure what these do or how to make them visible. All settings(both visible and hidden) have selected 'Yes' in the Show column. I'm confused why it says 'Yes' when it's hidden. The Reserved Memory setting defaults to 10. I wonder if this is the 1GB limit.

AMIBCP

xCuri0 commented 1 year ago

@val3nt33n can you also show the Prefetchable Memory description. I think that might be it not Reserved Memory

val3nt33n commented 1 year ago

@xCuri0 it says "Prefetchable Memory Range for this Root Bridge" And Prefetchable Memory above 4G description is same as the name: "Prefetchable Memory above 4G"

xCuri0 commented 1 year ago

try changing both to 13 I guess. but i think Prefetchable Memory is more likely the important one

val3nt33n commented 1 year ago

I tried changing all 3 one by one. No difference. I think some of these settings are hidden because they are not implemented.

val3nt33n commented 1 year ago

Oddly, if I enable on-aboard audio, only this device gets assigned at the end of the Large Memory. At least this confirms that above 4G decoding is working. audio

I think it's just a size restriction somewhere.

xCuri0 commented 1 year ago

@val3nt33n my system allocates multiple devices in the 64-bit region (gpu not visible in screenshot because it's in large memory)

Have you tried with Reserved Memory = 13, Prefetchable Memory = 13 and Prefetchable Memory above 4G = Enabled, all at the same time ?

val3nt33n commented 1 year ago

Have you tried with Reserved Memory = 13, Prefetchable Memory = 13 and Prefetchable Memory above 4G = Enabled, all at the same time ?

I have. No change.

Tokobotenkai commented 1 year ago

I have an i5-4690k and ASUS Z97-PRO but I only have Ubuntu 22.04 running on it. Any idea how can I contribute or something? Maybe if there's a H97 mainboard that works.

cursemex commented 1 year ago

Browsing through AMIBCP I found some PCIe settings that don't show up in BIOS. Not sure what these do or how to make them visible. All settings(both visible and hidden) have selected 'Yes' in the Show column. I'm confused why it says 'Yes' when it's hidden. The Reserved Memory setting defaults to 10. I wonder if this is the 1GB limit.

Found this while surfing on the web: https://www.bios-mods.com/forum/Thread-GUIDE-How-to-modify-AMI-MSI-BIOS-to-unlock-full-features-UEFI-Aptio4

Step 5 made those options visible in BIOS

val3nt33n commented 1 year ago

@cursemex yeah, that’s what I did when I tested(it’s enough to set USER on the top menu and it will make everything under it visible). But playing with different setting in these new options didn’t do anything for me.

xCuri0 commented 1 year ago

@val3nt33n in cursemex's screenshot it shows "PCIE Port 1 is assigned to LAN". The one with the GPU on is what needs to be changed.

Tokobotenkai commented 1 year ago

I wonder if we have a working board and is there a difference in the order of assignment?

xCuri0 commented 1 year ago

@Tokobotenkai

Prazola on win-raid's Asrock Z97 Extreme9

It also is different for Port 5

My Gigabyte B75M-D3H

Port 7 has "Extra Bus Reserved" set to 7

Romulus_ut3 on win-raid's Asus P8Z77-V PRO

Like my board port 7 has "Extra Bus Reserved" set to 7

all of these don't have the "Prefetchable Memory above 4G"

val3nt33n commented 1 year ago

@val3nt33n in cursemex's screenshot it shows "PCIE Port 1 is assigned to LAN". The one with the GPU on is what needs to be changed.

They all say that(there is not option to change this). And I changed the settings on all ports. I also noticed that on higher ports there are more extra bus reserved and higher Reserved I/O. Playing with them didn't do anything for me.

cursemex commented 1 year ago

Silly question, but how do I know which "root port" my gpu is connected to? HWinfo doesn't state the gpu to be on a "root port", but does for my sound card.

xCuri0 commented 1 year ago

i think i found the problem. PciBus driver is downgrading 64-bit resources to 32-bit ones automatically on Z97 it seems.

Asus Z97(bad)

Asus Z77 (good)

aquamacho1 commented 1 year ago

@xCuri0 can this also be related to #9 ?

xCuri0 commented 1 year ago

@val3nt33n

here's the hex patch if you want to try it. it probably will work, pcihostbridge seems ok for it but only using 32-64GB range for resource allocation which won't be enough for 32GB BAR cards (eg 3090 or others with more than 16GB of VRAM). They seem to have re-used code from Ivy Bridge because Haswell supports upto 512GB physical address. Still there's a risk of it not working so wouldn't recommend trying on non-dual bios unless you have flasher.

in PciBus.efi replace 83 3E 06 75 06 C7 06 04 00 00 00 83 3E 07 75 06 C7 06 05 00 00 00 with 66 90 66 90 66 90 66 90 66 90 66 90 66 90 66 90 66 90 66 90 66 90.

If you want to use UEFIPatch

# PciBus | Don't downgrade 64-bit BARs to 32-bit (Haswell)
3C1DE39F-D207-408A-AACC-731CFB7F1DD7 10 P:833E067506C70604000000833E077506C70605000000:66906690669066906690669066906690669066906690

You'll need to do the fix for pad file problem which is in README

val3nt33n commented 1 year ago

I was getting ready to re-program the chip in case it doesn't boot anymore and to my surprise I found a tiny chip that I don't think I can program(see macro photo attached). My programmer is for a generic(much bigger chip) with fewer pins. Also I don't have USB flashback. The manual suggests that I can upload a new BIOS from a USB drive but inside the BIOS UI. Looks like I was adventuring in all this without any backup.

z97i-plus

val3nt33n commented 1 year ago

@Tokobotenkai can you try to patch first since your board has dual BIOS? @cursemex I think you can try it too.

@xCuri0 does the same code exist on Asus Z97-Pro?

xCuri0 commented 1 year ago

@val3nt33n yes it does (i tried uefipatch it found and patched). There is the pad file problem though but I've written in README how to fix it. Without doing the fix for the pad file you get no boot on Asus boards

val3nt33n commented 1 year ago

@xCuri0 BTW is there any reason why you are suggesting to use MMTool to do the replacement instead of UEFITool? I tried both and in the final file they have different size for the PciBus module. Not sure which one is correct. I have UEFITool_0.28.0_win32 and MMTool 4.50.0.23

xCuri0 commented 1 year ago

MMTool doesn't mess up the pad file like UEFITool does (broken pad file causes no boot). But MMTool can only replace the FFS unlike UEFITool which lets you replace the PE32 executable inside the FFS. Which is why in the README it says to create your modified FFS with UEFITool.

val3nt33n commented 1 year ago

I see. When I replaced the NvramSmi module I used UEFITool because MMTool did not show the string for NvramSmi (it was empty at the line where it should have been. If I tried to replace that, it stores only a 24B empty module (similar to a pad, but not a pad). NvramSmi is Combined SMM/DXE type, which looks like MMTool can't replace properly.

Tokobotenkai commented 1 year ago

I am going to work soon so probably get back to you guys in about 15-16 hours

But great catch @xCuri0 !

@val3nt33n

here's the hex patch if you want to try it. it probably will work, pcihostbridge seems ok for it but only using 32-64GB range for resource allocation which won't be enough for 32GB BAR cards (eg 3090 or others with more than 16GB of VRAM). They seem to have re-used code from Ivy Bridge because Haswell supports upto 512GB physical address.

in PciBus.efi replace 83 3E 06 75 06 C7 06 04 00 00 00 83 3E 07 75 06 C7 06 05 00 00 00 with 66 90 66 90 66 90 66 90 66 90 66 90 66 90 66 90 66 90 66 90 66 90.

If you want to use UEFIPatch
# PciBus | Don't downgrade 64-bit BARs to 32-bit (Haswell)
3C1DE39F-D207-408A-AACC-731CFB7F1DD7 10 P:833E067506C70604000000833E077506C70605000000:66906690669066906690669066906690669066906690
You'll need to do the fix for pad file problem which is in README

@xCuri0 does that mean that I can use a hex editor to modify the existing BIOS? Just search for the string and replace it.

xCuri0 commented 1 year ago

@Tokobotenkai Use UEFIPatch to do it as it's the easiest way you just have to put the UEFIPatch code in patches.txt and run UEFIPatch on the BIOS file in command line.

You still do have to do the workaround with MMTool to avoid getting no boot because of pad file problem though (it's in README).

Tokobotenkai commented 1 year ago

@xCuri0 Just to check, I extracted the body from the CAP, and applied the patch using UEFITool, where do I get the MMTool? Is there a linux version?

val3nt33n commented 1 year ago

run UEFIPatch Z97PRO.cap patches.txt -o Z97PRO-patched.cap
use UEFITool to extract(as is) the PciBus module from the patched file and save the result in a .ffs file
use MMTool (go to replace tab) and load Z97PRO.cap In the Browse section load the .ffs file
got to PciBus line, click on it and then click on Replace, then save the result to a new .cap file(this is now the patched bios that you can use as baseline)
follow @xCuri0's steps to insert the ReBarDxe driver in the new .cap file
do anything else you need to do(i.e. replace NvramSmi if you need to, etc.)

In patches.txt only keep this patch to make sure you are not applying anything else that is not necessary

# PciBus | Don't downgrade 64-bit BARs to 32-bit (Haswell)
3C1DE39F-D207-408A-AACC-731CFB7F1DD7 10 P:833E067506C70604000000833E077506C70605000000:66906690669066906690669066906690669066906690

Note: there is a space at the end of the last line.

I got MMTool 4.50.0023 from https://www.tweaktownforum.com/forum/tech-support-from-vendors/gigabyte/30823-latest-overclocking-programs-system-info-benchmarking-stability-tools

cursemex commented 1 year ago

Thanks for this. I was tryina figure it. I'll do the steps and give you an update

Edit: @val3nt33n first set of the tests are good. I've got up to 8gb with 1 long beep after post, then I've got the beeping error(1 long, 3 short) once I've set 16gb. I have 32gb of ram btw. Gonna recover BIOS now.

Edit2:-

rebar didn't activate on AMD's Adrenaline during these tests.

val3nt33n commented 1 year ago

OK, so after you flashed, the system booted normally? You saw problems only once you experimented with larger bar sizes?

You can try to clear CMOS if a large bar doesn't boot(or you can set iGPU as default right after you flash, if you have one), instead of recovering the BIOS.

cursemex commented 1 year ago

No problems going up to 8GB from my observation aside from 1 long beep after post. I'll redo the testing to confirm it.

As for clearing CMOS, I've noticed that it doesn't like my dedicated gpu on. It'll keep beeping and rebooting until I remove the gpu, then it'll switch to iGPU. I haven't tried it this time around, though.

val3nt33n commented 1 year ago

Can you show device manager (View->Resources by Type) and expand Memory and Large Memory with 8GB?

Your GPU has 12GB of VRAM, no? In that case the max BAR you would need is 16GB. To get 16GB working you probably also need this patch

# PciBus | Remove <16GB BAR size limit (Ivy Bridge/Haswell)
3C1DE39F-D207-408A-AACC-731CFB7F1DD7 10 P:B8FFFFFFFF030000004C3B:B8FFFFFFFFFFFFFF004C3B

cursemex commented 1 year ago

Can you show device manager (View->Resources by Type) and expand Large Memory with 8GB?

Your GPU has 12GB of VRAM, no? In that case the max BAR you would need is 16GB. To get 16GB working you probably also need this patch
# PciBus | Remove <16GB BAR size limit (Ivy Bridge/Haswell)
3C1DE39F-D207-408A-AACC-731CFB7F1DD7 10 P:B8FFFFFFFF030000004C3B:B8FFFFFFFFFFFFFF004C3B 

I was thinking about applying it too, but didn't. Let me restart the process and remake the roms and test it again. And yes, my gpu is 6700XT 12GB.

Tokobotenkai commented 1 year ago

I did another way (I don't have Windows on any of my machines, maybe I will try using the MMTool via a virtual machine):

I patched the modified Body ROM file
Using a hex editor, I pasted the entire patched ROM file into the unmodified Capsule file after the offset
Flashed using USB Flashback

For some reason, my USB wireless keyboard is not detected on boot, means I can't enter BIOS, else once Ubuntu boots, it is automatically detected. EDIT: This is fixed by enabling Full Initialization of USB devices at boot in BIOS.

For now ReBar up to 8GB work, I haven't tried 16GB as the A380 only has 6GB of RAM. I am waiting for a new power supply to connect the A770, or can I just up the limit to 16GB (the A380 from lspci only supports up to 8GB ReBar)?

sudo lspci -vvv
03:00.0 VGA compatible controller: Intel Corporation Device 56a5 (rev 05) (prog-if 00 [VGA controller])
 Subsystem: ASRock Incorporation Device 6004
 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
 Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
 Latency: 0, Cache Line Size: 64 bytes
 Interrupt: pin ? routed to IRQ 44
 Region 0: Memory at de000000 (64-bit, non-prefetchable) [size=16M]
 Region 2: Memory at c00000000 (64-bit, prefetchable) [size=8G]
 Expansion ROM at df000000 [disabled] [size=2M]
 Capabilities: [40] Vendor Specific Information: Len=0c <?>
 Capabilities: [70] Express (v2) Endpoint, MSI 00
  DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
   ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
  DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
   RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
   MaxPayload 128 bytes, MaxReadReq 128 bytes
  DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
  LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
   ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
  LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
   ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
  LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
   TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
  DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR+
    10BitTagComp+ 10BitTagReq+ OBFF Not Supported, ExtFmt+ EETLPPrefix-
    EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
    FRS- TPHComp- ExtTPHComp-
    AtomicOpsCap: 32bit- 64bit- 128bitCAS-
  DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ OBFF Disabled,
    AtomicOpsCtl: ReqEn-
  LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- Retimer- 2Retimers- DRS-
  LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
    Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
    Compliance De-emphasis: -6dB
  LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
    EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
    Retimer- 2Retimers- CrosslinkRes: unsupported
 Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit+
  Address: 00000000fee02004  Data: 0027
  Masking: 00000000  Pending: 00000000
 Capabilities: [d0] Power Management version 3
  Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
  Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
 Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
  ARICap: MFVC- ACS-, Next Function: 0
  ARICtl: MFVC- ACS-, Function Group: 0
 Capabilities: [420 v1] Physical Resizable BAR
  BAR 2: current size: 8GB, supported: 256MB 512MB 1GB 2GB 4GB 8GB
 Capabilities: [400 v1] Latency Tolerance Reporting
  Max snoop latency: 71680ns
  Max no snoop latency: 71680ns
 Kernel driver in use: i915
 Kernel modules: i915

lspci -v |grep -A8 VGA
03:00.0 VGA compatible controller: Intel Corporation Device 56a5 (rev 05) (prog-if 00 [VGA controller])
 Subsystem: ASRock Incorporation Device 6004
 Flags: bus master, fast devsel, latency 0, IRQ 44
 Memory at de000000 (64-bit, non-prefetchable) [size=16M]
 Memory at c00000000 (64-bit, prefetchable) [size=8G]
 Expansion ROM at df000000 [disabled] [size=2M]
 Capabilities: <access denied>
 Kernel driver in use: i915
 Kernel modules: i915

sudo dmesg | grep BAR
[    0.176035] pci 0000:03:00.0: BAR 2: assigned to efifb

cursemex commented 1 year ago

@val3nt33n same issue with the 2 patches. 8 beeps, then reboots with 1 long beep, and 3 short ones then pause, and another short beep with d4 error (D4 means "PCI resource allocation error. Out of Resources".)

It did work with 8gb before I switched it to 16gb (Again). I'll edit with pictures once I reboot with my iGPU.

Edit: Here's the 8GB device manager picture as requested.

xCuri0 commented 1 year ago

@cursemex 16GB doesn't work because the BIOS only uses 0xC00000000 (48GB) to 0xFBFFFFFFFF (63GB) for resource allocation (see Large Memory in Device Manager). It's easy to patch though (did it for the Z97 user on win-raid) so I can make a fix for it but DSDT needs to be edited.

cursemex commented 1 year ago

If you do make them, I'll happily test it out. I may be slow in doing so in the upcoming days due to school though.

Edit: @xCuri0 here's the DSDT that I extracted yesterday

Tokobotenkai commented 1 year ago

From Hardinfo in Ubuntu,

-Memory-
<tt>00000000-00000fff </tt>     : Reserved
<tt>00001000-00057fff </tt>     : System RAM
<tt>00058000-00058fff </tt>     : Reserved
<tt>00059000-0008ffff </tt>     : System RAM
<tt>00090000-00090fff </tt>     : Reserved
<tt>00091000-0009dfff </tt>     : System RAM
<tt>0009e000-0009ffff </tt>     : Reserved
<tt>00000000-00000000 </tt>     : PCI Bus 0000:00
<tt>00000000-00000000 </tt>     : PCI Bus 0000:00
<tt>00000000-00000000 </tt>     : PCI Bus 0000:00
<tt>00000000-00000000 </tt>     : PCI Bus 0000:00
<tt>00000000-00000000 </tt>     : PCI Bus 0000:00
<tt>00000000-00000000 </tt>     : PCI Bus 0000:00
<tt>00000000-00000000 </tt>     : PCI Bus 0000:00
<tt>00000000-00000000 </tt>     : PCI Bus 0000:00
<tt>000a0000-000dffff </tt>     : PCI Bus 0000:00
<tt>000f0000-000fffff </tt>     : System ROM
<tt>00100000-bbbac017 </tt>     : System RAM
<tt>bbbac018-bbc6e057 </tt>     : System RAM
<tt>bbc6e058-bbe17fff </tt>     : System RAM
<tt>bbe18000-bbe1efff </tt>     : ACPI Non-volatile Storage
<tt>bbe1f000-bc27efff </tt>     : System RAM
<tt>bc27f000-bc6f3fff </tt>     : Reserved
<tt>bc6f4000-cd165fff </tt>     : System RAM
<tt>cd166000-cdc93fff </tt>     : Reserved
<tt>cdc94000-cdcb1fff </tt>     : ACPI Tables
<tt>cdcb2000-ce1d5fff </tt>     : ACPI Non-volatile Storage
<tt>ce1d6000-ceffefff </tt>     : Reserved
<tt>cefff000-ceffffff </tt>     : System RAM
<tt>cf000000-cfffffff </tt>     : RAM buffer
<tt>d0000000-feafffff </tt>     : PCI Bus 0000:00
<tt>  de000000-df3fffff </tt>       : PCI Bus 0000:01
<tt>    de000000-df3fffff </tt>     : PCI Bus 0000:02
<tt>      de000000-df1fffff </tt>       : PCI Bus 0000:03
<tt>        de000000-deffffff </tt>     : <b><small>PCI</small></b>  Intel Corporation Device 56a5 (rev 05) (prog-if 00 [VGA controller])
<tt>          de102040-de1020bf </tt>       : i915.spi.768
<tt>          de373000-de373ffb </tt>       : i915.mei-gsc.768
<tt>          de374000-de374ffb </tt>       : i915.mei-gscfi.768
<tt>        df000000-df1fffff </tt>     : <b><small>PCI</small></b>  Intel Corporation Device 56a5 (rev 05) (prog-if 00 [VGA controller])
<tt>      df300000-df3fffff </tt>       : PCI Bus 0000:04
<tt>        df300000-df303fff </tt>     : <b><small>PCI</small></b>  Intel Corporation Device 4f92
<tt>          df300000-df303fff </tt>       : ICH HD audio
<tt>  df400000-df4fffff </tt>       : PCI Bus 0000:0c
<tt>    df400000-df407fff </tt>     : <b><small>PCI</small></b>  ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller (prog-if 30 [XHCI])
<tt>      df400000-df407fff </tt>       : xhci-hcd
<tt>  df500000-df5fffff </tt>       : PCI Bus 0000:06
<tt>    df500000-df5fffff </tt>     : PCI Bus 0000:07
<tt>      df500000-df5fffff </tt>       : PCI Bus 0000:0b
<tt>        df500000-df5001ff </tt>     : <b><small>PCI</small></b>  ASMedia Technology Inc. ASM1061 SATA IDE Controller (rev 02) (prog-if 85 [PCI native mode-only controller, supports bus mastering])
<tt>          df500000-df5001ff </tt>       : <b><small>Module</small></b> AHCI SATA low-level driver
<tt>  df600000-df61ffff </tt>       : <b><small>PCI</small></b>  Intel Corporation Ethernet Connection (2) I218-V
<tt>    df600000-df61ffff </tt>     : <b><small>Module</small></b> Intel(R) PRO/1000 Network Driver
<tt>  df620000-df6207ff </tt>       : <b><small>PCI</small></b>  Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode] (prog-if 01 [AHCI 1.0])
<tt>    df620000-df6207ff </tt>     : <b><small>Module</small></b> AHCI SATA low-level driver
<tt>  df621000-df6213ff </tt>       : <b><small>PCI</small></b>  Intel Corporation 9 Series Chipset Family USB EHCI Controller #1 (prog-if 20 [EHCI])
<tt>    df621000-df6213ff </tt>     : ehci_hcd
<tt>  df622000-df6223ff </tt>       : <b><small>PCI</small></b>  Intel Corporation 9 Series Chipset Family USB EHCI Controller #2 (prog-if 20 [EHCI])
<tt>    df622000-df6223ff </tt>     : ehci_hcd
<tt>  df623000-df623fff </tt>       : <b><small>PCI</small></b>  Intel Corporation Ethernet Connection (2) I218-V
<tt>    df623000-df623fff </tt>     : <b><small>Module</small></b> Intel(R) PRO/1000 Network Driver
<tt>  dffe0000-dffeffff </tt>       : pnp 00:06
<tt>  dfff0000-dfffffff </tt>       : pnp 00:06
<tt>  e0000000-efffffff </tt>       : PCI MMCONFIG 0000 [bus 00-ff]
<tt>    e0000000-efffffff </tt>     : Reserved
<tt>      e0000000-efffffff </tt>       : pnp 00:06
<tt>fec00000-fec00fff </tt>     : Reserved
<tt>  fec00000-fec003ff </tt>       : IOAPIC 0
<tt>fed00000-fed03fff </tt>     : Reserved
<tt>  fed00000-fed003ff </tt>       : HPET 0
<tt>    fed00000-fed003ff </tt>     : PNP0103:00
<tt>fed10000-fed17fff </tt>     : pnp 00:06
<tt>fed18000-fed18fff </tt>     : pnp 00:06
<tt>fed19000-fed19fff </tt>     : pnp 00:06
<tt>fed1c000-fed1ffff </tt>     : Reserved
<tt>  fed1c000-fed1ffff </tt>       : pnp 00:06
<tt>    fed1f410-fed1f414 </tt>     : iTCO_wdt.1.auto
<tt>fed20000-fed3ffff </tt>     : pnp 00:06
<tt>fed40000-fed44fff </tt>     : pnp 00:00
<tt>fed45000-fed8ffff </tt>     : pnp 00:06
<tt>fed90000-fed93fff </tt>     : pnp 00:06
<tt>fee00000-fee00fff </tt>     : Local APIC
<tt>  fee00000-fee00fff </tt>       : Reserved
<tt>ff000000-ffffffff </tt>     : Reserved
<tt>  ff000000-ffffffff </tt>       : INT0800:00
<tt>    ff000000-ffffffff </tt>     : pnp 00:06
<tt>100000000-42effffff </tt>       : System RAM
<tt>  27ec00000-27fc020bf </tt>     : Kernel code
<tt>  27fe00000-2808bafff </tt>     : Kernel rodata
<tt>  280a00000-280e5dc7f </tt>     : Kernel data
<tt>  28113c000-2815fffff </tt>     : Kernel bss
<tt>42f000000-42fffffff </tt>       : RAM buffer
<tt>800000000-fbfffffff </tt>       : PCI Bus 0000:00
<tt>  c00000000-e007fffff </tt>     : PCI Bus 0000:01
<tt>    c00000000-dffffffff </tt>       : PCI Bus 0000:02
<tt>      c00000000-dffffffff </tt>     : PCI Bus 0000:03
<tt>        c00000000-dffffffff </tt>       : <b><small>PCI</small></b>  Intel Corporation Device 56a5 (rev 05) (prog-if 00 [VGA controller])
<tt>    e00000000-e007fffff </tt>       : <b><small>PCI</small></b>  Intel Corporation Device 4fa1 (rev 01) (prog-if 00 [Normal decode])
<tt>  fbff00000-fbff0ffff </tt>     : <b><small>PCI</small></b>  Intel Corporation 9 Series Chipset Family USB xHCI Controller (prog-if 30 [XHCI])
<tt>    fbff00000-fbff0ffff </tt>       : xhci-hcd
<tt>  fbff10000-fbff13fff </tt>     : <b><small>PCI</small></b>  Intel Corporation 9 Series Chipset Family HD Audio Controller
<tt>    fbff10000-fbff13fff </tt>       : ICH HD audio
<tt>  fbff14000-fbff140ff </tt>     : <b><small>PCI</small></b>  Intel Corporation 9 Series Chipset Family SMBus Controller
<tt>  fbff16000-fbff1600f </tt>     : <b><small>PCI</small></b>  Intel Corporation 9 Series Chipset Family ME Interface #1
<tt>    fbff16000-fbff1600f </tt>       : <b><small>Module</small></b> Intel(R) Management Engine Interface
-DMA-
<tt> 4</tt>     : cascade

c00000000 (48GB) to dffffffff (56GB)

So without editing the DSDT, only 8GB is the maximum, am I interpreting correctly?

xCuri0 commented 1 year ago

@Tokobotenkai it's 800000000 (32GB) to fbfffffff (63GB) (PCI Bus 0000:00) which should be good for upto 16GB BAR (given that you have the 16GB patch).

Tokobotenkai commented 1 year ago

@xCuri0 Okay, will try it with A770, just to clarify a bit, I can set it to 16GB when my system only have a total of 16GB of RAM without crashing right?

xCuri0 commented 1 year ago

@Tokobotenkai yes. only if you use 32GB RAM there might be a problem because no free address space. These BIOS only use upto 63GB on Haswell when the hardware supports upto 512GB.

Tokobotenkai commented 1 year ago

@xCuri0 Okay thanks! Let me get back to you then.

val3nt33n commented 1 year ago

@Tokobotenkai it doesn't matter how much RAM your system has. The GPU BAR is not allocated from the physical RAM. It's allocated from the virtual space(Haswell's virtual space is 512GB), within the limits allowed by your BIOS(63 - 32 = 31GB limit that @xCuri0 mentioned above). So, you can have a system with only 4GB of RAM and still be able to use a GPU with a 16GB BAR. They are both independent and are both mapped in the virtual space for easy access by the CPU and applications. The memory ranges on the left in your Hardinfo output represent the virtual addresses where different devices and some RAM(used by the OS) are mapped in the virtual space.

Tokobotenkai commented 1 year ago

@val3nt33n Understood! Thank you.

val3nt33n commented 1 year ago

I plan to try the patch myself tomorrow once I'm done with work.

xCuri0 commented 1 year ago

@cursemex you can try this along with the "Remove <16GB BAR size limit" patch. It worked for a user on win-raid with 16GB GPU (RX 6900XT) and 16GB RAM. Not sure if it will work with 32GB RAM but it's worth trying

# PciHostBridge | Replace 8-16GB MMIO region with complete use of 64GB address space (Haswell). MAY REQUIRE DSDT MODIFICATION
8D6756B9-E55E-4D6A-A3A5-5E4D72DDF772 10 P:080000004823C1483BC3770C48BE0000000004000000EB7A48BB000000000C000000483BC3770C48BE0000000002000000EB5F48BB000000000E000000483BC3770C48BE0000000001000000EB4448BB000000000F000000483BC37707BE00000080EB2E48BB000000800F000000483BC37707BE00000040EB1848BB000000C00F000000483BC30F87A4FEFFFFBE00000020:010000004821C84839D8480F47D848BE000000E00F0000004829DE9090909090909066909066906690909090909090909066906690909090909090909066909066906690909090909090909066906690909090909090909066909066909090909090669066909090909090909090669090669090909090906690669090909090909090906690909090909090909090909090

yes i know it's really long. the same mmtool trick to avoid pad file problems is needed for this too

cursemex commented 1 year ago

Unfortunately, it doesn't apply the patch. Used with "Don't downgrade 64-bit BARs to 32-bit (Haswell)" and "Remove <16GB BAR size limit (Ivy Bridge/Haswell)"

Having "Replace 8-16GB MMIO region with complete use of 64GB address space (Haswell)." by itself didn't work as well.

xCuri0 / ReBarUEFI