iommu=nobypass appears to not work

[mpe]

Booting a powernv box with iommu=nobypass dmesg says:

[    0.000000] PowerNV: IOMMU bypass window disabled.
...
[    2.723971] pci 0001:08     : [PE# fd] Using 64-bit DMA iommu bypass (through TVE#0)

Which is the IPR: 0001:08:00.0 RAID bus controller: IBM PCI-E IPR SAS Adapter (ASIC) (rev 02)

The command line logic is definitely working to set the disabled bool:

d:mon> d4 $pnv_iommu_bypass_disabled
c000000001505f39 00000001

The 2nd messages comes from pnv_pci_ioda_dma_64bit_bypass(). Which is called only from pnv_pci_ioda_iommu_bypass_supported().

So this hunk looks to be at fault:

    /*
     * If the device can't set the TCE bypass bit but still wants
     * to access 4GB or more, on PHB3 we can reconfigure TVE#0 to
     * bypass the 32-bit region and be usable for 64-bit DMAs.
     * The device needs to be able to address all of this space.
     */
    if (dma_mask >> 32 &&
        dma_mask > (memory_hotplug_max() + (1ULL << 32)) &&
        /* pe->pdev should be set if it's a single device, pe->pbus if not */
        (pe->device_count == 1 || !pe->pbus) &&
        phb->model == PNV_PHB_MODEL_PHB3) {
        /* Configure the bypass mode */
        s64 rc = pnv_pci_ioda_dma_64bit_bypass(pe);

Which was part of torvalds/linux@8e3f1b1d8255 powerpc/powernv/pci: Enable 64-bit devices to access >4GB DMA space.

[mpe]

Possible fix:

diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index ec48ea25a674..913f66ce7b9d 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1833,16 +1833,18 @@ static bool pnv_pci_ioda_iommu_bypass_supported(struct pci_dev *pdev,
        struct pnv_phb *phb = hose->private_data;
        struct pci_dn *pdn = pci_get_pdn(pdev);
        struct pnv_ioda_pe *pe;
+       u64 top;

        if (WARN_ON(!pdn || pdn->pe_number == IODA_INVALID_PE))
                return false;

        pe = &phb->ioda.pe_array[pdn->pe_number];
-       if (pe->tce_bypass_enabled) {
-               u64 top = pe->tce_bypass_base + memblock_end_of_DRAM() - 1;
-               if (dma_mask >= top)
-                       return true;
-       }
+       if (!pe->tce_bypass_enabled)
+               return false;
+
+       top = pe->tce_bypass_base + memblock_end_of_DRAM() - 1;
+       if (dma_mask >= top)
+               return true;

        /*
         * If the device can't set the TCE bypass bit but still wants