search

Dasharo / open-source-firmware-validation

OSFV infrastructure with automated tests and scripts for managing test results
Apache License 2.0
6 stars 1 forks source link

APU: Disabling "Enable PCI Express power management features" doesn't disable L1 #242

Closed pkubaj closed 2 months ago

pkubaj commented 3 months ago

Device

APU2/3/4/6

RTE version

1.1.0

Affected component(s) or functionality

No response

Brief summary

ASPM management doesn't work correctly on APU

How reproducible

No response

How to reproduce

Disable "Enable PCI Express power management features" in UEFI menu and check available ASPM states.

Expected behavior

ASPM L0s and L1 should be disabled.

Actual behavior

Only L0s is getting disabled.

Link to screenshots or logs

--- enabled.log 2024-03-15 16:38:34.509006812 +0100
+++ disabled.log    2024-03-15 16:41:29.500855313 +0100
@@ -201,7 +201,7 @@
        LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x1
-           TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
+           TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
        SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
            Slot #0, PowerLimit 0W; Interlock- NoCompl+
        SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
@@ -431,7 +431,7 @@
        DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
        LnkCap: Port #1, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 <16us
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
-       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
+       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x1
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
@@ -492,7 +492,7 @@
        DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
        LnkCap: Port #2, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 <16us
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
-       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
+       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x1
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
@@ -553,7 +553,7 @@
        DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
        LnkCap: Port #3, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 <16us
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
-       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
+       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x1
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

Additional context

No response

Solutions you've tried

No response

macpijan commented 3 months ago

@SergiiDmytruk Can you please take a look? See also the related discussion in PR.

SergiiDmytruk commented 3 months ago

As far as I can see, coreboot should instruct AGESA to disable ASPM and not enable it anywhere. Can verify that and see if release with SeaBIOS behaved any differently with regard to L1.

SergiiDmytruk commented 3 months ago

Added excessive logging and there seems to be no bug in the logic, coreboot doesn't try to enable ASPM if the option is off.

Device APU2/3/4/6

@pkubaj, have you checked all of them? I remember seeing L1s enabled (maybe on APU3), but this is from Ubuntu on APU4 when the option is disabled:

$ sudo lspci -vvv | grep -i LnkCtl
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-

@miczyg1 Could it be that the option just can't fully disable ASPM on some models? This was my assumption on seeing L1s stay after disabling ASPM.

mkopec commented 3 months ago

I'm seeing the same as @SergiiDmytruk on APU4 and APU6

@pkubaj which platform did you observe this behavior on? I guess it could also be OS dependent

mkopec commented 3 months ago

logs from ubuntu 22.04

apu4:

$ diff lspci_noaspm.log lspci_aspm.log
48c48
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
123c123
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
198c198
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
201c201
<           TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
---
>           TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
273c273
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
276c276
<           TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
---
>           TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
348c348
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
595c595
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
656c656
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
717c717
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
778c778
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
834c834
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+

apu6:

$ diff lspci_noaspm.log lspci_aspm.log
48c48
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
123c123
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
198c198
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
273c273
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
348c348
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
594c594
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
654c654
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
715c715
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
776c776
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
832c832
<       LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
---
>       LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
pkubaj commented 3 months ago

I'm seeing the same as @SergiiDmytruk on APU4 and APU6

@pkubaj which platform did you observe this behavior on? I guess it could also be OS dependent

APU2 with Debian 12.

macpijan commented 3 months ago

So does it mean that it works as expected on ubuntu, and @pkubaj can finalize (extend) this test case?

mkopec commented 3 months ago

Yes, we can proceed with finishing this test case (adding L1 check)

miczyg1 commented 3 months ago

Clock PM seems not to be supported on Root Ports on apu... Weird...

pkubaj commented 3 months ago

Seems to happen only on Debian.

macpijan commented 2 months ago

@pkubaj Can you please summarize this?

  1. The tests were fully implemented as we wanted? Can you link to them?
  2. There was a problem with the outcome (but not with the tests itself), on Debian OS only?
pkubaj commented 2 months ago
  1. Yes, https://github.com/Dasharo/open-source-firmware-validation/pull/232
  2. I'm not sure whether it's Debian ONLY (could be also on other systems), but it works fine on Ubuntu.
macpijan commented 2 months ago

I'm not sure whether it's Debian ONLY (could be also on other systems), but it works fine on Ubuntu.

Yes, that is what I meant. Thanks. So nothing more to do on the tests side.