thesofproject / sof

Sound Open Firmware
Other
547 stars 313 forks source link

[BUG] Error on go to hibernate #5908

Closed ikrivosheev closed 2 years ago

ikrivosheev commented 2 years ago

Describe the bug

[25488.097912] sof-audio-pci-intel-tgl 0000:00:1f.3: error: PM_GATE ipc error -19
[25488.097922] sof-audio-pci-intel-tgl 0000:00:1f.3: failed to set requested target DSP state 0 substate 1
[25488.097923] sof-audio-pci-intel-tgl 0000:00:1f.3: error: setting dsp state 0 substate 1
[25488.097924] sof-audio-pci-intel-tgl 0000:00:1f.3: error: failed to power down DSP during suspend -19
[25488.097925] sof-audio-pci-intel-tgl 0000:00:1f.3: PM: pci_pm_suspend(): snd_sof_suspend+0x0/0x20 [snd_sof] returns -19
[25488.097937] sof-audio-pci-intel-tgl 0000:00:1f.3: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x160 returns -19
[25488.097941] sof-audio-pci-intel-tgl 0000:00:1f.3: PM: failed to suspend async: error -19

To Reproduce systemctl suspend-then-hibernate

Reproduction Rate Always

Expected behavior Computer go to hibernate

Environment 1) Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).

lgirdwood commented 2 years ago

@ikrivosheev thanks for the report, the DSP should be OFF for hibernate S4, but it looks like something is blocking it. @keqiaozhang @miRoox can someone try the same test on TGL. @ujfalusi @plbossart fyi.

ujfalusi commented 2 years ago

Likely duplicate of #5892 ?

ujfalusi commented 2 years ago

In S4 and S5 the IMR boot is not possible due to the fact that the backup sram loosing it's content. The Linux side fix is: https://github.com/thesofproject/linux/pull/3687

ikrivosheev commented 2 years ago

@ujfalusi yes! This is a duplicate, close

ikrivosheev commented 2 years ago

@ujfalusi how do changes come into linux main repository? I see that changes have been made in fork but I cannot understand when it comes into mainline.

ujfalusi commented 2 years ago

@ikrivosheev, in a second look at the log fragment you have, this might not be a duplicate as this happens when trying to enter to suspend. The issue which is fixed by the linked PR was on the resume side of the hibernate (https://github.com/thesofproject/sof/issues/5892)

Let me re-open this and please add this file sof-dyndbg.conf.txt as /etc/modprobe.d/sof-dyndbg.conf (the extension matters), reboot and attach the kernel log gathered in this mode.

ujfalusi commented 2 years ago

@plbossart, fyi, I can not recall seeing such error.

ikrivosheev commented 2 years ago

@ujfalusi thank you! I will add it today.

squat commented 2 years ago

I'm having exactly the same issue when using sof-firmware v2.1.1 on Linux x 5.18.5-arch1-1 on my Dell XPS 13.

If I downgrade SOF to v2.0 then suspending works as expected.

Note: I see this issue with any suspend operation, not just suspend-then-hibernate.

Edited 16:09 UTC: Although suspending is fixed after downgrading v2.0, hibernation still doesn't seem to function. The machine wakes back up almost instantly. I'll see if it works by downgrading further and collect some logs, though I don't want to hijack this issue.

plbossart commented 2 years ago

Agree @ujfalusi this is a different error - we should not have any IPC errors, ever. The firmware downgrade clearly hints at a firmware regression.

igor-winograd commented 2 years ago

I also having sof-firmware v2.1.1-1 on Linux x 5.18.5-arch1-1 with platform Lenovo Thinkpad e15g2. The problem is relevant to me. My kernel log with /etc/modprobe.d/sof-dyndbg.conf: dmesg.log

plbossart commented 2 years ago

@igor-winograd I don't see any issues in your log file?

ikrivosheev commented 2 years ago

@ujfalusi my dmesg logs

  1. after first suspend-then-hibernate (success): dmesg-after-hibernate.log
  2. after second suspend-then-hibernate (fail): dmesg-after-second-hibernate.log
plbossart commented 2 years ago

It's the same error actually, the IPC error is preceded by a boot failure. The second hibernate fails because the resume failed already. We should probably have a check that we don't try ANY IPC during the suspend operation if the DSP didn't boot...

ikrivosheev commented 2 years ago

@ujfalusi @plbossart close issue as duplicate?

And my question:

@ujfalusi how do changes come into linux main repository? I see that changes have been made in fork but I cannot understand when it comes into mainline.

ujfalusi commented 2 years ago

@ikrivosheev, the patches are in linux-next, it depends when stable will pick them (the last patch out of the three will have conflict, but it is trivial). While waiting for them to be backported you can disable the IMR booting (it was introduced in 5.18 and it is a nice to have feature - for those who do not suspend to disk ;) ) by adding

options snd_sof sof_debug=128

to /etc/modprobe.d/sof.conf for example (and remove other module parameter files adding options for SOF modules) and reboot. This will make things work like they used to prior the IMR support regarding to resuming from suspend/hibernate.

Steven-- commented 2 years ago

Amazing, thanks! I was actually having troubles on suspend but only after either:

Laptop model: Thinkpad E14 (Gen 12). Kernel 5.18.0.

Controller: 00:1f.3 Multimedia audio controller: Intel Corporation Tiger Lake-LP Smart Sound Technology Audio Controller (rev 20)

lgirdwood commented 2 years ago

@ujfalusi can we close now ? i.e. are all patches in -next now ?

ujfalusi commented 2 years ago

@lgirdwood, yes all patches are in linux-next and we also have a workaround for 5.18 kernel (sof_debug=128) Closing.

ikrivosheev commented 2 years ago

Thank you all!