search

sonyxperiadev / bug_tracker

Empty repository that is used as a bugtracker for Open Devices project
52 stars 13 forks source link

[Tama][XZ3] SDM845 PostmarketOS on mailine kernel #799

Open phodina opened 1 year ago

phodina commented 1 year ago

Platform: Tama Device: XZ3 Kernel version: 6.4.0 Android version: None Software binaries version: edge

Hi, this is continuation of ticket to get PostmarketOS working on Tama XZ3. Currently it boots but do to the missing UFS drivers, that won't wipe the UFS storage the device needs workaround to boot off the SDcard.

https://github.com/sonyxperiadev/kernel/issues/2558

@MarijnS95 @jerpelea @konradybcio @Thaodan

MartinX3 commented 1 year ago

cc @rinigus Our tama sodp Sailfish OS maintainer

rinigus commented 1 year ago

For SFOS, we use Sony kernel sources with Jolla's patches and Tama config. Kernel and config are available at https://github.com/sailfishos-sony-tama/android_kernel_sony_msm . That one boots from storage just fine. But, as you can see, kernel version is 4.14.xxx

MarijnS95 commented 1 year ago

@rinigus thanks, but yes this issue "solely" concerns with PmOS on a mainline kernel.

(@phodina can you update the title with that? Afaik PmOS is also totally designed for and capable of running on older kernels)

Maybe SfOS can utilize that some day when we resolve more issues and missing features upstream.

phodina commented 1 year ago

Btw is this the commit that fixes the UFS? (on the downstream kernel)

https://github.com/sailfishos-sony-tama/android_kernel_sony_msm/commit/cfd148999d17ad8f1a7cbce44bb108570ad631ad

commit cfd148999d17ad8f1a7cbce44bb108570ad631ad
Author: Angelo G. Del Regno <kholk11@gmail.com>
Date:   Fri May 24 16:19:17 2019 +0200

    scsi: ufs: Forward port Yoshino/Tama UFS quirks/hacks from OD k4.9

    This is a forward port of the SoMC Yoshino and SoMC Tama quirks/hacks
    for their UFS cards,
    ************* required to avoid a full UFS card erase *************
    *************        at kernel boot, which would      *************
    ************* E R A S E   T H E   B O O T L O A D E R *************

    .... taken from the current implementation found in the Open Devices
    kernel 4.9 as of the current HEAD (24/05/2019) and partially
    reimplemented for kernel 4.14.

    BEWARE: DO NOT BOOT SoMC YOSHINO OR TAMA WITHOUT THIS COMMIT!!!!
    ------- BOOTING ONE OF THEM WITHOUT THIS COMMIT WILL PERMANENTLY
    ------- (HARD) BRICK YOUR DEVICE WITH ZERO CHANCES TO RECOVER IT.
MarijnS95 commented 1 year ago

mainline :wink:

That is the commit, and there should be a port to mainline somewhere but no-one dared to sacrifice their device to test it yet.

phodina commented 1 year ago

Sorry, yeah meant mainline.

Okay, let's leave it as the last thing to do in case it bricks the device ;-)

konradybcio commented 1 year ago

0001-arm64-dts-qcom-sdm845-tama-Set-serial-indices-and-st.zip try this

(unzip and then git am 0001-.....patch)

phodina commented 1 year ago

Thanks @konradybcio for the patch. So the debug output has to be enabled in the patch in the kernel.

The bootloader has no such functionality and the logs before the kernel boots are therefore not accessible.

phodina commented 1 year ago

Another question regarding the UFS.

  1. Take backup of the UFS from LineageOS
  2. Boot Linux into initramfs with connectivity e.g. ETH over USB.
  3. Have the UFS enabled but compiled as a module
  4. Load the kernel module
  5. In case it wipes the UFS just dd it back?
MarijnS95 commented 1 year ago

So the debug output has to be enabled in the patch in the kernel.

Yes, always. And this is mostly where you want to look as we've already gone through the work to make these kernels boot fully: if anything is up (and you get no USB for example), it'll be in the kernel logs.

The bootloader has no such functionality and the logs before the kernel boots are therefore not accessible.

Seems so, unfortunately. I was able to see them on Loire but that's a different (older) platform, YMMV on newer ones.

phodina commented 1 year ago

0001-arm64-dts-qcom-sdm845-tama-Set-serial-indices-and-st.zip try this

(unzip and then git am 0001-.....patch)

Since the patch just enables the UART how are the pins managed?

The Debug UART is on the SD card pins and I can see on the scope the CLK signal from the SD card controller to attempt to talk to the card.

Is there some mux that is also need to be enabled?

The modified micro SD card adapter I have has the micro SD card form factor and it triggers the Card Detect pin, that tells the kernel, there's a card inserted.

phodina commented 1 year ago

The LineageOS wiki entry lists 3 models. Do we know the differences?

phodina commented 1 year ago

0001-arm64-dts-qcom-sdm845-tama-Set-serial-indices-and-st.zip try this

(unzip and then git am 0001-.....patch)

I've applied the patch but didn't get any output. Probed the pins and the pads on the uSD inserted close the circuit if I connect my multimeter to the other side of the flex cable.

MarijnS95 commented 1 year ago

Typically dualsim and singlesim, and of that list H9436 has 6GB RAM whereas the others have 4.

konradybcio commented 1 year ago

0001-arm64-dts-qcom-sdm845-tama-Set-serial-indices-and-st.zip try this (unzip and then git am 0001-.....patch)

I've applied the patch but didn't get any output. Probed the pins and the pads on the uSD inserted close the circuit if I connect my multimeter to the other side of the flex cable.

hmm.. I wonder if the tray detection kills off uart and kicks in sdcard.. do you think you could jerry rig pushing the sdcard adapter in without the tray?

phodina commented 1 year ago

0001-arm64-dts-qcom-sdm845-tama-Set-serial-indices-and-st.zip try this (unzip and then git am 0001-.....patch)

I've applied the patch but didn't get any output. Probed the pins and the pads on the uSD inserted close the circuit if I connect my multimeter to the other side of the flex cable.

hmm.. I wonder if the tray detection kills off uart and kicks in sdcard.. do you think you could jerry rig pushing the sdcard adapter in without the tray?

That's my concern https://github.com/sonyxperiadev/bug_tracker/issues/799#issuecomment-1594210432.

I have dremel so I can remove part of the pcb.

Do we have some photo of something Sony or somebody else used in order to get the UART working?

Or can't we just revert the CD pin? Insertion disables the uSD card interface.

phodina commented 1 year ago

Good news. The issue was in loading the firmware. It was in incorrect path so using a framebuffer I could get to the logs and then modify accordingly the path. image

phodina commented 1 year ago

@MarijnS95 In order to get the device supported in the PostmarketOS we'll have to pick the patches and apply it to the linux-postmarketos-sdm845 kernel.

Is my assumption correct that what's missing in the mainline is the display panel, display stream controller support and touch controller?

phodina commented 1 year ago

dmesg.log

phodina commented 1 year ago

There are few things in the log that stand out:

phodina commented 1 year ago

Also I noticed the touchscreen has input issues on the edges, same can be said if LineageOS runs on the device.

Is that due to missing configuration or support in the driver?

Sony has the Side Sense feature which works on the edges. Do we now how it works? Is it implemented in the kernel or directly in the firmware of the touchscreen controller?

phodina commented 1 year ago

https://gitlab.com/postmarketOS/pmaports/-/merge_requests/4183

MartinX3 commented 1 year ago

@phodina Thank you very much for your work Could you also add apollo and akari? :) These have two different panel id's but we only saw the two id's of apollo in the wild. It seems that the second kind of akari panel was never sold international)

phodina commented 1 year ago

@MartinX3 I can definitely add those.

Will there be volunteers to test the images on their devices?

Can you point out the panels? I'll have to port them later as well.

MartinX3 commented 1 year ago

@phodina Thank you :)

Since I own only a single device I couldn't test it, but I could setup a build server and provide images on xda for volunteers like I already do for SODP AOSP on tama.

About the panel: Issue tickets:

PRs:

phodina commented 1 year ago

Sharing prebuild images through Github Actions (or also instructions) on XDA is a great idea.

Thanks for the Issues and PRs.

I'll add the Apollo and Akari and ping once done.

phodina commented 1 year ago

@MartinX3 Oki, so the device is now renamed to tama and has the 3 subpackages.

[17:38:50] Device codename: tama
[17:38:54] Which kernel do you want to use with your device?
[17:38:54] Available kernels (3):
[17:38:54] * akari: Sony Xperia XZ2
[17:38:54] * akatsuki: Sony Xperia XZ3
[17:38:54] * apollo: Sony Xperia XZ2 Compact
phodina commented 1 year ago

Btw do we have a firmware for the tama platform? I pulled this one from the device but have no clue which is the latest one.

https://gitlab.com/phodina/firmware-sony-akatsuki

phodina commented 1 year ago

I'll look later into building those images in automated fashion.

phodina commented 1 year ago

Btw does the tama platform support avb_custom_key partition for signing the builds and relocking the bootloader?

phodina commented 1 year ago

Btw does the tama platform support avb_custom_key partition for signing the builds and relocking the bootloader?

Found ticket that says only newer devices are supported

https://github.com/sonyxperiadev/bug_tracker/issues/743

phodina commented 1 year ago

Also I noticed the touchscreen has input issues on the edges, same can be said if LineageOS runs on the device.

Is that due to missing configuration or support in the driver?

Sony has the Side Sense feature which works on the edges. Do we now how it works? Is it implemented in the kernel or directly in the firmware of the touchscreen controller?

I've found a thread on the XDA forum. So If my understanding is correct this feature is just APK and does not rely on any special functionality of the touch controller?

I just listens for all the touch events and in case it happens on the edges of the screen and it's evaluated as a double tap it launches the app container?

https://forum.xda-developers.com/t/sidesense-xperia-xz3-feature-for-other-xperia-device.3851426/

jerpelea commented 1 year ago

Btw does the tama platform support avb_custom_key partition for signing the builds and relocking the bootloader?

no

MarijnS95 commented 1 year ago

hmm.. I wonder if the tray detection kills off uart and kicks in sdcard.. do you think you could jerry rig pushing the sdcard adapter in without the tray?

Is there tray detection? I thought it was only SIM and SDCard detection, because the tray can be present with nothing in it... (at least that is the case for the SDCard cd pin, afaik).

we'll have to pick the patches and apply it to the linux-postmarketos-sdm845 kernel.

Sounds like unnecessary pain, wait until we land the rest in upstream instead of contributing to another downstream.

Is my assumption correct that what's missing in the mainline is the display panel, display stream controller support and touch controller?

Almost. Display Stream Compression but my patches for that should have made it into 6.3 or 6.4 already (the current DSC work is for newer hardware, sdm845 is a relic by comparison). Only the panel driver and DT for that is missing, and maybe a few minor bits and bobs. The touch controller should be upstream, I only have a patch to confirm a wakeup method.

lots of blah about dmesg contents

TBH I'm not interested in your opinion on what's in dmesg. Please only spend bandwidth identifying and discussing actual issues.

[17:38:54] Which kernel do you want to use with your device?
[17:38:54] Available kernels (3):
[17:38:54] * akari: Sony Xperia XZ2
[17:38:54] * akatsuki: Sony Xperia XZ3
[17:38:54] * apollo: Sony Xperia XZ2 Compact

These should all use the same kernel, only differentiated by DTB(O), or is this a necessary hack to get pmOS to share things across the same device family (:vomiting_face:)? Fwiw nowadays most Sony phones are very similar (so far as generic mainline userspace drivers are concerned) and only differ by partition layout.

phodina commented 1 year ago

These should all use the same kernel, only differentiated by DTB(O), or is this a necessary hack to get pmOS to share things across the same device family (vomiting_face)? Fwiw nowadays most Sony phones are very similar (so far as generic mainline userspace drivers are concerned) and only differ by partition layout.

@MarijnS95 the kernel is the same, this is just "semantic" to select the right DTB across the same family.

Almost. Display Stream Compression but my patches for that should have made it into 6.3 or 6.4 already (the current DSC work is for newer hardware, sdm845 is a relic by comparison). Only the panel driver and DT for that is missing, and maybe a few minor bits and bobs. The touch controller should be upstream, I only have a patch to confirm a wakeup method.

In that case I won't go through the patches. In the meantime I'll try the 6.3.9 and 6.4-rc7 releases. The idea is to have single mainline kernel + maybe some minor patches that are waiting to be accepted for the SDM845 devices.

Is there tray detection? I thought it was only SIM and SDCard detection, because the tray can be present with nothing in it... (at least that is the case for the SDCard cd pin, afaik).

Hm, you are probably right that it's for both the SIM and uSD card as the device just detects the tray as whole. Nevertheless you'd need to make some tray that does not trigger the CD pin in that case.

MarijnS95 commented 1 year ago

this is just "semantic"

Eww :joy:

In that case I won't go through the patches

Indeed, don't bother. There's a lot of dev-stuff... and it is available on mainline when it is available. Unless you really need it now and feel comfortable searching through our massive backlog.

Fwiw the panel is on the list already, but has seen many cleanups since.

Hm, you are probably right that it's for both the SIM and uSD card as the device just detects the tray as whole. Nevertheless you'd need to make some tray that does not trigger the CD pin in that case.

I'm saying that it's not, if I remember correctly the CARD detect pin only changes if there actually is a card, not if there's a tray.

Unless Tama/Akatsuki is different from the device I've last observed this at.

phodina commented 1 year ago

Well the UART is still on the backlog as the uSD card is the only storage currently available.

Is the UFS disabled because nobody yet tried the patches? Or tested and they didn't work?

konradybcio commented 1 year ago

maybe try taping over pins that aren't meant for serial..

MartinX3 commented 1 year ago

Is the UFS disabled because nobody yet tried the patches? Or tested and they didn't work?

Because everyone including me is too scared to loose his own device. You said you've got a lab to restore a formatted UFS? Maybe you're our hope to test this patch. :D

MarijnS95 commented 1 year ago

Indeed, Akatsuki's Card Detect pin triggers on tray insertion, not based on a physical card being in the slot.

But you can easily turn off sdhc_2 to not have the pinmuxes fight.

Is the UFS disabled because nobody yet tried the patches? Or tested and they didn't work?

We discussed this to length before?

phodina commented 1 year ago

Because everyone including me is too scared to loose his own device. You said you've got a lab to restore a formatted UFS? Maybe you're our hope to test this patch. :D

Fair enough. Yes, I can replace the UFS as we have a lab at work where we changed some BGA eMMC and SoCs as part of R&D.

But my understanding is I can't just jump into LineageOS and do dd id=/dev/ufs of=backup.img, right? The partitions are stored as LUNs so not just the content but also this metadata needs to be backup.

MarijnS95 commented 1 year ago

@phodina now I'm very curious what you do for work :grimacing:

In theory dding the full UFS block device should work? So /dev/sda, not the individual partitions, that afaik includes all bytes on the block device including LUNs?

konradybcio commented 1 year ago

you won't get a full ufs backup, the secure side makes parts of it read-as-zeros

phodina commented 1 year ago

@konradybcio By the secure side you mean the Replay Protected Memory Block?

Do you know if it will be problem booting the system (The secure world side with Trust Zone)? If so soldering new UFS won't help (unless the secure firmware provisions the UFS). Though there are many videos on Youtube where people upgrade the storage and show the phone boots and recognizes the bigger storage. However can't say how genuine they are.

Is that area protected against wiping if the patch does not work? If it's kept they removing the chip, programming it in socket, reballing and putting it back might work. If it's wiped then the RPMB won't probably allow to be provisioned for the second time, right?

phodina commented 1 year ago

@phodina now I'm very curious what you do for work grimacing

In theory dding the full UFS block device should work? So /dev/sda, not the individual partitions, that afaik includes all bytes on the block device including LUNs?

@MarijnS95 Working on biometric cameras something like Windows Hello :wink:

konradybcio commented 1 year ago

@konradybcio By the secure side you mean the Replay Protected Memory Block?

Both that and the hypervisor+trustzone+etc stack

Do you know if it will be problem booting the system (The secure world side with Trust Zone)? If so soldering new UFS won't help (unless the secure firmware provisions the UFS). Though there are many videos on Youtube where people upgrade the storage and show the phone boots and recognizes the bigger storage. However can't say how genuine they are.

Depends on the vendor.

Is that area protected against wiping if the patch does not work? If it's kept they removing the chip, programming it in socket, reballing and putting it back might work. If it's wiped then the RPMB won't probably allow to be provisioned for the second time, right?

Probably.

phodina commented 1 year ago

@MarijnS95 So I've applied the patch set to a mainline kernel and booted on the device.

Unfortunately I don't get anything on the display. The patches applied on the mainline without any issue and I just enabled the DRM_PANEL_SONY_AKATSUKI_LGD. Not sure if I'm not missing some other config. Checked against the one from your repo and there isn't anything that would suggest otherwise IMHO.

https://gitlab.com/phodina/pmaports/-/commit/f972caefd2133add20bf2a90d7820626d962219a

MarijnS95 commented 1 year ago

@phodina you might be missing out on the DTS panel patch (and maybe the touchscreen patch):

https://github.com/SoMainline/linux/commits/marijn/longbois-next/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama-akatsuki.dts

Specifically: https://github.com/SoMainline/linux/commit/a628ddfefd98784b0af61f2f1dbdfaf4a1e2a293

If that doesn't work share a dmesg. It's always tricky to pry a few relevant commits from such a massive history of in-progress work.

phodina commented 1 year ago

@MarijnS95 here's the dmesg log from booting to initramfs where I can get already display output on the SoMainline branch.

sony-mainline-akatsuki-panel-patches-initramfs.dmesg.txt sony-mainline-akatsuki-panel-patches-userspace.dmesg.txt

$ zcat /proc/config.gz | grep AKAT
CONFIG_DRM_PANEL_SONY_AKATSUKI_LGD=y

Also dumped from the device:

The panel definition is present there:

panel@0 {
                    reg = <0x00>;
                    vddio-supply = <0xcd>;
                    pinctrl-0 = <0xce 0xcf>;
                    pinctrl-1 = <0xd0 0xcf>;
                    pinctrl-names = "default\0sleep";
                    compatible = "sony,akatsuki-lgd";
                    reset-gpios = <0x79 0x06 0x00>;

                    port {

                        endpoint {
                            remote-endpoint = <0xd1>;
                            phandle = <0xcc>;
                        };
                    };
                };

And yes the touchscreen won't work without the additional patches but that's currently known and not a concern as the display panel is important step to get output information.

MarijnS95 commented 1 year ago

where I can get already display output

@phodina does that mean it is working now (contrary to the comment before)? The dmesg looks good and I cannot tell what issues you are observing on which configuration.