lineageos4microg / docker-lineage-cicd

Docker microservice for LineageOS Continuous Integration and Continous Deployment
https://hub.docker.com/r/lineageos4microg/docker-lineage-cicd
GNU General Public License v3.0
502 stars 196 forks source link

Lineageos 20 #335

Closed Exodusnick closed 1 year ago

Exodusnick commented 2 years ago

Linaeageos 20.0 is upcoming. https://www.lineageoslog.com/20.0

We might want to adopt the build script and patch to that :-)

Sam13 commented 1 year ago

Same here, dirty flashed FP4 build - everything works!

shaunmulligan commented 1 year ago

Hi everyone, I just flashed a build of lineage 20.0 onto pixel 4a (sunfish) and it works but its completely missing all the microG packages and only runs 2 of 4 CUSTOM_PACKAGES I added in. I can see the other two APKs in /product/app but they are not listed in app drawer.

This is my build command:

sudo docker run -d \
    -e "BRANCH_NAME=lineage-20.0" \
    -e "DEVICE_LIST=sunfish" \
    -e "SIGNATURE_SPOOFING=restricted" \
    -e "WITH_GMS=true" \
    -e "CUSTOM_PACKAGES=Termux TermuxBoot Magisk Xserver"  \
    -v "/home/shaunmulligan/lineage:/srv/src" \
    -v "/home/shaunmulligan/zips:/srv/zips" \
    -v "/home/shaunmulligan/logs:/srv/logs" \
    -v "/home/shaunmulligan/cache:/srv/ccache" \
    -v "/home/shaunmulligan/manifests:/srv/local_manifests" \
    lineageos4microg/docker-lineage-cicd

Any ideas or help troubleshooting this?

Thanks in advance!

petefoth commented 1 year ago

Any ideas or help troubleshooting this?

Does your local manifest include lineageos4microg/android_vendor_partner_gms?

shaunmulligan commented 1 year ago

No it doesn't, I thought that it was already include by default. Do I need to include it manually in local manifests?

petefoth commented 1 year ago

Do I need to include it manually in local manifests?

Yes - see https://github.com/lineageos4microg/docker-lineage-cicd#gms--microg . There's no need to add the apps to the CUSTOM_PACKAGES variable, but you do need the following in one of your local manifests

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
    <project path="vendor/partner_gms" name="lineageos4microg/android_vendor_partner_gms" remote="github" revision="master" />
</manifest>
shaunmulligan commented 1 year ago

oh I missed that detail. Thanks, I have added it in and kicked off another build. Still not sure why only half of my CUSTOM_PACKAGES are listed in the app drawer but think its got to do with the LOCAL_CERTIFICATE option as I am getting this in logcat: PackageManager: Failed to scan /product/app/Xserver: No APK Signature Scheme v2 signature in package /product/app/Xserver/Xserver.apk

petefoth commented 1 year ago

Still not sure why only half of my CUSTOM_PACKAGES are listed in the app drawer but think its got to do with the LOCAL_CERTIFICATE option as I am getting this in logcat: PackageManager: Failed to scan /product/app/Xserver: No APK Signature Scheme v2 signature in package /product/app/Xserver/Xserver.apk

Yes there are a few problems with app signing in LOS 20. See this StackOverflow post for details, amd a couple of workarounds.

The fix will involve making changes to the Android.mk file for the app in question. Adding the line LOCAL_REPLACE_PREBUILT_APK_INSTALLED := $(LOCAL_PATH)/$(LOCAL_MODULE).apk should ensure that your original .apk file is installed rather than a version that has its signature stripped by the build system

That fix mostly worked for me, but I still have a problem with one of my added apps: when I run the app I see this error: EXCEPTION java.lang.UnsatisfiedLinkError: dlopen failed: library "libconscrypt_jni.so" not found I haven;t worked out how to fix that one yet :(

shaunmulligan commented 1 year ago

Amazing, thanks @petefoth I will give that a try!

bananer commented 1 year ago

By my estimates, building 20.0, 19.1 and 18.1 on our build server takes more than 14 days to complete. Since 18.1 is still officially supported by LineageOS I also want to support it for lineageos4microg.

For this reason, only one round of builds per month will run. In the current round (started Feb 10), 19.1 and 20.0 mostly succeded, now 18.1 builds should start sometime tomorrow.

Iey4iej3 commented 1 year ago

By my estimates, building 20.0, 19.1 and 18.1 on our build server takes more than 14 days to complete. Since 18.1 is still officially supported by LineageOS I also want to support it for lineageos4microg.

For this reason, only one round of builds per month will run. In the current round (started Feb 10), 19.1 and 20.0 mostly succeded, now 18.1 builds should start sometime tomorrow.

I would suggest to compile two rounds for non A/B devices — A/B devices have an appropriate setup, if I understand correctly, to downgrade from a seriously unbootable update, while it seems extremely inconvenient for non A/B devices, since "Nandroid backups" no longer work.

nanoandrew4 commented 1 year ago

Anyone else have safetynet failing and gcm disabled? I've been seeing this for a week or two on my newer builds

kurt-by commented 1 year ago

Anyone else have safetynet failing and gcm disabled? I've been seeing this for a week or two on my newer builds

Safetynet failing is a result of server side updates. There are adapted magisk/LSPosed modules to pass.

Disabled cloud messaging is caused by a microG update. Simply enable it.

Nothing lineage4microG docker related.

js290 commented 1 year ago

What am I doing wrong for lineage-20.0?

docker run \
    -e BRANCH_NAME="lineage-20.0" \
    -e DEVICE_LIST="payton" \
    -v "~/android/build/lineage:/srv/src" \
    -v "~/android/build/zips:/srv/zips" \
    -v "~/android/build/logs:/srv/logs" \
    -v "~/android/build/cache:/srv/ccache" \
    -v "~/android/build/manifests:/srv/local_manifests" \
    lineageos4microg/docker-lineage-cicd

Set cache size limit to 50.0 GB
>> [Sun May  7 23:46:45 UTC 2023] Building branch lineage-20.0 is not (yet) suppported
petefoth commented 1 year ago

Are you running the latest docker image? Support for building lineage-20.0 was added in January:

Interestingly, that code sets themuppets_branch="lineage-20.0" - looking at TheMuppets repos, the value should be lineage-20. I'm confused :)

FintasticMan commented 1 year ago

See #364 for the TheMuppets branch situation.

js290 commented 1 year ago

See #364 for the TheMuppets branch situation.

which image registry/source is docker run... lineageos4microg/docker-lineage-cicd pulling from?

tokenwizard commented 1 year ago

I saw an earlier comment that builds for LOS 20 were to start in Jan. I do not see any for LOS20 in the Releases section. Am I missing something? Do we have a working patch for LOS20 at this point?

LuccoJ commented 1 year ago

Uh? I'm running LOS for microG 20 on two phones, Oneplus 8T and Oneplus One. There are plenty of 20 builds...

tokenwizard commented 1 year ago

Uh? I'm running LOS for microG 20 on two phones, Oneplus 8T and Oneplus One. There are plenty of 20 builds...

Ok, I was looking to patch LOS20 for haydn. I don't see any ROMS out there for haydn with Signature Spoofing patched in. And I don't see a patch for Android 13 on the Releases page (latest one says for Lineage 19.1).

UPDATE: You can disregard. I see there is a LOS20 build for haydn on the Lineage for MicroG site. I did not see it last I checked. I was planning to build by own.

js290 commented 1 year ago

in my previous comment, docker run... lineageos4microg/docker-lineage-cicd has not been working for private LOS20 builds likely because the lineageos4microg/docker-lineage-cicd image has not been updated. The official MicroG builds are likely pulling from an updated image from a different docker repo source.

petefoth commented 1 year ago

docker run... lineageos4microg/docker-lineage-cicd has not been working for private LOS20 builds likely because the lineageos4microg/docker-lineage-cicd image has not been updated.

The docker image from https://hub.docker.com/r/lineageos4microg/docker-lineage-cicd has been working for me for private LOS20 builds for some time. It was last updated by @bananer on May 5, 2023 at 11:57 am

The latest commit in the repo was shortly before that, so the docker hub image is the latest version

The official MicroG builds are likely pulling from an updated image from a different docker repo source.

Not so. Official builds use the latest image, and the same docker run command as in the README examples

If your local LOS20 builds are failing, it must be for some other reason

js290 commented 1 year ago

If your local LOS20 builds are failing, it must be for some other reason

Set cache size limit to 50.0 GB
> [Sun May  7 23:46:45 UTC 2023] Building branch lineage-20.0 is not (yet) suppported

either way, I think the examples in the README may no longer be compatible with the new docker buildx...

unknown shorthand flag: 'e' in -e
unknown shorthand flag: 'v' in -v
See 'docker buildx build --help'.
petefoth commented 1 year ago

>> [Sun May 7 23:46:45 UTC 2023] Building branch lineage-20.0 is not (yet) suppported

I think the docker image you are using is not the latest. Support for building lineage-20.0 was added to the code months ago (see https://github.com/lineageos4microg/docker-lineage-cicd/blame/master/src/build.sh#L139), and merged to the master image shortly after that (see https://github.com/lineageos4microg/docker-lineage-cicd/pull/366). When did you last pull the image from DockerHub?

I would suggest you delete your local copy of the docker image, pull the latest from DockerHub, and try your build again

docker pull lineageos4microg/docker-lineage-cicd

unknown shorthand flag: 'e' in -e unknown shorthand flag: 'v' in -v See 'docker buildx build --help'.

What docker run command did you use to generate those errors?

js290 commented 1 year ago

When did you last pull the image from DockerHub?

I must have missed this in the README.

What docker run command did you use to generate those errors?

docker build \
-e BRANCH_NAME="lineage-20.0" \
-e DEVICE_LIST="payton" \
-v "~/android/build/lineage:/srv/src" \
-v "~/android/build/zips:/srv/zips" \
-v "~/android/build/logs:/srv/logs" \
-v "~/android/build/cache:/srv/ccache" \
-v "~/android/build/manifests:/srv/local_manifests" \
lineageos4microg/docker-lineage-cicd

with the following packages;

||/ Name                  Version                       Architecture Description
+++-=====================-=============================-============-=================================================>
ii  containerd.io         1.6.21-1                      amd64        An open and reliable container runtime
ii  docker-buildx-plugin  0.10.5-1~ubuntu.23.04~lunar   amd64        Docker Buildx cli plugin.
ii  docker-ce             5:24.0.2-1~ubuntu.23.04~lunar amd64        Docker: the open-source application container eng>
ii  docker-ce-cli         5:24.0.2-1~ubuntu.23.04~lunar amd64        Docker CLI: the open-source application container>
ii  docker-compose-plugin 2.18.1-1~ubuntu.23.04~lunar   amd64        Docker Compose (V2) plugin for the Docker CLI.
petefoth commented 1 year ago

My apologies. Having looked into this a bit more, you need to use the docker image taggedlineage-20

So your docker run command would be

docker run \
    -e BRANCH_NAME="lineage-20.0" \
    -e DEVICE_LIST="payton" \
    -v "~/android/build/lineage:/srv/src" \
    -v "~/android/build/zips:/srv/zips" \
    -v "~/android/build/logs:/srv/logs" \
    -v "~/android/build/cache:/srv/ccache" \
    -v "~/android/build/manifests:/srv/local_manifests" \
    lineageos4microg/docker-lineage-cicd:lineage-20

I have created #439 to ensure the lineage-20 image gets pushed to master at dockerhub

FintasticMan commented 1 year ago

That command is slightly wrong, it should be docker run rather than docker build.

petefoth commented 1 year ago

That command is slightly wrong, it should be docker run rather than docker build.

Corrected - thanks

petefoth commented 1 year ago

The

My apologies. Having looked into this a bit more, you need to use the docker image taggedlineage-20

That image isn't working either: I'm seeing build errors probably caused by missing commits to do with git-lfs

Log output ``` error: Cannot checkout LineageOS/android_external_chromium-webview_prebuilt_x86_64 error: Unable to fully sync the tree error: Checking out local projects failed. Failing repos: external/chromium-webview/prebuilt/arm64 external/chromium-webview/prebuilt/x86 external/chromium-webview/prebuilt/x86_64 ```

I'll keep digging

petefoth commented 1 year ago

I'm closing this issue, as building lineageos-20.0 branch does work, although local builds need to use the latest or master tags. That issue will be tracked (and hopefully fixed) in #441