dkrivoruchko / ScreenStream

ScreenStream Android App
https://screenstream.io
MIT License
1.56k stars 319 forks source link

F-Droid can't build #227

Closed licaon-kter closed 5 months ago

licaon-kter commented 11 months ago

4.0.1 is live I see, plenty of changes, webrtc stuff

But, updated recipe:

  - versionName: 4.0.1
    versionCode: 40001
    commit: 3ee6ae213ac6408da198d269818035fda0145f46
    subdir: app
    sudo:
      - apt-get update
      - apt-get install -y openjdk-17-jdk-headless
      - update-java-alternatives -a
      - chmod -R 777 /opt/android-sdk
    gradle:
      - firebasefree
    srclibs:
      - Material-Dialogs@3.3.0
    rm:
      - app/libs/bottomsheets-release.aar
    prebuild:
      - sed -i -e '/gms/d; /firebase/d' ../build.gradle.kts
      - sed -i -e '/google-services/d; /id("com.google.firebase.crashlytics")/d; /CrashlyticsExtension/,+2d;
        /SIGNALING_SERVER/d; /CLOUD_PROJECT/d' build.gradle.kts
      - sed -i -e '/play/d' ../webrtc/build.gradle.kts
      - pushd $$Material-Dialogs$$
      - sed -i -e '241d' bottomsheets/src/main/java/com/afollestad/materialdialogs/bottomsheets/BottomSheet.kt
    build:
      - pushd $$Material-Dialogs$$
      - JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 gradle bottomsheets:assembleRelease
      - popd
      - cp $$Material-Dialogs$$/bottomsheets/build/outputs/aar/bottomsheets-release.aar
        libs/

...fails with

> Task :webrtc:compileReleaseKotlin
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/StandardIntegrityManagerWrapper.kt:5:12 Unresolved reference: google
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/StandardIntegrityManagerWrapper.kt:6:12 Unresolved reference: google
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/StandardIntegrityManagerWrapper.kt:9:27 Unresolved reference: tasks
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/StandardIntegrityManagerWrapper.kt:13:52 Unresolved reference: StandardIntegrityManager
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/StandardIntegrityManagerWrapper.kt:14:50 Unresolved reference: StandardIntegrityManager
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/StandardIntegrityManagerWrapper.kt:18:36 Unresolved reference: IntegrityManagerFactory
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/StandardIntegrityManagerWrapper.kt:25:17 Unresolved reference: StandardIntegrityManager
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/StandardIntegrityManagerWrapper.kt:36:13 Unresolved reference: StandardIntegrityManager
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/StandardIntegrityManagerWrapper.kt:42:39 Unresolved reference: it
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:4:12 Unresolved reference: google
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:5:12 Unresolved reference: google
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:31:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:32:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:33:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:34:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:35:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:36:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:37:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:42:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:43:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:44:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:49:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:50:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:51:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:56:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:57:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:58:13 Unresolved reference: StandardIntegrityErrorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:61:21 Unresolved reference: StandardIntegrityException
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:124:26 Unresolved reference: StandardIntegrityException
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:124:62 Unresolved reference: errorCode
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:139:26 Unresolved reference: StandardIntegrityException
e: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/src/main/kotlin/info/dvkr/screenstream/webrtc/internal/PlayIntegrity.kt:139:62 Unresolved reference: errorCode

> Task :webrtc:compileReleaseKotlin FAILED
> Task :app:collectFirebaseFreeReleaseDependencies
> Task :app:processFirebaseFreeReleaseManifestForPackage
> Task :app:bundleFirebaseFreeReleaseResources
> Task :mjpeg:compileReleaseKotlin

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':webrtc:compileReleaseKotlin'.
> A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
   > Compilation error. See log for more details

...because your webrtc component lib is dependent of proprietary Google lib

dkrivoruchko commented 11 months ago

Hi. 4.x is not yet ready for full rollout. It's in beta test and code is in separate brunch (not “master”) Once it will be ready – it will be pushed to master. Not sure if F-Droid version will support WebRTC at all.

licaon-kter commented 11 months ago

4.x is not yet ready for full rollout. It's in beta test and code is in separate brunch (not “master”)

:shrug: https://github.com/dkrivoruchko/ScreenStream/releases/tag/4.0.1

Not sure if F-Droid version will support WebRTC at all.

We have plenty of webrtc apps that don't depend on Google libs, how come?

dkrivoruchko commented 11 months ago

It's just a tag. It's not a release: https://github.com/dkrivoruchko/ScreenStream/releases I removed the tag.

To protect signaling server, Google Play Integrity lib is used. If it's ok to use it for F-Droid builds, then WebRTC module will be eventually available for F-Droid builds too.

licaon-kter commented 11 months ago

Ah, signaling server? You mean we'd need to add NonFreeNet to your app too :(

But yeah, no Google proprietary libs can be used... :-1:

Then what's the path forward? A webrtc-less fdroid flavor or no more updates?

It's just a tag. It's not a release:

F-Droid is instructed to pull tags as git has no concept of "release", this is just a Github/Gitlab webui thing.

dkrivoruchko commented 11 months ago

Not sure if I got correctly about NonFreeNet.

This app to run WebRTC module use signaling server also written by me https://github.com/dkrivoruchko/ScreenStreamWeb It's hosted on Heroku (non-free) and published via Cloudflare (non-free) along with web client. To protect this singling server, Android app uses Google Play Integrity lib and there is no alternative. At least known to me.

Cloudflare Turnstile is used for web clients.

If there is no way to make WebRTC module work within F-Droid requirements, then 4.x builds for F-Droid will be only with current MJPEG module only once it's ready.

licaon-kter commented 11 months ago

then 4.x builds for F-Droid will be only with current MJPEG module only once it's ready.

:+1:

licaon-kter commented 6 months ago

after the switch to kotlin we need to update the recipe, disabled for now https://gitlab.com/fdroid/fdroiddata/-/commit/9aefce583ed5489e30bb40a7382a7166f54da950

I started working on it locally, brb

licaon-kter commented 5 months ago

This recipe:

  - versionName: 4.0.30
    versionCode: 40030
    commit: 079ff11fe69836328426d179beddfa1449f1c1ea
    subdir: app
    sudo:
      - chmod -R 777 /opt/android-sdk
      - apt-get update
      - apt-get install -y openjdk-17-jdk-headless
      - update-java-alternatives -a
    gradle:
      - firebasefree
    srclibs:
      - Material-Dialogs@3.3.0
    rm:
      - app/libs/bottomsheets-release.aar
      - mjpeg/libs/bottomsheets-release.aar
    prebuild:
      - sed -i -e '/google.services/d' -e '/firebase.crashlytics/d' -e '/googleServices/d'
        -e '/firebaseCrashlytics/d' ../build.gradle.kts
      - sed -i -e '/googleServices/d' -e '/firebaseCrashlytics/d' -e '/buildtools.gradle.CrashlyticsExtension/,+2d'
        -e '/firebaseImplementation/d' build.gradle.kts ../gradle/libs.versions.toml
      - sed -i -e '/google-/d' -e '/play-/d' -e '/firebase-/d' ../gradle/libs.versions.toml
      - sed -i -e '/:webrtc/d' ../settings.gradle.kts
    build:
      - pushd $$Material-Dialogs$$
      - export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
      - gradle bottomsheets:assembleRelease
      - popd
      - cp $$Material-Dialogs$$/bottomsheets/build/outputs/aar/bottomsheets-release.aar
        libs/
      - cp $$Material-Dialogs$$/bottomsheets/build/outputs/aar/bottomsheets-release.aar
        ../mjpeg/libs/

fails: info.dvkr.screenstream_40030.log.gz

What am I missing ?

dkrivoruchko commented 5 months ago

Oh, there was a problem in code. Sorry. Fixed. Try now. The 4.0.30 tag now point to fixed version 4.0.30 https://github.com/dkrivoruchko/ScreenStream/commit/b532f81b3c870f84c7ba0dafb9c7b92ab7c51053

licaon-kter commented 5 months ago

nope :( info.dvkr.screenstream_40030.log.gz

dkrivoruchko commented 5 months ago

I cannot find any issues on my side. It builds successfully.

Maybe the is something with prebuild script as log show: /etc/bash.bashrc: line 7: PS1: unbound variable Not sure what this means.

2024-01-10 09:26:04,447 DEBUG: > bash -e -u -o pipefail -x -c -- sed -i -e '/google.services/d' -e '/firebase.crashlytics/d' -e '/googleServices/d' -e '/firebaseCrashlytics/d' ../build.gradle.kts; sed -i -e '/googleServices/d' -e '/firebaseCrashlytics/d' -e '/buildtools.gradle.CrashlyticsExtension/,+2d' -e '/firebaseImplementation/d' build.gradle.kts ../gradle/libs.versions.toml; sed -i -e '/google-/d' -e '/play-/d' -e '/firebase-/d' ../gradle/libs.versions.toml; sed -i -e '/:webrtc/d' ../settings.gradle.kts
/etc/bash.bashrc: line 7: PS1: unbound variable
+ sed -i -e /google.services/d -e /firebase.crashlytics/d -e /googleServices/d -e /firebaseCrashlytics/d ../build.gradle.kts
+ sed -i -e /googleServices/d -e /firebaseCrashlytics/d -e /buildtools.gradle.CrashlyticsExtension/,+2d -e /firebaseImplementation/d build.gradle.kts ../gradle/libs.versions.toml
+ sed -i -e /google-/d -e /play-/d -e /firebase-/d ../gradle/libs.versions.toml
+ sed -i -e /:webrtc/d ../settings.gradle.kts
licaon-kter commented 5 months ago

You can ignore the PS1 thing.

The sed is there to cleanup non-FOSS stuff, is that an issue?

It builds for you AFTER you clean up like the recipe above?

dkrivoruchko commented 5 months ago

I get the code from git, run all sed commands, and it dosn't sync in Android Studio Next I removed firebase config sed -i -e '/create("firebase")/,+5d' build.gradle.kts And it builds successfully. Other difference I see is that I use embedded pre-build bottomsheets-release.aar as original one has unfixed bug.

licaon-kter commented 5 months ago

We can 't use prebuilt one... is the unfixed bug related to the error?

licaon-kter commented 5 months ago

Next I removed firebase config sed -i -e '/create("firebase")/,+5d' build.gradle.kts

This is not needed since we call the flavor directly

I get the code from git, run all sed commands, and it dosn't sync in Android Studio

you've run all these is the app folder yes?

Other difference I see is that I use embedded pre-build bottomsheets-release.aar as original one has unfixed bug.

tested with it too, same error

dkrivoruchko commented 5 months ago

you've run all these is the app folder yes?

Yes.

Will try on fresh laptop and come back.

dkrivoruchko commented 5 months ago

Well, I cannot reproduce the issue. I take fresh VM, downloaded Java, SDK, App code, run all sed and run gradlew assembleFirebaseFreeRelease without any issues.

licaon-kter commented 5 months ago

can you run gradle clean after the sed ?

dkrivoruchko commented 5 months ago

yes

licaon-kter commented 5 months ago

Can you give me a log of that run ?

dkrivoruchko commented 5 months ago

Not sure what log you want. A console log from my test:

C:\Users\WDAGUtilityAccount\Desktop\ScreenStream-4.0.30>gradlew clean

BUILD SUCCESSFUL in 3s
3 actionable tasks: 3 up-to-date
C:\Users\WDAGUtilityAccount\Desktop\ScreenStream-4.0.30>gradlew assembleFirebaseFreeRelease

> Task :mjpeg:compileReleaseKotlin
w: file:///C:/Users/WDAGUtilityAccount/Desktop/ScreenStream-4.0.30/mjpeg/src/main/java/info/dvkr/screenstream/mjpeg/MjpegStreamingModule.kt:56:23 This is a delicate API and its use requires care. Make sure you fully read and understand documentation of the declaration that is marked as a delicate API.

> Task :app:kspFirebaseFreeReleaseKotlin
w: [ksp] No module found for 'info.dvkr.screenstream.notification.NotificationHelperImpl'. Definition is added to 'defaultModule'
w: [ksp] No module found for 'info.dvkr.screenstream.notification.NotificationHelperImpl'. Definition is added to 'defaultModule'
BUILD SUCCESSFUL in 4m 11s
131 actionable tasks: 11 executed, 120 up-to-date
C:\Users\WDAGUtilityAccount\Desktop\ScreenStream-4.0.30>
licaon-kter commented 5 months ago

Can you run with verbose or something? my log is 80k ...

licaon-kter commented 5 months ago

I've run directly in the VM, without F-Droid

git reset
git clean -fdx
cd app
sed -i -e '/google.services/d' -e '/firebase.crashlytics/d' -e '/googleServices/d' -e '/firebaseCrashlytics/d' ../build.gradle.kts
sed -i -e '/googleServices/d' -e '/firebaseCrashlytics/d' -e '/buildtools.gradle.CrashlyticsExtension/,+2d' -e '/firebaseImplementation/d' build.gradle.kts ../gradle/libs.versions.toml
sed -i -e '/google-/d' -e '/play-/d' -e '/firebase-/d' ../gradle/libs.versions.toml
sed -i -e '/:webrtc/d' ../settings.gradle.kts
gradle clean
gradle assembleFirebaseFreeRelease

and fails the same

dkrivoruchko commented 5 months ago

Ok, will try on clean Ubuntu VM.

licaon-kter commented 5 months ago

@linsui, also tried as simple as possible

git reset --hard
git clean -fdx
cd app
sed -i -e '/:webrtc/d' ../settings.gradle.kts # else it complains about missing signaling servers
sed -i -e '/webrtc/d' build.gradle.kts
gradle assembleFirebaseFreeRelease

...same fail

ok, let me try firebase flavor but first I need to

cd webrtc
sed -i 's/localProps.getProperty("SIGNALING_SERVER_DEV")/"example.com"/g' build.gradle.kts
sed -i 's/localProps.getProperty("SIGNALING_SERVER_RELEASE")/"example.com"/g' build.gradle.kts
sed -i 's/localProps.getProperty("CLOUD_PROJECT_NUMBER_DEV")/"1"/g' build.gradle.kts
sed -i 's/localProps.getProperty("CLOUD_PROJECT_NUMBER_RELEASE")/"1"/g' build.gradle.k

...or else it fails then I

gradle assembleFirebaseRelease

to

...
> Configure project :webrtc
w: file:///home/vagrant/build/info.dvkr.screenstream/webrtc/build.gradle.kts:26:9: Variable 'localProps' is never used

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':app:lintVitalReportFirebaseRelease'.
> Could not create task ':app:lintVitalAnalyzeFirebaseRelease'.
   > Cannot invoke "Object.toString()" because "value" is null

what next?

dkrivoruchko commented 5 months ago

Ok. please try new fix 4.0.30 https://github.com/dkrivoruchko/ScreenStream/commit/f9f8f748615fde83716ac4eefde6a500211be940

Important: gradle assembleFirebasefreeRelease not gradle assembleFirebaseFreeRelease

gradle clean will not work unless

            manifestPlaceholders += mapOf("adMobPubId" to localProps.getProperty("ad.pubId"))
            buildConfigField("String", "AD_UNIT_ID_A", localProps.getProperty("ad.unitIdA", ""))
            buildConfigField("String", "AD_UNIT_ID_B", localProps.getProperty("ad.unitIdB", ""))
            buildConfigField("String", "AD_UNIT_ID_C", localProps.getProperty("ad.unitIdC", ""))

is removed also

licaon-kter commented 5 months ago

Important: gradle assembleFirebasefreeRelease

Always has been

not gradle assembleFirebaseFreeRelease

has not made a difference anyway :shrug:

retrying now...

licaon-kter commented 5 months ago

That builds fine, thanks https://gitlab.com/fdroid/fdroiddata/-/commit/a0b4cf6e43dca03c02ac49672e4aee11e902ebd9