search

polymorphicshade / Tubular

A fork of NewPipe that implements SponsorBlock and ReturnYouTubeDislike.
GNU General Public License v3.0
1.7k stars 43 forks source link

F-droid availability? #7

Closed Mr-Bajs closed 7 months ago

Mr-Bajs commented 10 months ago

Checklist

Feature description

Great to se the new app. I hope that the new app will be on f-droid. Maybe it's even in the pipeline due to the delay in f-droid, but it would be nice to know if it's planned to be there.

Why do you want this feature?

.

Additional information

No response

polymorphicshade commented 10 months ago

I have no idea how to do this (someone else did this with my old fork).

I'll do this whenever I find the time, unless someone else wants to do this... 😅

IzzySoft commented 10 months ago

@polymorphicshade inclusion was just requested with my repo. I'm fine with adding it, but have a few questions a.o. from my scanner reports, so let me copy those over here for you:

Offending libs:
---------------
* ACRA (/org/acra): Tracking

1 offenders.

Permissions:
------------
* android.permission.INTERNET
* android.permission.WAKE_LOCK
* android.permission.ACCESS_NETWORK_STATE
* android.permission.WRITE_EXTERNAL_STORAGE
* android.permission.SYSTEM_ALERT_WINDOW
* android.permission.FOREGROUND_SERVICE
* android.permission.POST_NOTIFICATIONS
* android.permission.RECEIVE_BOOT_COMPLETED
* org.polymorphicshade.tubular.DYNAMIC_RECEIVER_NOT_EXPORTED_PERMISSION
* android.permission.READ_EXTERNAL_STORAGE*

SigningBlock blobs:
-------------------
0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)

Top-down:

SigningBlock: I guess Android Studio (or IntelliJ IDEA) is used for signing? In that case, the following should be integrated with build.gradle?

android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it contains.

The entire Fastlane tree is untouched from NewPipe. Could it please be updated to match the app? Especially some screenshots would be good to have, and the proper shortdesc/fulldesc.

Thanks for checking (and implementing what's needed :wink:)

polymorphicshade commented 10 months ago

@IzzySoft

IzzySoft commented 10 months ago

ARCA is only used for sending crash reports

The one asking the user to confirm? Fine, then I can remove the Tracking flag (done, effective with the next sync).

the SYSTEM_ALERT_WINDOW permission is only used for the app to draw over other apps

Yeah, that's what the permission SYSTEM_ALERT_WINDOW is described as :wink: I mean, how does the app make use of that? A floating video player?

code related to the WRITE_EXTERNAL_STORAGE permission hasn't changed from vanilla NewPipe, so I'm opting not to change anything related to that

Vanilla NewPipe is not in my repo, so I cannot tell. But with the latest updates to my APK checks those permissions turn up "red" as being sensitive, unless an explanation is added what for they are necessary. I have no issues with READ_EXTERNAL_STORAGE being granted implicitly as long as I know what it's needed for :wink:

I'll work on the the other changes soon

Thanks a lot!

ColorfulRhino commented 10 months ago

Yeah, that's what the permission SYSTEM_ALERT_WINDOW is described as 😉 I mean, how does the app make use of that? A floating video player?

Yes, Newpipe has a floating mini-player.

Btw, I'm sure you @IzzySoft are aware that @polymorphicshade did have another Newpipe fork based on the previous Newpipe version in your repo (org.polymorphicshade.newpipe). Since this one is basically an updated version of this fork and the other one is out-of-development (https://github.com/polymorphicshade/NewPipe), consider removing the old fork from the repo :)

IzzySoft commented 10 months ago

@ColorfulShire thanks! And yes, I'm aware of that – and yes again, it's already "marked for removal" here. I'll just need to prepare the transition (was busy with other things), like adding a comment to the description of the "old one" that it's continued as "new one", to give those currently using that a chance to find there way "over".

polymorphicshade commented 10 months ago

@IzzySoft Oh, sorry... yes the SYSTEM_ALERT_WINDOW permission is used for the floating popup player.

The WRITE_EXTERNAL_STORAGE permission is used for downloading videos. The app explicitly asks for permission, see here

IzzySoft commented 10 months ago

Thanks, added them all to the allow-list here then:

image

As for the signing BLOB, please see above. Easy to get rid of that – which would also remove it from the screenshot. Err, from where the screenshot was taken from of course…

kaoneko commented 10 months ago

like adding a comment to the description of the "old one" that it's continued as "new one", to give those currently using that a chance to find there way "over".

@IzzySoft seems the actual app name went MIA, it currently reads:

Note: the app's repository has been archived 2023-12-29, so there won't be updates anymore. The app is however continued as

Major cliffhanger! 😄

Also, might I suggest putting the note above the regular description instead of below? Most F-Droid clients don't show the full description of an app until you tap a Read more button, which someone who is already using the app and just trying to find out if there are updates isn't likely to do.

IzzySoft commented 10 months ago

@kaoneko thanks for the pointers! Fixed both, should go live within the next half hour.

IzzySoft commented 9 months ago

Hmpf, one more thing:

image

What is this? Someone complained to me:

Showed an alert and tap opened the .apk download in browser. No explanation or nothing.

That's violating the inclusion criteria concerning self-updaters. Those must be strictly opt-in, with all details explained – as such side-loaded updates bypass the scans performed by the repo (F-Droid's and mine). Can you please remove that?

IzzySoft commented 9 months ago

So no word, @polymorphicshade? You're still there?

polymorphicshade commented 8 months ago

So no word, @polymorphicshade? You're still there?

OH! Sorry... I acknowledged your comment with a thumbs-up, not realizing that wasn't enough 😅 Yes next version the auto update prompt will be disabled by default.

IzzySoft commented 8 months ago

np – and thanks for the explicit statement. I must admit I saw 4 thumbs when I first checked, but didn't check whom they came from :see_no_evil:

shuvashish76 commented 8 months ago

https://github.com/TeamNewPipe/NewPipe/discussions/10785#discussioncomment-8950428 | https://github.com/TeamNewPipe/NewPipe/pull/10790

@polymorphicshade This is already fixed in upstream for upcoming vv0.27.0 Here for Tubular, does it connect to official NewPipe domain? Then disabling makes sense but if checks from this repository then you may consider the same behavior as NewPipe.

shuvashish76 commented 8 months ago

Requested at F-Droid: https://gitlab.com/fdroid/rfp/-/issues/2707

shuvashish76 commented 7 months ago

Why closed? Official F-Droid repo inclusion not planned? @polymorphicshade

polymorphicshade commented 7 months ago

Why closed? Official F-Droid repo inclusion not planned? @polymorphicshade

It is on F-Droid under the IzzyOnDroid repo.

shuvashish76 commented 7 months ago

Yes. I assume originally the issue was created for official F-Droid repo. I've requested for packaging at F-Droid. So I'm asking if you intend to publish there or no interest for it.

polymorphicshade commented 7 months ago

@shuvashish76 oh! I understand what you're saying now. I have little knowledge of F-Droid and how to publish things, and I don't have the time to figure it out, so I let IzzyOnDroid deal with it.

shuvashish76 commented 7 months ago

https://gitlab.com/fdroid/rfp/-/issues/2707

@licaon-kter & @eighthave from F-Droid team working on it do have a look on #52 Thanks. See their instructions here.

polymorphicshade commented 7 months ago

@shuvashish76 I recently pushed a fix to master that should fix that Git clone issue they mention (hopefully...)

IzzySoft commented 7 months ago

F-Droid team working on it

And what solution did the two find? That issue is closed with no reference to a solution.

I recently pushed a fix to master that should fix that Git clone issue

Ah, you found it yourself? Glad to read it's solved then! And of course Tubular is welcome to stay in the IzzyOnDroid repo if you want it to.

so I let IzzyOnDroid deal with it.

Deal with what?

licaon-kter commented 7 months ago

And what solution did the two find? That issue is closed with no reference to a solution.

I couldn't even clone the repo locally, since there's some olden corruption or whatever :(

I recently pushed a fix to master that should fix that Git clone issue they mention (hopefully...)

but indeed, now it is fixed :tada:

@polymorphicshade

It is on F-Droid under the...

There's no Izzy repo unless you add it.

this issue was closed because of a misunderstanding I guess?!

the OP asks for the app being built by f-droid.org (like NewPipe is built too)

Will take a look asap, now that we can clone :)

IzzySoft commented 7 months ago

@licaon-kter

I couldn't even clone the repo locally, since there's some olden corruption or whatever :(

Yes, that was described. I was wondering about the solution. Meanwhile I know it: @obfusk has provided a fix, see #60 – as that's implemented, you should now be able to clone.

but indeed, now it is fixed 🎉

Yupp :smiley:

obfusk commented 7 months ago

I couldn't even clone the repo locally, since there's some olden corruption or whatever :(

Being unable to clone was because of a file with wrong mode bits (see #60 for my explanation). There was never any kind of corruption; you mistook a transient network error ("fetch-pack: unexpected disconnect while reading sideband packet") for an fsck failure.

shuvashish76 commented 4 months ago

@polymorphicshade any updates? planned label but issue closed ‽...please reopen the issue.

licaon-kter commented 4 months ago

This draft recipe metadata/org.polymorphicshade.tubular.yml

AntiFeatures:
  NonFreeNet:
    en-US: Depends on Youtube for videos.
Categories:
  - Internet
  - Multimedia
License: GPL-3.0-or-later
SourceCode: https://github.com/polymorphicshade/Tubular

RepoType: git
Repo: https://github.com/polymorphicshade/Tubular.git
Binaries: https://github.com/polymorphicshade/Tubular/releases/download/v%v/tubular_v%v.apk

Builds:
  - versionName: 0.27.2
    versionCode: 999
    commit: c2765316542743c2a4fc02d75ccf44bc22f53878
    subdir: app
    gradle:
      - yes
    rm:
      - doc
      - app/src/test

AllowedAPKSigningKeys: 8ad7025a8c911454e2a7b4515e360c52ca63ec0410a042ff46e9ad05b509e187

AutoUpdateMode: None
UpdateCheckMode: Tags .*[0-9]$

builds, but it's not build reproducible (ref: https://f-droid.org/docs/Inclusion_How-To/#reproducible-builds)

difflog: tub999.log

was the APK in https://github.com/polymorphicshade/Tubular/releases/tag/v0.27.2 built from https://github.com/polymorphicshade/Tubular/commit/c2765316542743c2a4fc02d75ccf44bc22f53878 ? Or maybe from some local dirty tree?

asandikci commented 4 months ago

what is the reason that release.yml could build the apk but f-droid reproducible build cannot?

https://github.com/polymorphicshade/Tubular/blob/c2765316542743c2a4fc02d75ccf44bc22f53878/.github/workflows/release.yml#L23-L55

licaon-kter commented 4 months ago

Not sure I follow.

As said, F-Droid can build it just fine.

But, as you can read in the attached log, there are differences compared to the released tagged apk.

Somebody familiar with the codebase can read and maybe figure out why this happens.

YoshiRulz commented 4 months ago

It looks like a commit updating the Hebrew locale hasn't been pushed.

IzzySoft commented 4 months ago

It's RB at IzzyOnDroid btw. Both last versions, as you can see:

image

Green shield = verified. See here for details on the shields.

licaon-kter commented 4 months ago

OK, so interesting, 0.27.1 has the same issue, same affected files, IW translation

If we dissemble and compare the res/values-iw folders we see that upstream APK files look incomplete, hence the diff log that appears to show that the F-Droid version has the missing bits.

Tubular 0.27.2 IW vs HE from the release page https://github.com/polymorphicshade/Tubular/releases/tag/v0.27.2

tubular_v0.27.2/res/values-iw $ ls -latr
-rw-r--r--   1 fdroid fdroid   907 aug  1 09:57 plurals.xml
-rw-r--r--   1 fdroid fdroid 13510 aug  1 09:57 strings.xml
-rw-r--r--   1 fdroid fdroid   328 aug  1 09:57 arrays.xml
tubular_v0.27.2/res/values-he $ ls -latr
-rw-r--r--   1 fdroid fdroid  3734 aug  1 09:57 plurals.xml
-rw-r--r--   1 fdroid fdroid 70708 aug  1 09:57 strings.xml

comparing this with Newpipe 0.27.2 IW vs HE (this is reproducible in F-Droid)

org.schabi.newpipe_999/res/values-iw $ ls -latr
-rw-r--r--   1 fdroid fdroid  4577 aug  1 15:31 plurals.xml
-rw-r--r--   1 fdroid fdroid 84154 aug  1 15:31 strings.xml
-rw-r--r--   1 fdroid fdroid   328 aug  1 15:31 arrays.xml
org.schabi.newpipe_999/res/values-he $ ls -latr
-rw-r--r--   1 fdroid fdroid  3734 aug  1 15:31 plurals.xml
-rw-r--r--   1 fdroid fdroid 70708 aug  1 15:31 strings.xml

Did you change the way IW strings are generated compared to NewPipe?

compare with my local builds of Tubular 0.27.2 IW 

org.polymorphicshade.tubular_999/res/values-iw $ ls -latr
-rw-r--r--   1 fdroid fdroid  4577 aug  1 09:54 plurals.xml
-rw-r--r--   1 fdroid fdroid 84154 aug  1 09:54 strings.xml
-rw-r--r--   1 fdroid fdroid   328 aug  1 09:54 arrays.xml
org.polymorphicshade.tubular_999/res/values-he $ ls -latr
-rw-r--r--   1 fdroid fdroid  3734 aug  1 09:54 plurals.xml
-rw-r--r--   1 fdroid fdroid 70708 aug  1 09:54 strings.xml

...which seem to match sizes to those of NewPipe

Can somebody speaking IW confirm that the Github/Izzy APK shows all the strings ok and compare with NewPipe?

licaon-kter commented 4 months ago

So looks like the issue is symlinks treatment, looking at the comment above https://github.com/polymorphicshade/Tubular/issues/7#issuecomment-2256728047

and doing that

    prebuild:
      - cd ../..
      - rm -fr org.polymorphicshade.tubular
      - git clone --no-checkout https://github.com/polymorphicshade/Tubular org.polymorphicshade.tubular
      - cd org.polymorphicshade.tubular
      - git config core.symlinks false
      - git checkout --progress --force v0.27.2

and now it's repro

two issues here:

YoshiRulz commented 4 months ago

https://github.com/polymorphicshade/Tubular/blob/master/app/src/main/res/values-iw It's the only symlink in the repo.

obfusk commented 4 months ago

NewPipe isn't built on Windows or with core.symlinks=false so the values-iw -> values-he symlink works as expected.

Back in February, I just did rm app/src/main/res/values-iw to get a reproducible build (with both builds then having broken iw locale of course).

Before #60 was fixed I used the same workaround as the CI still does -- core.symlinks=false -- to get a reproducible build. As the CI still uses that even though it is no longer needed for the git clone failure, it's still in place here too.

My recommendation would be to stop building with core.symlinks=false in CI now that #60 is fixed. That way the iw locale is included properly and no workaround is needed for reproducible builds.

rugk commented 3 months ago

As Youtube breaks the app so often, do you also consider setting up your own F-Droid repository so people can add it?

https://newpipe.net/FAQ/tutorials/install-add-fdroid-repo/

marek22k commented 3 months ago

An alternative to this is IzzyOnDroid (since he doesn't build the apps himself, but takes the APKs from GitHub and co) or Obtainium.

IzzySoft commented 3 months ago

@rugk scroll up a little, it is already available at IzzyOnDroid: https://apt.izzysoft.de/packages/org.polymorphicshade.tubular :wink:

rugk commented 3 months ago

Sure, but I don't want to include that as it contains many unrelated apps and I basically just want an up-to-date NewPipe fork here. Officially from the devs is another plus (although I trust @IzzySoft of course, but you know). Just as the old NewPipe repo I want one simple repo that I can rely on, which does not come with yet totally unrelated hundreds or so apps, which I do not really want in F-Droid (the reasons should not matter, but e.g. one could be because IIRC the Izzysoft apps do not need to be 100% FLOSS etc/did not were built through F-Droid, which is a quality feature, after all). I just want an exception for NewPipe(fork) for obvious reasons (aka it needs to stay more up-to-date due to YouTube breaking stuff at all times :upside_down_face:).

IzzySoft commented 3 months ago

although I trust @IzzySoft of course

thanks :heart_eyes:

did not were built through F-Droid

No, luckily we do not depend on F-Droid builds. We couldn't provide the timely updates then :see_no_evil: and that would be a problem with things like…

it needs to stay more up-to-date

right? So IzzyOnDroid takes the APKs signed by the devs your trust, and ships those – but not before having scanned them left and right, see Ramping up security: additional APK checks are in place with the IzzyOnDroid repo.

which is a quality feature

Ah. You mean it's only quality if a certain person or org builds? Or did you mean something like… Reproducible Builds, special client support and more in our repo maybe? More than 20% of the apps at IoD are covered by RB meanwhile, and that number is still increasing.

And Tubular is one of those RB apps. So one of our verification builders built it from source (in this case it was Fay's, see above). There will probably soon be a verification builder verificator, if you want to call it that – one that runs all the recipes from the existing builders to see if it can reproduce their verification…

So the only point left from your wishlist, @rugk, is to want a bunch of "1 app only repos". Which then lack all the extra checks (and extra trust). Those are pretty good for development of course, to provide test builds to team and testers. But it also puts extra work on the shoulders of the dev team, which then has to maintain that repo as well…

rugk commented 3 months ago

Yeah nothing against your repo, but my point stays, that I don't want my F-Droid cluttered with apps I do not need, which may or may not be (fully) open-source and whatever they may be scanned or nor, it's more about tracking issues (yeah you may also scan that, and we have exodus for that, but really, if I were to install such apps, I could use Google Play/Aurora Store) and that the app comes from the devs itself.

It was mostly inspired by the old Newpipe thing, which was convenient and I thought F-Droid also had https://f-droid.org/repomaker/ for setting up such a repo relatively simple.

je-vv commented 3 months ago

With latest F-Droid, one can choose the repo source for the app, per app. One can as well select which repo to pick by default by the order of the repos in the settings, the repo listed first with the app available will be the on picked by default.

So with current F-Droid there are several options to select packages from multiple repositories, particularly if one prefers packages from one over the other. Some prefer apks signed by the developer (IzzyOnDroid does this for us), some prefer to have apks from official F-Droid repo if possible, and some other would prefer the most up to date packages. For the latest, IzsyOnDroid is pretty close to the latest release on the developer releases, so that's a great option to grab latest, and still using F-Droid instead of grabbing the apk directly from the developer releases, and if not wanting the rest of the packages, keep IzzyOnDroid after the official F-Droid repo in the list of repos, or even place it as the last one, and when searching for apps, make sure to select them from other repos...

I really see no need for additional repos, if unofficial ones like IzzyOnDroid grab the apks pretty close to their developers release.

I still like for most apps to have the option between the official F-Droid repo, which might not be as up to date, but is built and signed by the F-Droid folks, and the IzzyOnDroid one, or other alternatives if available.

BTW, newPipe is a good example of a multi-repo app. When you search for it (I don't have it installed, since I prefer Tubular), and you select it, you'll see in the repositories field: F-Droid (preferred). That's because I have the F-Droid official repo before the newPipe repo (for some reason I haven't deleted that repo yet). But if I wanted to change it for the newPipe repo, I can repo source of the apk to newPipe upstream and it'll be marked as the preferred one at once. So that's the per app way. I'm using F-Droid version 1.21.0-alpha0 BTW. So there you go...

The only down side of multiple repos as of now, is that the more amount of packages from the repo, the longer it takes for the F-Droid app to update the metadata for each repo, jeje.

obfusk commented 3 months ago

Yeah nothing against your repo, but my point stays, that I don't want my F-Droid cluttered with apps I do not need, which may or may not be (fully) open-source and whatever they may be scanned or nor, it's more about tracking issues (yeah you may also scan that, and we have exodus for that, but really, if I were to install such apps, I could use Google Play/Aurora Store) and that the app comes from the devs itself.

Google Play/Aurora Store doesn't allow developers to sign their own apps any more. And they certainly don't have a way to check the provided APK matches the source code. Or have any kind of checks for non-free or tracking libraries like IzzyOnDroid does.

With Reproducible Builds, both IzzyOnDroid and F-Droid provide the exact same APK the developer does, verified to correspond to the published source code, with additional checks (IzzyOnDroid has several checks F-Droid doesn't, and vice versa). And IzzyOnDroid updates faster, which as you yourself pointed out is very useful for apps like Tubular :)

Edit: and any non-free components in IzzyOnDroid are clearly labelled with anti-features.