aclap-dev / vdhcoapp

Companion application for Video DownloadHelper browser add-on
GNU General Public License v2.0
1.72k stars 280 forks source link

Add support for system ffmpeg on Linux #188

Closed kuanyui closed 7 months ago

kuanyui commented 9 months ago

For the users who don't want to have pre-built ffmpeg binaries (e.g. for saving disk storage) :

Currently I've tested on Linux, and confirmed that build.sh can package vdhcoapp without shipping ffmpeg binaries, and still can correctly download video from YouTube. However Windows and MacOS are not tested because I have no systems can test.

paulrouget commented 9 months ago

Hi! This is amazing work @kuanyui ! Thank you so much for taking the time to look into this.

Here are some comments:

Please make findexecutable more ES6 friendly:

    return envPath.replace(/["]+/g, '').split(path.delimiter).map(function (chunk) {
        return envExt.split(path.delimiter).map(function (ext) {
            return path.join(chunk, bin + ext);
        });
    }).reduce(function (a, b) {
        return a.concat(b);
    });

could be (didn't test):

return envPath.replace(/["]+/g, '')
  .split(path.delimiter)
  .map((chunk) => path.resolve(chunk, bin))
  .find(fileExistsSync);   // Make fileExistsSync inline maybe? As it doesn't need to be its own function.

with all this rewrite, I don't think we need to worry about the GPL thing.

Let me know if I can assist for any of these comments.

Thank you.

kuanyui commented 9 months ago
  • please make this a Linux only thing for now. I'd argue that only Linux user need that, and it's easier to implement for Linux systems.
  • Only use PATH, PATHEXT won't be nessary

For these two points, just for curiosity, because I had ever used Windows + Cygwin / MinGW and macOS + homebrew 7 years ago, but I don't sure if the runtime of vdhcoapp can get the executable installed via Cygwin / MinGW / homebrew and maybe there are already some magical mechanism that can let user to do more magics here (Sorry, I'm not familiar with Win/Mac). Therefore I decided not to restrict user from use this flag. Because I can reasonably assume that those who built vdhcoapp by themselves with special options are advanced users and should know what exactly they are doing. (Is there any normal user install ffmpeg by themself?)

Surely, if you still think this flag should be applied on Linux only, I'm going to modify them :)

paulrouget commented 9 months ago

Thanks for asking.

You're correct: it's possible to have an install of ffmpeg for Windows and Mac (through different means).

But, supporting Mac and Windows means:

We could also have the option to NOT publish the no-ffmpeg version for Mac and Windows, but then this will most likely never be tested then. I'd rather have all ou builds published.

And also, I don't believe this feature interest Mac and Windows users.

So let's just do it for Linux :)

kuanyui commented 9 months ago

Encountered two questions...

How linux-i686 is built & published?

Even --all will not build linux-i686? I don't understand how linux-i686 is generated in your publish workflow at all...

Conflicted output directory

Currently, the output Tarball & *.deb will be placed in the same folder ($dist_dir_name/$target_os/$target_arch) and the later one will rm -rf the previously built files when using build.sh --all...

Screenshot_20231214_195918 Screenshot_20231214_200121

At last... I'm considering 2 ways:

  1. When target_os == "linux" , exclude *.deb and *.tar.bz2 when rm -rf $target_dist_dir. (And maybe clear everything when initializing --all)

    • If you plan to regularly provide *-no-ffmpeg packages for linux, I guess this way would be better (and easier to write build.sh).
  2. add one more layer to dist folder ($dist_dir_name/$target_os/$target_arch/$variety) -- but looks ugly and complex.

paulrouget commented 9 months ago

Encountered two questions...

How linux-i686 is built & published?

Even --all will not build linux-i686? I don't understand how linux-i686 is generated in your publish workflow at all...

This is the only target that can't be cross-compiled via --all. So I build it manually on a Linux 32bits machine. So you need a 32bits machine to test it. But don't bother, I can do the builds myself!

Conflicted output directory

At last... I'm considering 2 ways:

I agree with you, solution 1 might be better.

  • If you plan to regularly provide *-no-ffmpeg packages for linux

Yes I'd like to.

  1. When target_os == "linux" , exclude *.deb and *.tar.bz2 when rm -rf $target_dist_dir. (And maybe clear everything when initializing --all)

Maybe always build the no-ffmpeg packages along the normal builds.

Basically, building with --target linux-xxx would produce 4 files: with ffmpeg & without ffmpeg, tab.bz2 and deb.

Does that make sense?

kuanyui commented 9 months ago
  1. When target_os == "linux" , exclude *.deb and *.tar.bz2 when rm -rf $target_dist_dir. (And maybe clear everything when initializing --all)

Maybe always build the no-ffmpeg packages along the normal builds.

Basically, building with --target linux-xxx would produce 4 files: with ffmpeg & without ffmpeg, tab.bz2 and deb.

Does that make sense?

Yeah... I was also considering like that yesterday but I supposed that you may think that is too aggressive therefore I removed it from the proposal:

Screenshot_20231215_163021 Screenshot_20231215_163322

paulrouget commented 9 months ago

I think that was a good idea :) Go ahead, and let me know if you have any questions.

kuanyui commented 9 months ago
  1. You mean, remove --no-ship-ffmpeg directly?
  2. And last but not least, if I didn't misunderstand, according to the naming convention of linux packages, it seems that the -no-ffmpeg suffix should be appended to very last of package name itself instead of after architecture? Ex: vdhcoapp-no-ffmpeg-2.0.9-linux-x86_64.deb
paulrouget commented 9 months ago
  1. You mean, remove --no-ship-ffmpeg directly?

Yes. Let's build it along the normal builds.

  1. And last but not least, if I didn't misunderstand, according to the naming convention of linux packages, it seems that the -no-ffmpeg suffix should be appended to very last of package name itself instead of after architecture?

I believe that's correct, yes.

Ex: vdhcoapp-no-ffmpeg-2.0.9-linux-x86_64.deb

Yes.

Make sure the Debian package name also have a different name (iirc, it's in the control file generated with yq).

kuanyui commented 9 months ago

Now the PR becomes to this:

  1. Refactored findExecutableFullPath()
  2. Tarballs have the same extracted folder name. (I think we should not use different folder name for those who install vdhcoapp via such method...)
  3. *.deb:
    • two packages have identical installed binary name. (/opt/vdhcoapp/vdhcoapp)
    • two packages have different package name in DEBIAN's control.
    • two packages are mutually conflicts. vvvv
paulrouget commented 9 months ago

This looks great! Almost there.

Can you also add a test here: https://github.com/aclap-dev/vdhcoapp/blob/master/.github/workflows/test.yaml#L16

You'll need to install ffmpeg of course.

kuanyui commented 9 months ago

The function are refactored, and tests are passed now.

Screenshot_20231226_184604

paulrouget commented 9 months ago

Thanks @kuanyui ! I'll look at this as soon as possible :)

paulrouget commented 9 months ago

Thank you!

I've added some comment.

One note: this won't work on the latest master. You have 2 options:

kuanyui commented 8 months ago

The PR branch has been rebased to upstream's master.

kuanyui commented 7 months ago

Any progression? I noticed lots of new commits since my last time to resolves this PR. Now I've fixed conflicts again. @paulrouget Screenshot_20240206_175908

https://github.com/kuanyui/vdhcoapp/actions/runs/7798508611/job/21267333344 Screenshot_20240206_185608

kuanyui commented 7 months ago

@paulrouget I don't sure it's because this Github's bug (?) so that this PR is still not merged.

Screenshot_20240206_224706

My Actions: https://github.com/kuanyui/vdhcoapp/actions

Screenshot_20240206_224755

After reading this I still donno why: https://manumagalhaes.medium.com/github-actions-bypassing-expected-waiting-for-status-to-be-reported-4712032ef129

Maybe you need to fetch this PR and merge locally. It should be able to be fast-forward without conflict.

paulrouget commented 7 months ago

Sorry for the late reply! I'll look into this tomorrow :)

paulrouget commented 7 months ago

Thank you! Tests have succeeded.

I've made some tweaks (just some wording), and uploaded the noffmpeg builds and made an announcement: https://github.com/aclap-dev/video-downloadhelper/discussions/47