search

aclap-dev / vdhcoapp

Companion application for Video DownloadHelper browser add-on
GNU General Public License v2.0
1.72k stars 280 forks source link

Flatpak registration doesn't alway work #160

Closed CHJ85 closed 11 months ago

CHJ85 commented 1 year ago

Hi there. vdhcoapp is not recognized by Firefox build 117.0, on Manjaro Linux. I installed it first through the official website. When that didn't work, I used the AUR build. Both are essentially the same package with the same files, same installation location and so on. The installation directory is the same as always "/opt/net.downloadhelper.coapp". At first I thought it was a file permission issue, so I changed the owner of the directory and files from root to my local user. This did not do the trick either. Sometimes it does when permissions are messed up for whatever reason.

paulrouget commented 1 year ago

Can you point me to the AUR build? I'll double check that it's correct.

If you run the binary that way, what happens:

/opt/net.downloadhelper.coapp/net.downloadhelper.coapp-linux-XX

?

In theory it should just hang and do nothing. Can you confirm?

Then, try this:

/opt/net.downloadhelper.coapp/net.downloadhelper.coapp-linux-XX install --user

And check if the extension works.

Thank you.

CHJ85 commented 1 year ago

The first command just hung like you said, as all it does is open the program. The 2nd command says: VdhCoApp: VdhCoApp is ready to be used The program itself seem to be working just fine.

Here's the AUR package: https://aur.archlinux.org/packages/vdhcoapp

It's the regular Firefox install from the Arch repo: https://archlinux.org/packages/extra/x86_64/firefox/

paulrouget commented 1 year ago

Does it create a file .mozilla/native-messaging-hosts/net.downloadhelper.coapp.json ? And you open that file, does the path correlate with the path of the coapp binary?

CHJ85 commented 1 year ago

Yes and yes. Here's the content of the file:```

{ "name": "net.downloadhelper.coapp", "description": "Video DownloadHelper companion app", "path": "/opt/net.downloadhelper.coapp/bin/net.downloadhelper.coapp-linux-64", "type": "stdio", "allowed_extensions": [ "weh-native-test@downloadhelper.net", "{b9db16a4-6edc-47ec-a1f4-b86292ed211d}" ] }



However, I'm still getting this screen whenever I try to download a video:
https://i.imgur.com/49j0sGo.png
paulrouget commented 1 year ago

If you go here:

https://github.com/aclap-dev/vdhcoapp/blob/master/assets/instruction1.png

Does it say the companion app is not installed?

CHJ85 commented 1 year ago

Idk what you want me to look for there. It's an image screenshot that says "No media to process in the current tab".

paulrouget commented 1 year ago
paulrouget commented 1 year ago

Also, can you give it a try with a Firefox official build?

Download https://download.mozilla.org/?product=firefox-latest-ssl&os=linux64&lang=en-US untar, and run. See if this changes anything.

Thanks for helping out.

CHJ85 commented 1 year ago

Oh, sorry about that. 😀 It still says "Companion App not installed"

mokshasoul commented 1 year ago

Hey I had a very similar issue on Arch*

The native messaging JSON contained the wrong add-on ID that was allowed, thus actually even if installed it wasn't allowing Firefox to communicate with the native host add-in

I fixed it by adding the addons actual ID (so replacing "{b9db16a4-6edc-47ec-a1f4-b86292ed211d}" with whatver ID you get if you inspect the addon in FF)

Took me months to solve ;D

paulrouget commented 1 year ago

@mokshasoul how is your ID not {b9db16a4-6edc-47ec-a1f4-b86292ed211d} ? Where are you getting your addon from? Are you sure you're getting the official addon?

paulrouget commented 1 year ago

@CHJ85 where are you getting the download helper extension from?

paulrouget commented 1 year ago

@mokshasoul what's the ID of the extension?

mokshasoul commented 1 year ago

I'm going to have a look in the vening its on my desktop, maybe I'm missremembering something but it was def. an issue with the native-messaging file

CHJ85 commented 1 year ago

I downloaded the addon from the mozilla addon website.

CHJ85 commented 1 year ago

I solved it. Not sure what the problem was. But now it says "Companion App installed" after I uninstalled the current one and installed the vdhcoapp-bin version from the AUR instead of the standard one. Guess there's a difference between the 2 packages.. Thanks for all your help.

paulrouget commented 1 year ago

@mokshasoul thanks - just let me know what the json file look like on your system.

paulrouget commented 1 year ago

I solved it. Not sure what the problem was. But now it says "Companion App installed" after I uninstalled the current one and installed the vdhcoapp-bin version from the AUR instead of the standard one. Guess there's a difference between the 2 packages.. Thanks for all your help.

Understood. I think the issue was the binary location on aur-based packages.

mokshasoul commented 1 year ago

This is what I ended up with btw: {0274c307-d9af-4970-96d5-964d6bf5d123}

boydthomson commented 1 year ago

I'm having the same issue with the FireFox snap 117.0.1 on Ubuntu 22.04.3

I did /opt/net.downloadhelper.coapp/bin/net.downloadhelper.coapp-linux-64 install --user VdhCoApp: VdhCoApp is ready to be used

but FF still says the companion app isn't installed. I tried on Chromium also with similar results.

CHJ85 commented 1 year ago

@boydthomson In Ubuntu, Snap packages are stored in a different location than traditional APT-based packages. Snap packages are typically stored in the /snap directory. Each Snap package will have its own subdirectory under /snap where it is mounted.

For Firefox installed via Snap, you should find the relevant files under /snap/firefox/ So, in order for vdhcoapp to work, I believe you have to place net.downloadhelper.coapp to the snap directory. Mind you, I do not use snaps, so I could be wrong on the exact location.

paulrouget commented 1 year ago

@boydthomson Hi! I think this has been fixed by the Firefox team recently. It should work "out of the box". Do you mind replying here: https://github.com/aclap-dev/vdhcoapp/issues/168 ? I'd like to investigate.

paulrouget commented 1 year ago

@mokshasoul Hey, do you mind telling me where you got your addon from? This ID is unknown to us. I think you might have installed an addon from the wrong place (might be unsafe).

kkovaletp commented 11 months ago

I have the same issue on Fedora KDE 38 with Firefox 118.0.2 and Chrome 118.0.5993.88, as a Flatpak, addon 8.0.0.6, vdhcoapp 2.0.4 from the https://www.downloadhelper.net/install-coapp-v2.

I did next steps:

Nothing has helped. Please let me know how to make it work.

UPD: this hint helped me: https://github.com/aclap-dev/vdhcoapp/issues/137#issuecomment-1544985068 however, I assume that it is temporary solution, which will work until the next update of the Firefox flatpak, so I still want to find a more straight-forward and permanent solution. While searching for some workarounds, I saw a proposal to create a flatpak extension with the coapp inside. To me it looks like a good idea, as it follows the flatpak's way to sandbox and extend apps instead of breaking the sandbox by additional communication channel to the host level.

paulrouget commented 11 months ago

@kkovaletp hmm, the fact you have to use the linked workaround is surprising to me. This is not supposed to be necessary anymore in Firefox. You're not the first one to run into that issue, and I'd like to get to the bottom of it. Would be great if you could help out.

kkovaletp commented 11 months ago

@paulrouget it is a surprise for me that there is a requirement \ dependency to the FF privacy settings: I didn't see that in the installation instructions. I think that providing the complete list of all the requirements as a prerequisite step of the installation guide would really help in many cases.

paulrouget commented 11 months ago

@kkovaletp

it is a surprise for me that there is a requirement \ dependency to the FF privacy settings

There is not. Firefox has that option, but it's off by default.

an exception occurred during the verification of the connection

What do you mean?

I doubled checked here, and it seems to work as expected. I'd like to understand how your setup differs from mine.

Can you confirm these 3 json have exactly the same content and are all pointing to /usr/local/vdhcoapp/vdhcoapp

Also, can you check in Firefox menu > Help > About Firefox that it says "mozilla-flatpak - 1.0"?

kkovaletp commented 11 months ago

@paulrouget

An exception occurred during the verification of the connection

VDH settings > General > Companion App not installed > Checking companion app returned: An unexpected error occurred

3 json have exactly the same content and are all pointing to /usr/local/vdhcoapp/vdhcoapp

yes, used KDiff to make sure)

Firefox menu > Help > About Firefox that it says "mozilla-flatpak - 1.0"

yes

paulrouget commented 11 months ago

What does this file says:

cat ~/.local/share/flatpak/overrides/org.mozilla.firefox
kkovaletp commented 11 months ago

@paulrouget

cat ~/.local/share/flatpak/overrides/org.mozilla.firefox

[Context]
devices=dri;
filesystems=/home/<user>/.mozilla;/usr/local/vdhcoapp:ro;
paulrouget commented 11 months ago

Everything seems correct. Thanks again for helping out!

Can you try 2 things for me:

Screenshot 2023-10-24 at 09 24 20 
# Quit Firefox (make sure no process is left hanging)
# and in that specific order:
mv ~/.mozilla ~/_mozilla_backup
mv ~/.var/app/org.mozilla.firefox ~/org_mozilla_firefox_backup
rm ~/.local/share/flatpak/overrides/org.mozilla.firefox
flatpak run org.mozilla.firefox
# Wait until Firefox is up and running
/usr/local/vdhcoapp/vdhcoapp install
# Install VDH from here: https://addons.mozilla.org/en-US/firefox/addon/video-downloadhelper/
# Check if coapp is recognized
# Quit Firefox (make sure no process is left hanging)
# Restore the backup files:
rm -rf ~/.mozilla && mv ~/_mozilla_backup ~/.mozilla 
rm -rf ~/.var/app/org.mozilla.firefox && mv ~/org_mozilla_firefox_backup ~/.var/app/org.mozilla.firefox
kkovaletp commented 11 months ago

@paulrouget

  • the addon permissions

I have all the same

  • trying with a clean profile & clean pak dir

I did all as described and got the same result: co-app not found, unexpected error in the add-on settings. BTW, you forgot to backup the overrides, I did it as well BTW 2: when I press the setup co-app button (next to the find co-app one in the add-on settings), the v1 page opens with the 1.6.3 version. Why isn't it updated to point to the v2 page with the latest co-app version?

I tried to analyze what else could prevent the browser from communicating with the co-app on the host. I'm not sure how flatpak apps communicate with the host, but I have a firewall and SELinux enabled. Don't I need to add some configuration there?

paulrouget commented 11 months ago

Why isn't it updated to point to the v2 page with the latest co-app version?

Because V2 is still in testing phase. It will open the V2 download page if you use the beta version of the addon: https://www.downloadhelper.net/firefox/betas

SELinux

Oh… that might be it.

Maybe this is relevant: https://discussion.fedoraproject.org/t/does-selinux-confine-flatpack-apps/73111/3

I don't have SELinux installed here. Hard to try to reproduce.

kkovaletp commented 11 months ago

@paulrouget

V2 is still in testing phase.

OK, I see. It must have been in beta for a long time already, as I remember that I've been using the 2.x version on my Mac for a while - it was installed and updated by the homebrew there.

Maybe this is relevant

I'm really far away from configuring SELinux manually) Here is what I see for the FF from the ls -lZ /var/lib/flatpak/app/org.mozilla.firefox command:

lrwxrwxrwx. 1 root root system_u:object_r:var_lib_t:s0   13 окт 15 17:17 current -> x86_64/stable
drwxr-xr-x. 3 root root system_u:object_r:var_lib_t:s0 4096 окт 15 17:17 x86_64

and here is the output for the /usr/local/vdhcoapp

-rwxr-xr-x. 1 root root unconfined_u:object_r:usr_t:s0 48653608 окт 19 08:04 ffmpeg
-rwxr-xr-x. 1 root root unconfined_u:object_r:usr_t:s0 48539368 окт 19 08:04 ffprobe
-rw-r--r--. 1 root root unconfined_u:object_r:usr_t:s0    15220 окт 19 08:04 LICENSE.txt
-rw-r--r--. 1 root root unconfined_u:object_r:usr_t:s0     4461 окт 19 08:04 README.md
-rwxr-xr-x. 1 root root unconfined_u:object_r:usr_t:s0 48349265 окт 19 08:04 vdhcoapp
-rwxr-xr-x. 1 root root unconfined_u:object_r:usr_t:s0    25778 окт 19 08:04 xdg-open

Maybe you could advise on how to modify it if needed?

Hard to try to reproduce.

setting up a VirtualBox VM isn't an option for you?

paulrouget commented 11 months ago

I installed Fedora, and I cannot reproduce the issue.

I'm running out of ideas.

@kkovaletp did this use to work with the older coapp?

paulrouget commented 11 months ago

@kkovaletp also, to make sure it's indeed a sandboxing issue (SELinux or Flatpak), can you try to download a non-flatpak version of Firefox (https://www.mozilla.org/en-US/firefox/linux/) and see if it works?

kkovaletp commented 11 months ago

@paulrouget, sorry for the delay, I was busy with other things...

I installed Fedora, and I cannot reproduce the issue.

Did you install the Fedora 38 KDE or the regular one with Gnome? I use the KDE build

did this use to work with the older coapp?

no, it is not. the behavior is completely the same as for 2.x version: no co-app was found in the case of system installation and unexpected error in the case of user installation. BTW, is there a way to see the text and stacktrace of the unexpected error, reported on the add-on settings page? it might help to debug the issue...

can you try to download a non-flatpak version of Firefox

it works in non-flatpak version with enabled SELinux right after installation of the add-on, as co-app is installed system-wide and user-wide, but for some reason, the dnf-installed FF don't use the same profile, as the flatpak one, so it was started clean and I had to install the add-on manually. also, I found a command to temporary disable SELinux for testing: setenforce 0. there is no difference after execution of the command and restart the flatpak FF, so the root cause is not in SELinux, but somewhere in flatpak

paulrouget commented 11 months ago

Did you install the Fedora 38 KDE or the regular one with Gnome? I use the KDE build

Gnome. It's possible that there is a KDE-specific thing going on here.

BTW, is there a way to see the text and stacktrace of the unexpected error, reported on the add-on settings page? it might help to debug the issue...

Sadly no. The process is spawned by Firefox itself. So it's not something we can easily debug. You can strace the firefox process see when the coapp is launched. But that's a little technical.

From what you're describing, it feels like the Firefox short-circuit to traverse the flatpak sandbox doesn't work on your machine. I think for these weird case, best is to install the coapp within the Firefox sandbox. It's stupid but I don't see a better way of doing it.

kkovaletp commented 11 months ago

@paulrouget, I finally got it working! and as usual, the fix was so easy and on the surface, but I dug too deep in the wrong direction)

when I wrote the previous comment and asked about the way to debug it, I started googling how to debug an add-on for FF and how to debug the FF itself on the flatpak level. and I found the command: flatpak run --log-session-bus --log-system-bus com.application.id &> logfile.txt

once I started the FF this way, I pressed a button to find the co-app to trigger the error again to be logged. then I analyzed the log and the 1st string there was: F: Do not share "/usr/local/vdhcoapp" with the sandbox: Path "/usr" reserved for Flatpak there are other errors, but they are obviously caused by this one. so, I've uninstalled the co-app from the /usr/local/vdhcoapp, moved it to the /opt folder, and installed it from there.

and it just works!

now I remember that there is a good practice on Linux to put all unpackable software in the /opt folder, as it is created exactly for such content, but I trusted the instructions on the download page and ignored this question until I saw the error...

paulrouget commented 11 months ago

@kkovaletp oh wow … that's amazing detective work. Thank you so much! I will update the documentation right away :)

mister-bum commented 8 months ago

@paulrouget, I finally got it working! and as usual, the fix was so easy and on the surface, but I dug too deep in the wrong direction)

when I wrote the previous comment and asked about the way to debug it, I started googling how to debug an add-on for FF and how to debug the FF itself on the flatpak level. and I found the command: flatpak run --log-session-bus --log-system-bus com.application.id &> logfile.txt

once I started the FF this way, I pressed a button to find the co-app to trigger the error again to be logged. then I analyzed the log and the 1st string there was: F: Do not share "/usr/local/vdhcoapp" with the sandbox: Path "/usr" reserved for Flatpak there are other errors, but they are obviously caused by this one. so, I've uninstalled the co-app from the /usr/local/vdhcoapp, moved it to the /opt folder, and installed it from there.

and it just works!

now I remember that there is a good practice on Linux to put all unpackable software in the /opt folder, as it is created exactly for such content, but I trusted the instructions on the download page and ignored this question until I saw the error...

Excuse me if I hijack this already closed thread, but I am neither very competent with github nor with Linux. I have installed the co-app using the /usr folder and it works with the vivaldi browser which is installed from the main repository. However, the /opt folder would have been my first choice, but the instructions clearly state not to install the co-app as sudo. And afaik, it is not even possible to copy files to /opt without sudo. Is it possible to run the install in the /opt folder without using sudo rights? I could of course simply try this, but I am reluctant to fiddle with a working configuration...

paulrouget commented 8 months ago

@mister-bum

Let me try to clarify the installation process on Linux:

If you use the .deb, just install it with dpkg -i the_deb_file.deb or double click on the file. This can be done as root, no problem.

If you use the .tar.bz2, extract it where ever you want, as root or not, doesn't matter. Just do not extract it in /usr/ if you're using a snap/flatpak based browser.

If you use the .tar.bz2, then you need to run as user (not sudo) the vdhcoapp install command (not necessary if you used the .deb).

Please not we have an issue at the moment with the CoApp and Ubuntu: https://github.com/aclap-dev/video-downloadhelper/discussions/12

Hope this helps.

kkovaletp commented 8 months ago

the instructions clearly state not to install the co-app as sudo

(I assume that) The idea behind this recommendation is to limit the co-app installation to the current user while installing it using sudo will apply the configuration system-wide, but won't affect other regular users (which is confusing, if you don't understand how Linux user management works). Anyway, chances that you'll need the co-app running on the system level are pretty low, so for most cases, you don't need the sudo installation.

Is it possible to run the install in the /opt folder without using sudo rights?

Yes, but you need to do some preparation:

mister-bum commented 8 months ago

Wow, many thanks to both of you for your very prompt replies!! Both of them are helpful to me.

The .deb approach is what I used successfully for the last months. But sometimes software is provided only as a .tar.bz2 and just to get used to handle these files, I decided to install the co-app from the .tar archive this time. Kind of a modest personal linux workout... ;-)

Thanks for pointing out the importance of the ownership - I have the feeling that this was the missing piece for me to understand how to use the /optfolder without using the install command as root.

Have a nice weekend both of you !!

philmcrakin commented 7 months ago

If you're using Libre Wolf it works if you move the native-messaging-hosts directory to /usr/lib/librewolf/

It should match this: /usr/lib/librewolf/native-messaging-hosts/net.downloadhelper.coapp.json

Sorry if this is in the wrong place or already mentioned elsewhere.