cannot connect to the service geckodriver (on arm64 ubuntu 20.04)

fluffyloopy commented 3 years ago

arm64 ubuntu 20.04

installed geckodriver from package manager and replaced the one in geckodriver folder with it.

animepahe-dlr git/main  35s
❯ python3 ap-dlr.py
Message: Can not connect to the Service geckodriver

Traceback (most recent call last):
  File "ap-dlr.py", line 9, in <module>
    from custom_modules import inbuilt_dlr
  File "/root/instances/animeDownloaders/animepahe-dlr/custom_modules/inbuilt_dlr.py", line 7, in <module>
    from custom_modules.initiate_driver import driver, WebDriverWait, EC, By
  File "/root/instances/animeDownloaders/animepahe-dlr/custom_modules/initiate_driver.py", line 38, in <module>
    driver.install_addon(parent_dir + os.path.sep + "driver_extensions" + os.path.sep + "universal-bypass.xpi", temporary=True)
NameError: name 'driver' is not defined

ajxv commented 3 years ago

Hey @fluffyloopy, thanks for bringing this to attention. I have pushed some new changes to the main branch. Please try again after pulling this update. If the issue still persists, please let me know. 👍🏻

fluffyloopy commented 3 years ago

  File "setup.py", line 3, in <module>
    with open("Readme.md", "r") as fh:
FileNotFoundError: [Errno 2] No such file or directory: 'Readme.md'

after renaming it to README.md and running python3 setup.py install

❯ animepahe-dlr
Message: Can not connect to the Service geckodriver

Traceback (most recent call last):
  File "/usr/local/bin/animepahe-dlr", line 11, in <module>
    load_entry_point('animepahe-dlr==0.1.14', 'console_scripts', 'animepahe-dlr')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.8/dist-packages/animepahe_dlr-0.1.14-py3.8.egg/ap_dlr.py", line 9, in <module>
    from custom_modules import inbuilt_dlr
  File "/usr/local/lib/python3.8/dist-packages/animepahe_dlr-0.1.14-py3.8.egg/custom_modules/inbuilt_dlr.py", line 7, in <module>
    from custom_modules.initiate_driver import driver, WebDriverWait, EC, By
  File "/usr/local/lib/python3.8/dist-packages/animepahe_dlr-0.1.14-py3.8.egg/custom_modules/initiate_driver.py", line 38, in <module>
    driver.install_addon(parent_dir + os.path.sep + "driver_extensions" + os.path.sep + "universal-bypass.xpi", temporary=True)
NameError: name 'driver' is not defined

ajxv commented 3 years ago

Can you install the package using pip install animepahe-dlr and try running it, And see if that too throws the same error.?

fluffyloopy commented 3 years ago

ajxv commented 3 years ago

@fluffyloopy , I tested the package on Ubuntu 20.04 and it seems to work fine. Here is an image of how it should look: animepahe-dlr I am looking into the possible areas in code you might be facing the issue. 😟 I'll update and correct the issue once I've identified the cause.

In the meantime, please make sure you are using the latest version of the Firefox browser. Also, the pip package can also be installed on Windows10, in case you are interested in trying it on windows.

PS: Take a look at the fix mentioned by @ rsuciu-gsg in this issue and see if it helps.

fluffyloopy commented 3 years ago

it does work on windows 10 but my interest with it working on ubuntu comes from the fact im using it under proot on termux with limited mobile data. My wifi gets so slow it fails the download halfway through and it has no resume function. As for firefox version, i am using 88.0.1 on both devices.

ajxv commented 3 years ago

it does work on windows 10 but my interest with it working on ubuntu comes from the fact im using it under proot on termux with limited mobile data. My wifi gets so slow it fails the download halfway through and it has no resume function. As for firefox version, i am using 88.0.1 on both devices.

Since you are using 'termux' and not default terminal, that could be what's causing the issue (could be).

And, sadly, the inbuilt downloader doesn't have a resume function 😞 . I'll look into what i can do about that.

As for the resume function, you can try using idm for capturing downloads using animepahe-dlr -idm if you have idm installed. I am also working on adding an option to capture download with FDM (since idm is not a free software). I hope that could help with the resume function to an extend.

fluffyloopy commented 3 years ago

i got it to work by putting in absolute path but geckodriver/firefox keeps crashing so i give up.

Also, is there no way to pass the download to wget in windows with -c for continuous download?

ajxv commented 3 years ago

I'll try and fix the geckodriver issue as soon as i can.

Edit: Using wget or curl for downloads might indeed be possible but it seems like it would require extra dependencies. Since requests can basically do the things the other two does, i'll be continuing with requests for the time being.

ajxv commented 3 years ago

@fluffyloopy The new version of the package now supports resuming incomplete downloads. Also do check if the new version solves your previous issue.

fluffyloopy commented 3 years ago

a lot of new errors which makes it unable to locate geckodriver easily on my arm device, forcing the script to use system one fails with thinking its a directory? thanks for making it support incomplete downloads tho, that'll help me a lot. as for arm, i'll check and see what i can do about it later by trying the same in an x86 ubuntu environment with same settings. the issue with geckodriver for now is that it tries to use x64 driver instead of arm64

fluffyloopy commented 3 years ago

[#] AnimePahe_ChaoS_Child_-_12_BD_720p_Retail.mp4:   0%|     | 48.1k/84.3M [00:23<3:51:54, 6.06kB/s]<class 'selenium.common.exceptions.TimeoutException'> occured! Retrying..
File already exists! Resuming download..
<class 'requests.exceptions.ChunkedEncodingError'> occured! Retrying..84.3M [00:46<4:07:31, 5.57kB/s]
File already exists! Resuming download..Retail.mp4:   2%|     | 1.59M/84.3M [00:57<4:07:29, 5.57kB/s]
[#] AnimePahe_ChaoS_Child_-_12_BD_720p_Retail.mp4:   1%|      | 918k/82.7M [00:24<3:24:41, 6.66kB/s]
[#] AnimePahe_ChaoS_Child_-_12_BD_720p_Retail.mp4:   1%|      | 933k/82.7M [00:36<3:24:39, 6.66kB/s]
[#] AnimePahe_ChaoS_Child_-_12_BD_720p_Retail.mp4:   0%|     | 32.8k/81.8M [00:04<3:11:02, 7.13kB/s]

Resume function restarts the download

ajxv commented 3 years ago

It doesn't. You can see it if you check the total size.

In line 4 : total size 84.3mb, downloaded 1.59mb
when it resumes (line 5) it shows the total size after subtracting the size that's already been downloaded hence 82.7 (84.3 - 1.59).
when the download resumes the progress being shown is relative to the remaining size to be downloaded.

You can keep an eye on the file size ,in the terminal or just using a file explorer, when the download resumes to confirm this.

fluffyloopy commented 3 years ago

Oh i did not realize. Thank you. Also i checked with x86_64 ubuntu under termux too.

  File "/root/animepahe-dlr/animepahe_dlr/ap_dlr_modules/initiate_driver.py", line 48, in <module>
    driver.install_addon(parent_dir + os.path.sep + "driver_extensions" + os.path.sep + "universal-bypass.xpi", temporary=True)
NameError: name 'driver' is not defined

so we can maybe go for the fact that using them in termux is what's causing this issue? But why would using termux with ubuntu 20.04 under qemu mess it up?

ajxv commented 3 years ago

That could be the issue if the script works fine while running it in the default terminal.

One other possible cause i can think of is not having write permission to the directory. From what is seen in the error, the script seems to be under the root directory. And i guess write permission to root directory is not granted by default. That could be keeping the script from downloading and unpacking 'geckodriver' and hence causing the given issue.

If not having write permission is indeed the cause:

move the script to a directory with write access and try executing it. (this should work)
give write permission to /root manually using chmod (NOT RECOMMENDED)

Edit: If permission error were the cause you should've seen the error message. Since no permission error was thrown it couldn't be a permission error. Also can you confirm that the issue is with using termux and the script works fine in default terminal ?

fluffyloopy commented 3 years ago

I've tried it within terminology and XTerm too. Both ended with failures. Could be due to it being under proot? I've seen people using docker have similar problems using docker. And geckodriver logfile shows

1623495031345   geckodriver     INFO    Listening on 127.0.0.1:59961
1623495032420   mozrunner::runner       INFO    Running command: "/usr/bin/firefox" "--marionette" "--headless" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile9kbAZH"
*** You are running in headless mode.
[Parent 3117, Main Thread] WARNING: read-only dup failed (Permission denied); not using memfd: file /build/firefox-tDblH3/firefox-88.0.1+build1/ipc/chromium/src/base/shared_memory_posix.cc:ix.cc:243

JavaScript error: resource:///modules/sessionstore/SessionStore.jsm, line 2161: TypeError: subject.QueryInterface is not a function
A content process crashed and MOZ_CRASHREPORTER_SHUTDOWN is set, shutting down
console.error: Region.jsm: "Error fetching region" (new TypeError("NetworkError when attempting to fetch resource.", ""))
console.error: Region.jsm: "Failed to fetch region" (new Error("NO_RESULT", "resource://gre/modules/Region.jsm", 419))
JavaScript error: resource://gre/modules/DeferredTask.jsm, line 215: Error: Unable to arm timer, the object has been finalized.

(/usr/lib/firefox/firefox:3250): GLib-GObject-CRITICAL **: 10:50:41.220: g_object_set: assertion 'G_IS_OBJECT (object)' failed
console.error: "Could not load engine wikipedia@search.mozilla.org: Error: Attempt to use XPI database when it is not initialized"
console.error: "Could not load engine bing@search.mozilla.org: TypeError: AddonManagerInternal._getProviderByName(...) is undefined"
console.error: "Could not load engine ddg@search.mozilla.org: TypeError: AddonManagerInternal._getProviderByName(...) is undefined"
console.warn: SearchService: "_init: abandoning init due to shutting down"
JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 575: uncaught exception: 2147500036
JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 575: uncaught exception: Object

fluffyloopy commented 3 years ago

OH COME ON, IT WAS ALL BECAUSE FIREFOX WOULDN'T LOAD WITHOUT MOZ_FAKE_NO_SANDBOX=1. THAT FIXED EVERYTHING

i had my main firefox settled to that using about:config and did not realize i'd have to set it again to be used with geckodriver. export MOZ_FAKE_NO_SANDBOX=1 fixed and it works as normal now.

ajxv commented 3 years ago

Oh. I am glad you were able to find a solution.

If possible could you note down the steps here for reference in case someone else runs into the same issue in future.

fluffyloopy commented 3 years ago

Typing export MOZ_FAKE_NO_SANDBOX=1 on terminal and enter. Or putting export MOZ_FAKE_NO_SANDBOX=1 onto .bashrc or .zshrc and restarting shell.

ajxv / animepahe-dlr

cannot connect to the service geckodriver (on arm64 ubuntu 20.04) #5