shailshouryya / yt-videos-list

Create and **automatically** update a list of all videos on a YouTube channel (in txt/csv/md form) via YouTube bot with end-to-end web scraping - no API tokens required. Multi-threaded support for YouTube videos list updates.
Apache License 2.0
107 stars 20 forks source link

Can't install, due to some issue with verison #20

Open Archie-2021 opened 1 year ago

Archie-2021 commented 1 year ago
❯ pipx install yt-video-list
Fatal error from pip prevented installation. Full pip output in file:
    /home/arash/.local/pipx/logs/cmd_2023-11-09_19.38.49_pip_errors.log

Some possibly relevant errors from pip install:
    ERROR: Could not find a version that satisfies the requirement yt-video-list (from versions: none)
    ERROR: No matching distribution found for yt-video-list

Error installing yt-video-list.

contents of pipx logfile:

PIP STDOUT
----------

PIP STDERR
----------
ERROR: Could not find a version that satisfies the requirement yt-video-list (from versions: none)
ERROR: No matching distribution found for yt-video-list
shailshouryya commented 1 year ago

Hi, thanks for filing this issue! I think this issue should might be addressed with release 0.6.7 (which I will release soon).

However, there might be another issue with pipx, since on both mac and Windows I get

$ pip install pipx
$ pipx install yt-videos-list
Fatal error from pip prevented installation. Full pip output in file:
    PATH_TO_THE-cmd_timestamp_pip_errors.log

pip seemed to fail to build package:
    yt-videos-list

Some possibly relevant errors from pip install:
    error: subprocess-exited-with-error
    error: metadata-generation-failed

Error installing yt-videos-list.

with the following in the PATH_TO_THE-cmd_timestamp_pip_errors.log file:

PIP STDOUT
----------
Collecting yt-videos-list
  Downloading yt_videos_list-0.6.6.tar.gz (41 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.4/41.4 kB 1.2 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'

PIP STDERR
----------
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [1 lines of output]
      error in yt_videos_list setup command: 'python_requires' must be a string containing valid version specifiers; Invalid specifier: '>=3.6.*'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

However, on Debian I get the following (but I might have set up my virtual environment incorrectly here/misconfigured something else):

$ sudo apt-get install python3-venv   # already have python3 so did not need to install that
$ python3 -m venv venv && source venv/bin/activate && pip install pipx
$ pipx install yt-videos-list

No apps associated with package yt-videos-list or its dependencies. If you are attempting to install a library, pipx should not be used. Consider using pip or a similar tool instead.

but all 3 of those outputs differ with what you saw:

PIP STDOUT
----------

PIP STDERR
----------
ERROR: Could not find a version that satisfies the requirement yt-video-list (from versions: none)
ERROR: No matching distribution found for yt-video-list

My guess is your issue might still related to the incorrect version specifier I was using in the setup.py file, so if this is indeed the case, your problem should be addressed with the new release. If you're interested, you can take a look at the recent commits Pin usable urllib3 verions, Update save_thread_result package dependency version number → 0.1.1, and Fix python_requires value (and the corresponding commit message for each).

Also, what OS are you using? If you're on Linux, even if release 0.6.7 fixes the installation problem, the package might not work properly since I haven't gotten around to making yt-videos-list fully compatible with Linux distributions yet due to the selenium, browser, and Linux specific distro compatibility complexity, and so using yt-videos-list on Linux requires a bit of manual troubleshooting. I have plans to make yt-videos-list easier to use on Linux in the future, but this is just a heads up in case you want to use this on a Linux distribution right away.

Archie-2021 commented 1 year ago

Hi, thanks for filing this issue! I think this issue should might be addressed with release 0.6.7 (which I will release soon).

However, there might be another issue with pipx, since on both mac and Windows I get

$ pip install pipx
$ pipx install yt-videos-list
Fatal error from pip prevented installation. Full pip output in file:
    PATH_TO_THE-cmd_timestamp_pip_errors.log

pip seemed to fail to build package:
    yt-videos-list

Some possibly relevant errors from pip install:
    error: subprocess-exited-with-error
    error: metadata-generation-failed

Error installing yt-videos-list.

with the following in the PATH_TO_THE-cmd_timestamp_pip_errors.log file:

PIP STDOUT
----------
Collecting yt-videos-list
  Downloading yt_videos_list-0.6.6.tar.gz (41 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.4/41.4 kB 1.2 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'

PIP STDERR
----------
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [1 lines of output]
      error in yt_videos_list setup command: 'python_requires' must be a string containing valid version specifiers; Invalid specifier: '>=3.6.*'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

However, on Debian I get the following (but I might have set up my virtual environment incorrectly here/misconfigured something else):

$ sudo apt-get install python3-venv   # already have python3 so did not need to install that
$ python3 -m venv venv && source venv/bin/activate && pip install pipx
$ pipx install yt-videos-list

No apps associated with package yt-videos-list or its dependencies. If you are attempting to install a library, pipx should not be used. Consider using pip or a similar tool instead.

but all 3 of those outputs differ with what you saw:

PIP STDOUT
----------

PIP STDERR
----------
ERROR: Could not find a version that satisfies the requirement yt-video-list (from versions: none)
ERROR: No matching distribution found for yt-video-list

My guess is your issue might still related to the incorrect version specifier I was using in the setup.py file, so if this is indeed the case, your problem should be addressed with the new release. If you're interested, you can take a look at the recent commits Pin usable urllib3 verions, Update save_thread_result package dependency version number → 0.1.1, and Fix python_requires value (and the corresponding commit message for each).

Also, what OS are you using? If you're on Linux, even if release 0.6.7 fixes the installation problem, the package might not work properly since I haven't gotten around to making yt-videos-list fully compatible with Linux distributions yet due to the selenium, browser, and Linux specific distro compatibility complexity, and so using yt-videos-list on Linux requires a bit of manual troubleshooting. I have plans to make yt-videos-list easier to use on Linux in the future, but this is just a heads up in case you want to use this on a Linux distribution right away.

Hi,

Yeah, I am on linux , EOS (which is Arch based). Hope it the issue get fixed on the next install. I also tries to run it directly in an virtual env directory instead of pipx but still failed.

Hope it get functional on Linux. I like to make watch list from YT channels quickly and this is just what I need. So thank you for your efforts.

shailshouryya commented 1 year ago

I just released version 0.6.7, so the following should now work:

# installing directly without any virtual environment
python3 -m pip install yt-videos-list
# installing in a virtual environment
python3 -m venv ytvl-venv
source ytvl-venv/bin/activate
pip3 install -U yt-videos-list

However, I'm still getting the following with pipx (on macOS using python version 3.12):

python3 -m venv ytvl-venv
source ytvl-venv/bin/activate
pip3 install pipx
pipx install yt-videos-list

# the output
No apps associated with package yt-videos-list or its dependencies. If you are attempting to install a library, pipx should not be used. Consider using pip or a similar tool instead.

I'm not entirely sure why this is happening and I'm not familiar enough with pipx to debug this right away, but for now I think you should be able to either install yt-videos-list directly into your system/user python environment or install yt-videos-list using venv. Also, the output does suggest using pip

If you are attempting to install a library, pipx should not be used. Consider using pip or a similar tool instead.

I anticipate that you'll still run into other (unrelated) problems after this with selenium if you're using yt-videos-list on Linux, though, so I'll make another issue outlining a roadmap for making yt-videos-list compatible with Linux, more robust, and less dependent on external dependencies.

Archie-2021 commented 1 year ago

This time I could install it with pip in venv without a problem. But when I try the script you put in the tutorial I get this:

===>ERROR!<===
There was an error while trying to open up the remote selenium instance. The exact error was:
Message: 'bravedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

Don't worry though, this is an easy fix!

=====> Now updating Selenium driver binaries and fixing any version incompatibility problems. <=====
This will update all corresponding Selenium drivers for browsers (which are installed in their default locations and) supported by the yt_videos_list package...
Traceback (most recent call last):
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/selenium/webdriver/common/service.py", line 72, in start
    self.process = subprocess.Popen(cmd, env=self.env,
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.11/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/bravedriver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/yt_videos_list/logic.py", line 198, in execute
    driver = open_user_driver()
             ^^^^^^^^^^^^^^^^^^
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/yt_videos_list/logic.py", line 55, in open_user_driver
    return supported_drivers[user_driver]()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/yt_videos_list/logic.py", line 90, in configure_bravedriver
    return webdriver.Chrome(options=options, executable_path=executable_path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    self.service.start()
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/selenium/webdriver/common/service.py", line 81, in start
    raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'bravedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/run", line 8, in <module>
    lc.create_list_for(url='https://www.youtube.com/user/schafer5')
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/yt_videos_list/__init__.py", line 323, in create_list_for
    video_data, write_information = logic.execute(deque([url]), file_name, log_silently, *instance_attributes, _DummyLock())
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/yt_videos_list/logic.py", line 200, in execute
    handle_opening_webdriver_exception(error_message)
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/yt_videos_list/logic.py", line 111, in handle_opening_webdriver_exception
    download_all()
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/yt_videos_list/download/selenium_webdriver_dependencies.py", line 339, in download_all
    download_all_dependencies(user_os)
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/yt_videos_list/download/selenium_webdriver_dependencies.py", line 40, in download_all_dependencies
    download_specific_dependency(driver, user_os)
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/yt_videos_list/download/selenium_webdriver_dependencies.py", line 30, in download_specific_dependency
    if selenium_user_os.verify_browser_exists(browser):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/arash/arashafile/ariroot/applications/app-source-repo/yt-videos-list/ytvl-venv/lib/python3.11/site-packages/yt_videos_list/download/selenium_linux.py", line 2, in verify_browser_exists
    raise RuntimeError('Browser detection and automatic selenium updates are not yet available for Linux distributions!\nPlease update your selenium driver manually.')
RuntimeError: Browser detection and automatic selenium updates are not yet available for Linux distributions!
Please update your selenium driver manually.
shailshouryya commented 1 year ago

Ah yes, this is the problem I was alluding to earlier in the thread when I said, "requires a bit of manual troubleshooting."

I noticed you tried to use brave based on the error logs above, but I added instructions for using firefox as well since using brave with yt-videos-list is already very tricky on macOS and Windows, and using brave on Linux is even more tricky. In case brave does not work, hopefully using firefox should be easier.

Also, refer to the "permissions for first run" section in the quick start to make sure your environment is set up properly.

using firefox with yt-vidoes-list

If you do not already have the Mozilla Firefox browser installed, follow the instructions at https://support.mozilla.org/en-US/kb/install-firefox-linux. I included the commands for downloading Mozilla Firefox on Debian from https://wiki.debian.org/Firefox#From_Debian_packages below, but substitute the equivalent commands for your Linux distribution as necessary if you use a different Linux distribution.

sudo apt update
sudo apt install firefox-esr

After ensuring Mozilla Firefox is installed on the machine, we can

We can then substitute the URL for the appropriate firefox binary into the command to download the binary (see example commands). For Linux on ARM, the URL will be https://github.com/mozilla/geckodriver/releases/download/v0.33.0/geckodriver-v0.33.0-linux-aarch64.tar.gz and the command is:

curl -SL https://github.com/mozilla/geckodriver/releases/download/v0.33.0/geckodriver-v0.33.0-linux-aarch64.tar.gz | tar -xzvf - -C /usr/local/bin/ && chmod +x /usr/local/bin/geckodriver

The following should now work:

# in a python script or python interpreter
from yt_videos_list import ListCreator
lc = ListCreator(driver='firefox')
lc.create_list_for('youtube.com/user/schafer5')

Refer to the More API information section for more configuration options.

using brave with yt-vidoes-list

If you do not already have Brave Browser installed, follow the instructions at https://brave.com/linux/. I included the commands for downloading Brave Browser on Debian from https://brave.com/linux/#release-channel-installation below, but substitute the equivalent commands for your Linux distribution as necessary if you use a different Linux distribution.

# Release Channel Installation
# from the Debian, Ubuntu, Mint section at https://brave.com/linux/

sudo apt install curl
sudo curl -fsSLo /usr/share/keyrings/brave-browser-archive-keyring.gpg https://brave-browser-apt-release.s3.brave.com/brave-browser-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/brave-browser-archive-keyring.gpg] https://brave-browser-apt-release.s3.brave.com/ stable main"|sudo tee /etc/apt/sources.list.d/brave-browser-release.list
sudo apt update
sudo apt install brave-browser

The page also mentions

Brave is supported on 64-bit AMD/Intel (amd64 / x86_64) and ARM (arm64 / aarch64) architectures. See our full system requirements for minimum OS versions.

so this should be reliable across different platforms, but I could not launch Brave Browser when I tested this,

and you can take a look at the error logs here
$ brave-browser
[5713:5713:1111/213525.617411:ERROR:object_proxy.cc(577)] Failed to call method: org.kde.KWallet.isEnabled: object_path= /modules/kwalletd6: org.freedesktop.DBus.Error.ServiceUnknown: The name org.kde.kwalletd6 was not provided by any .service files
[5713:5713:1111/213525.617756:ERROR:kwallet_dbus.cc(112)] Error contacting kwalletd6 (isEnabled)
[5713:5713:1111/213525.619992:ERROR:object_proxy.cc(577)] Failed to call method: org.kde.KLauncher.start_service_by_desktop_name: object_path= /KLauncher: org.freedesktop.DBus.Error.ServiceUnknown: The name org.kde.klauncher was not provided by any .service files
[5713:5713:1111/213525.620088:ERROR:kwallet_dbus.cc(81)] Error contacting klauncher to start kwalletd6
[5713:5713:1111/213525.621986:ERROR:object_proxy.cc(577)] Failed to call method: org.kde.KWallet.close: object_path= /modules/kwalletd6: org.freedesktop.DBus.Error.ServiceUnknown: The name org.kde.kwalletd6 was not provided by any .service files
[5713:5713:1111/213525.622077:ERROR:kwallet_dbus.cc(503)] Error contacting kwalletd6 (close)
[5713:5713:1111/213525.631770:ERROR:object_proxy.cc(577)] Failed to call method: org.kde.KWallet.isEnabled: object_path= /modules/kwalletd5: org.freedesktop.DBus.Error.ServiceUnknown: The name org.kde.kwalletd5 was not provided by any .service files
[5713:5713:1111/213525.631852:ERROR:kwallet_dbus.cc(112)] Error contacting kwalletd5 (isEnabled)
[5713:5713:1111/213525.633914:ERROR:object_proxy.cc(577)] Failed to call method: org.kde.KLauncher.start_service_by_desktop_name: object_path= /KLauncher: org.freedesktop.DBus.Error.ServiceUnknown: The name org.kde.klauncher was not provided by any .service files
[5713:5713:1111/213525.634019:ERROR:kwallet_dbus.cc(81)] Error contacting klauncher to start kwalletd5
[5713:5713:1111/213525.635351:ERROR:object_proxy.cc(577)] Failed to call method: org.kde.KWallet.close: object_path= /modules/kwalletd5: org.freedesktop.DBus.Error.ServiceUnknown: The name org.kde.kwalletd5 was not provided by any .service files
[5713:5713:1111/213525.635482:ERROR:kwallet_dbus.cc(503)] Error contacting kwalletd5 (close)
[5713:5713:1111/213525.644825:ERROR:object_proxy.cc(577)] Failed to call method: org.kde.KWallet.isEnabled: object_path= /modules/kwalletd: org.freedesktop.DBus.Error.ServiceUnknown: The name org.kde.kwalletd was not provided by any .service files
[5713:5713:1111/213525.644896:ERROR:kwallet_dbus.cc(112)] Error contacting kwalletd (isEnabled)
[5713:5713:1111/213525.647160:ERROR:object_proxy.cc(577)] Failed to call method: org.kde.KLauncher.start_service_by_desktop_name: object_path= /KLauncher: org.freedesktop.DBus.Error.ServiceUnknown: The name org.kde.klauncher was not provided by any .service files
[5713:5713:1111/213525.647219:ERROR:kwallet_dbus.cc(81)] Error contacting klauncher to start kwalletd
[5713:5713:1111/213525.648292:ERROR:object_proxy.cc(577)] Failed to call method: org.kde.KWallet.close: object_path= /modules/kwalletd: org.freedesktop.DBus.Error.ServiceUnknown: The name org.kde.kwalletd was not provided by any .service files
[5713:5713:1111/213525.648343:ERROR:kwallet_dbus.cc(503)] Error contacting kwalletd (close)
[5713:5713:1111/213525.651091:ERROR:policy_logger.cc(157)] :components/enterprise/browser/controller/chrome_browser_cloud_management_controller.cc(163) Cloud management controller initialization aborted as CBCM is not enabled.
[1111/213526.641929:ERROR:file_io_posix.cc(144)] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq: No such file or directory (2)
[1111/213526.646667:ERROR:file_io_posix.cc(144)] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq: No such file or directory (2)
/usr/bin/brave-browser: line 48:  5713 Segmentation fault      (core dumped) "$HERE/brave" "$@"
[Sat Nov 11 21:35:27] ~ 1950|10|0 1 -bash5.1.4 

# logs seem to exit then resume:
libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
[5757:5757:0100/000000.545609:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.547433:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.547905:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.548187:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.548404:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.548619:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.548753:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.548890:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.549125:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.549356:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.551583:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.552131:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.552554:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.552786:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.553040:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.553294:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.553474:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.553720:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.553974:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.554153:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.554412:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.554692:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.555075:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.555364:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.555650:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.555860:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.556165:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.556390:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.556530:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.556644:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.556753:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.556989:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.557138:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.557376:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.557659:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.557839:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.558094:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.558405:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.558738:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)
[5757:5757:0100/000000.559103:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2)

However, once we do actually install Brave Browser on the machine and get it working properly, we can

We can then substitute the URL for the appropriate operadriver binary into the command to download the binary (see example commands). For Linux on ARM, the URL will be https://github.com/operasoftware/operachromiumdriver/releases/download/v.118.0.5993.89/operadriver_linux64.zip and the suggested, modified command is:

curl -SL https://github.com/operasoftware/operachromiumdriver/releases/download/v.118.0.5993.89/operadriver_linux64.zip -o temp_selenium.zip && unzip temp_selenium.zip -d /usr/local/bin/brave && mv /usr/local/bin/brave/operadriver_linux64/operadriver /usr/local/bin/bravedriver && rm temp_selenium.zip && rm -rf /usr/local/bin/brave/ && chmod +x /usr/local/bin/bravedriver

NOTE that the example commands will NOT work with some implementations of tar, hence why I suggest the modified command above using unzip.

After this, I think the following should work if you have Brave Browser already installed and working on your machine, (although I could not verify compatiibility with yt-videos-list due to errors I could not debug):

# in a python script or python interpreter
from yt_videos_list import ListCreator
lc = ListCreator(driver='brave')
lc.create_list_for('youtube.com/user/schafer5')
# results in the following,
# which might be because of the installation problem mentioned above

# alternatively, it is possible that the download commands from
# the official https://brave.com/linux/ page still incorrectly downloaded
# the binary for AMD architecture instead of the ARM architecture, since
# I have seen
# OSError: [Errno 8] Exec format error: '/usr/local/bin/{browser}driver'
# before when I incorrectly use a binary for a different platform

# another alternative is that the single operadriver binary
# is not cross-platform, and operadriver will NOT work with ARM
# architectures even if the Brave Browser is correctly installed

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python3.9/site-packages/yt_videos_list/__init__.py", line 323, in create_list_for
    video_data, write_information = logic.execute(deque([url]), file_name, log_silently, *instance_attributes, _DummyLock())
  File "/home/user/.local/lib/python3.9/site-packages/yt_videos_list/logic.py", line 198, in execute
    driver = open_user_driver()
  File "/home/user/.local/lib/python3.9/site-packages/yt_videos_list/logic.py", line 55, in open_user_driver
    return supported_drivers[user_driver]()
  File "/home/user/.local/lib/python3.9/site-packages/yt_videos_list/logic.py", line 90, in configure_bravedriver
    return webdriver.Chrome(options=options, executable_path=executable_path)
  File "/home/user/.local/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    self.service.start()
  File "/home/user/.local/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 72, in start
    self.process = subprocess.Popen(cmd, env=self.env,
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1823, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/usr/local/bin/bravedriver'

Refer to the More API information section for more configuration options.

Debugging Brave Browser installation I initially followed the instructions at https://www.linuxcapable.com/how-to-install-brave-browser-on-debian-linux/ and could not install Brave Borwser. I'm including this here in case this might be useful for anyone else, since although the instuctions at https://brave.com/linux/ should be more reliable, this might still be useful to know. The commands I ran: ``` sudo apt update && sudo apt upgrade sudo apt install software-properties-common apt-transport-https curl ca-certificates -y # Stable version of brave browser import command: wget -qO- https://brave-browser-apt-release.s3.brave.com/brave-browser-archive-keyring.gpg | sudo gpg --dearmor | sudo tee /usr/share/keyrings/brave-browser-archive-keyring.gpg > /dev/null echo "deb [arch=amd64 signed-by=/usr/share/keyrings/brave-browser-archive-keyring.gpg] https://brave-browser-apt-release.s3.brave.com/ stable main" | sudo tee /etc/apt/sources.list.d/brave-browser-release.list sudo apt update sudo apt install brave-browser ### results in ### # Reading package lists... Done # Building dependency tree... Done # Reading state information... Done # E: Unable to locate package brave-browser # NOTE: this seems to work after running the commands from # https://brave.com/linux/ (but after running the commands # from https://brave.com/linux/, there is no need to run this) ``` so then I tried installing the Beta build: ``` sudo apt update && sudo apt upgrade sudo apt install software-properties-common apt-transport-https curl ca-certificates -y # Beta version of brave browser import command: wget -qO- https://brave-browser-apt-beta.s3.brave.com/brave-browser-beta-archive-keyring.gpg | sudo gpg --dearmor | sudo tee /usr/share/keyrings/brave-browser-beta-archive-keyring.gpg > /dev/null echo "deb [arch=amd64 signed-by=/usr/share/keyrings/brave-browser-beta-archive-keyring.gpg] https://brave-browser-apt-beta.s3.brave.com/ stable main" | sudo tee /etc/apt/sources.list.d/brave-browser-beta.list sudo apt update sudo apt install brave-browser-beta ### results in ### # Reading package lists... Done # Building dependency tree... Done # Reading state information... Done # E: Unable to locate package brave-browser-beta ``` After these unsuccessful installation attempts, I searched for potential solutions and found https://community.brave.com/t/brave-install-on-chromebook-linux-e-unable-to-locate-package-brave-browser/333195, which mentions > It’s an ARM chromebook and you need the ARM version of Brave, but you’ve added the AMD64 repository. Unfortunately, I don’t know if there is an official or unofficial ARM version at this time, I know someone was working on it a couple years ago but I don’t know what came of it. I then checked the official https://brave.com/linux/ page with the suspicion that the AMD/ARM architecture was the cause of the installation problems (and ran the commands I included under the `# Release Channel Installation` section above). Considering this was posted in January 2022 and https://brave.com/linux/ now (November 2023) mentions > Brave is supported on 64-bit AMD/Intel (amd64 / x86_64) and ARM (arm64 / aarch64) architectures. it seems likely that official support **does now indeed** exist (as of November 2023).

Other potentially useful commands

sudo apt remove firefox-esr

sudo apt remove brave-browser
sudo apt remove brave-browser-beta
sudo apt remove brave-browser-nightly

# remove Brave beta repository
sudo rm /etc/apt/sources.list.d/brave-browser-beta.list

# remove all Brave repositories
sudo rm /etc/apt/sources.list.d/brave-browser-*

Unrelated

Do NOT type out long posts in GitHub's UI! I typed all this out a couple hours ago, and right before posting, accidentally clicked away from the page... and proceeded to lose everything I typed since GitHub does not save your content and automatically navigates away from the page with no warning. Sooo I typed this huge explanation and proofread this not once, but twice :( I used a text editor the second time, though, which I should have done the first time :)

There is https://github.com/orgs/community/discussions/42581 (and https://twitter.com/Nunatic02/status/1628066714968592386?s=20, which did not work for me), but the discussion does not look like it has gained traction.

Archie-2021 commented 1 year ago

Unrelated

Do NOT type out long posts in GitHub's UI! I typed all this out a couple hours ago, and right before posting, accidentally clicked away from the page... and proceeded to lose everything I typed since GitHub does not save your content and automatically navigates away from the page with no warning. Sooo I typed this huge explanation and proofread this not once, but twice :( I used a text editor the second time, though, which I should have done the first time :)

There is https://github.com/orgs/community/discussions/42581 (and https://twitter.com/Nunatic02/status/1628066714968592386?s=20, which did not work for me), but the discussion does not look like it has gained traction.

Thank you very much for you help , I know the pain of writing a long text and losing it in the browser. I still write in the browser being lazy, but I think the best solution is to write it in a text editor and copy paste it at the end.

I will test it with firefox as soon as I find some free time in this week. I have firefox preinstalled as it is the default Linux browser. My only confusion for now is why did you suggested ARM version for download. As I think 64 bit be the right architecture for any Linux machine. Only non-mobile computers which use ARM I think are the new Apple ones.

shailshouryya commented 1 year ago

My only confusion for now is why did you suggested ARM version for download.

This was a mistake, I mixed up the Linux Arch distribution with the aarch64 hardware architecture (more details here) when you mentioned "EOS (which is Arch based)."

As I think 64 bit be the right architecture for any Linux machine.

Since I mixed up the Arch architecture with the arch hardware, I think you are correct, and the corresponding command would actually be curl -SL https://github.com/mozilla/geckodriver/releases/download/v0.33.0/geckodriver-v0.33.0-linux64.tar.gz | tar -xzvf - -C /usr/local/bin/ && chmod +x /usr/local/bin/geckodriver instead for downloading the Firefox selenium binary (or curl -SL https://github.com/mozilla/geckodriver/releases/download/v0.33.0/geckodriver-v0.33.0-linux32.tar.gz | tar -xzvf - -C /usr/local/bin/ && chmod +x /usr/local/bin/geckodriver for 32 bit machines).

Only non-mobile computers which use ARM I think are the new Apple ones.

I think you are also correct here. I know some chromebooks and tablets use ARM, but we can categorize those as "mobile computers" too. I think some cloud providers are starting to provide ARM based servers as well, such as AWS Graviton, but I am unsure how much these are used. However, I did actually test this on an ARM device because of my Arch distribution/hardware confusion, so everything in my previous comment was verified on an ARM device. 🤓