Move yt-dlp into site-packages

[xibr]

I will move yt-dlp to site-package directory next to mutagen and I will replace the current yt-dlp update method where yt-dlp will be fetched from pypi.org I will also replace Jackson Databind with gson All these changes will speed up yt-dlp.

For example I will use the file yt_dlp-2022.10.4-py2.py3-none-any.whl

What do you think?

[JunkFood02]

Gson seems like deprecated. I think Moshi is a more modern choice.

[xibr]

After merging #195 I will take a look at it.

[BobbyESP]

@xibr Hi! I wanted to ask you which way do you though for installing the whl files. I'm very interested on it for my project Spowlo because we use spotdl and using Pip for installing the .whl file would be nice, direct dependencies downloading. I think I found the way for using Pip, and it's the next:

Link here

The problem is that when trying to run it, the process exists with error 1, but well, it doesn't matter, I just wanted to ask which way do you though that would be good for implementing this. Thanks for the response! Wanted to ask you if there is a faster way to communicate with you if it's okay, thanks another time!

[BobbyESP]

If you need some extra info about what I have done, I'll be glad to give it to you

[BobbyESP]

@xibr Sorry for the ping but can you tell me which way would you follow for achieving this?

[xibr]

Hi I will not use pip but I will somehow download the file yt_dlp-x-py2.py3-none-any.whl from https://pypi.org for example I will get the following link https://files.pythonhosted.org/packages/5c/da/ef08140cea3392288a8f6cd60f6f12510a4c5776eead53e90151f139af19/yt_dlp-2023.7.6-py2.py3-none-any.whl I will get the file yt_dlp-2023.7.6-py2.py3-none-any.whl In this case I will unzip the file in the site-packages folder after that the cache files will be created in __pycache__ folders for example YoutubeDL.cpython-38.pyc which I think might be faster than the current method.

[BobbyESP]

Wow, that's a nice idea. Really thanks for this explanation. I will be looking for the changes you do because this would be nice

[BobbyESP]

By the way, related to #230 , I have created the Python package and tried to get the libpython.so file from inside the zip, but can't execute it (gives something like error 139). Just wanted to ask if it needs to be compiled or just borrowed from the zip file?

[BobbyESP]

I leave here a script that generates the site-packages folder (depending on your Python version). UPDATED VERSION

import zipfile
import tarfile
import os
import shutil
import subprocess
import json

def load_config(config_file):
    if os.path.exists(config_file):
        with open(config_file, 'r') as f:
            config = json.load(f)
            return config
    else:
        return {
            "download_directory": "",
            "destination_dir": ""
        }

def save_config(config, config_file):
    with open(config_file, 'w') as f:
        json.dump(config, f, indent=4)

config_file = "whl_packages_downloader_config.json"
config = load_config(config_file)

download_directory = config["download_directory"]
destination_dir = config["destination_dir"]

if not download_directory:
    download_directory = input("Enter the download directory: ")
if not destination_dir:
    destination_dir = input("Enter the destination directory: ")

# Prompt the user to input the package name
package_name = input("Enter the package name to download from Pypi and it's dependencies (e.g., spotdl): ")

# Execute the pip download command for the specified package
pip_command = f'pip3.8 download -d "{download_directory}" {package_name}'
subprocess.run(pip_command, shell=True)

# Create the destination directory if it doesn't exist
if not os.path.exists(destination_dir):
    os.makedirs(destination_dir)

# Iterate through all files in the download directory
for filename in os.listdir(download_directory):
    file_path = os.path.join(download_directory, filename)

    if filename.endswith(".whl"):
        with zipfile.ZipFile(file_path, 'r') as zip_ref:
            zip_ref.extractall(destination_dir)
        print(f".whl files extracted from {file_path} to {destination_dir}")

    if filename.endswith(".tar.gz"):
        with tarfile.open(file_path, 'r:gz') as tar_ref:
            tar_ref.extractall(destination_dir)
        print(f".tar.gz files extracted from {file_path} to {destination_dir}")

    # Remove folders ending with "dist-info"
    for root, dirs, files in os.walk(destination_dir):
        for dir_name in dirs:
            if dir_name.endswith("dist-info"):
                dir_path = os.path.join(root, dir_name)
                shutil.rmtree(dir_path)
                print(f"Removed {dir_path} directory.")

# Open the destination directory
os.startfile(destination_dir)

config["download_directory"] = download_directory
config["destination_dir"] = destination_dir
save_config(config, config_file)

print("Process completed.")

[BobbyESP]

I tried running yt-dlp by using "pythonPath -m yt-dlp" and the process exits with code 1.

I have generated the site-packages folder with the above script