breuerfelix / chromedriver-py

chromedriver self updated binaries for all platforms
https://pypi.org/project/chromedriver-py/
Apache License 2.0
50 stars 16 forks source link

Missing release for Chrome 115? #30

Closed RileyXX closed 1 year ago

RileyXX commented 1 year ago

I noticed Chrome 115 has launched recently but still don't see it as part of chromedriver-py releases. https://pypi.org/project/chromedriver-py/#history.

Could there be a bug?

Where does chromedriver-py pull releases from?

Edit: I see now it is probably not a bug. Chrome version 115 is just not listed in their api yet https://chromedriver.storage.googleapis.com/

breuerfelix commented 1 year ago

seems like 115 is still not released... do you know if they have a new api? or is this version just not available? i can't find anything on the web

RileyXX commented 1 year ago

After looking at it, looks like according to https://chromedriver.chromium.org/downloads they might have changed the API for versions 115 and newer.

"If you are using Chrome version 115 or newer, please consult the Chrome for Testing availability dashboard. This page provides convenient JSON endpoints for specific ChromeDriver version downloading."

"Consult our JSON API endpoints if you’re looking to build automated scripts based on Chrome for Testing release data."

RileyXX commented 1 year ago

From https://github.com/GoogleChromeLabs/chrome-for-testing#json-api-endpoints:

Endpoint Description
known-good-versions.json The versions for which all CfT assets are available for download. Useful for bisecting.
known-good-versions-with-downloads.json Same as above, but with an extra downloads property for each version, listing the full download URLs per asset.
last-known-good-versions.json The latest versions for which all CfT assets are available for download, for each Chrome release channel (Stable/Beta/Dev/Canary).
last-known-good-versions-with-downloads.json Same as above, but with an extra downloads property for each channel, listing the full download URLs per asset.
latest-patch-versions-per-build.json The latest versions for which all CfT assets are available for download, for each known combination of MAJOR.MINOR.BUILD versions.
latest-patch-versions-per-build-with-downloads.json Same as above, but with an extra downloads property for each version, listing the full download URLs per asset.
latest-versions-per-milestone.json The latest versions for which all CfT assets are available for download, for each Chrome milestone.
latest-versions-per-milestone-with-downloads.json Same as above, but with an extra downloads property for each milestone, listing the full download URLs per asset.

The set of “all CfT assets” for a given Chrome version is a matrix of supported binaries × platforms.

The current list of supported binaries is:

The current list of supported platforms is:

RileyXX commented 1 year ago

Looks like they added support for win64 in the new api as well

RileyXX commented 1 year ago

I asked chatgpt to modify some things so take the following with a grain of salt.

Modified chromedriver_py /__init__.py with added support for win64 :

import os
import platform

_BASE_DIR = os.path.dirname(os.path.abspath(__file__))

def _get_filename():
    path = os.path.join(_BASE_DIR, "chromedriver_")
    machine = platform.machine().lower()

    sys = platform.system().lower()
    if sys == "windows":
        if "64" in machine:
            path += "win64.exe"  # Use the 64-bit binary for Windows
        else:
            path += "win32.exe"  # Default to 32-bit binary for Windows
    elif sys == "darwin":
        path += "mac"
        if "arm" in machine:
            path += "_arm64"
        else:
            path += "64"
    elif sys == "linux":
        if machine.endswith("32"):
            raise Exception("Google doesn't compile 32bit chromedriver versions for Linux. Sorry!")
        if "arm" in machine:
            raise Exception("Google doesn't compile chromedriver versions for Linux ARM. Sorry!")
        path += "linux64"
    else:
        raise Exception("Could not identify your system: " + sys)

    if not path or not os.path.isfile(path):
        msg = "Couldn't find a binary for your system: " + sys + " / " + machine + ". "
        msg += "Please create an Issue on github.com/breuerfelix/chromedriver-py and include this Message."
        raise Exception(msg)

    return path

binary_path = _get_filename()
RileyXX commented 1 year ago

Modified chromedriver-py / updater.py to account for the new api and added win64 support.

import os
import sys
import urllib
import zipfile
import requests
from bs4 import BeautifulSoup as bs

# constants
CHROMEDRIVER_URL = "https://googlechromelabs.github.io/chrome-for-testing/latest-versions-per-milestone-with-downloads.json"
CHROMEDRIVER_FILE_NAME = "chromedriver_"
CHROMEDRIVER_EXTENSION = ".zip"

DOWNLOAD_DIR = "./chromedriver_py/"
VERSION_FILE = "./CURRENT_VERSION.txt"

# don't forget to include all platforms in 'MANIFEST.in' file!
# also update the import path in the package
PLATFORMS = [
    "linux64",
    "win32",
    "win64",  # Add support for Windows 64-bit
    "mac64",
    "mac_arm64",
]

# Function to compare two version numbers
def compare_int_arrays(old, new):
    shortest = old if len(old) < len(new) else new

    for i in range(len(shortest)):
        if new[i] == old[i]:
            continue

        if new[i] > old[i]:
            return True

        break

    return False

# Function to check for updates
def check_for_update(old_version_param=None):
    if not old_version_param:
        old_version_param = "0.0"

    old_version = []
    try:
        old_version_param = old_version_param.split(".")
        for v in old_version_param:
            old_version.append(int(v))
    except:
        print("error parsing old version!")
        return None, None

    print("old version: " + str(old_version))

    try:
        response = requests.get(CHROMEDRIVER_URL)
        json_data = response.json()

        versions_to_update = set()
        checked_keys = []

        highest_version = []

        milestones = json_data.get("milestones", {})
        for milestone, milestone_data in milestones.items():
            version_list = milestone_data.get("version", "").split(".")
            if not version_list:
                continue

            versions = []
            for v in version_list:
                try:
                    versions.append(int(v))
                except:
                    print("failed parsing to version number: " + milestone_data.get("version", ""))
                    versions = []
                    break

            if not versions:
                continue

            # continue if version is already checked
            if milestone_data.get("milestone", "") in checked_keys:
                continue

            checked_keys.append(milestone_data.get("milestone", ""))

            if compare_int_arrays(old_version, versions):
                versions_to_update.add(milestone_data.get("version", ""))

                # check for highest possible version
                if not highest_version:
                    highest_version = versions
                elif compare_int_arrays(highest_version, versions):
                    highest_version = versions

        highest_version = ".".join(str(x) for x in highest_version)

        print("versions to update: " + str(versions_to_update))
        print("highest version to update: " + highest_version)

        return highest_version, versions_to_update

    except Exception as e:
        print("Failed to fetch data from the API: ", str(e))

    return None, None

# Function to update the version
def update_version(version):
    for p in PLATFORMS:
        filename = CHROMEDRIVER_FILE_NAME + p
        filename_version = version + "/" + filename + CHROMEDRIVER_EXTENSION
        url = CHROMEDRIVER_URL + filename_version
        print(url)

        # downloads the files
        try:
            file = urllib.request.urlopen(url)
        except:
            print("could not get file: " + filename)
            continue

        # save file to system
        path = os.path.join(DOWNLOAD_DIR, filename + CHROMEDRIVER_EXTENSION)
        with open(path, "wb") as output:
            print("write to: " + path)
            output.write(file.read())

        # unzip file
        print("unzip file: " + path)
        with zipfile.ZipFile(path, "r") as zip_ref:
            zip_ref.extractall(DOWNLOAD_DIR)

        # rename file
        extracted_name = "chromedriver"
        if p in ["win", "win32", "win64"]:
            extracted_name += ".exe"
            filename += ".exe"

        print("rename file to: " + filename)
        final_path = os.path.join(DOWNLOAD_DIR, filename)
        os.rename(os.path.join(DOWNLOAD_DIR, extracted_name), final_path)

        # give execute permission
        print("setting permissions")
        os.chmod(final_path, 0o755)

        # delete zip file
        print("removing file: " + path)
        os.remove(path)

    return version

# Function to get version from file
def get_version_from_file(path):
    try:
        with open(path, "r") as f:
            current_version = f.read().strip()

        return current_version
    except:
        pass

    return None

if __name__ == "__main__":
    # get environment version for custom travis builds
    env_version = os.environ.get("VERSION")

    if not env_version:
        current_version = get_version_from_file(VERSION_FILE)
        print("current version: " + str(current_version))

        version, _ = check_for_update(current_version)

        if not version:
            # exit with code 1 to prevent auto deploy
            sys.exit(1)
    else:
        print("got version from environment: " + env_version)
        version = env_version

    version = update_version(version)
    print("version updated: " + version)

    # update version file
    with open(VERSION_FILE, "w") as f:
        # only write the major and minor version to file
        f.write(version)

    # exit with code 0 to enable auto-deploy
    sys.exit(0)
RileyXX commented 1 year ago

Both of those pieces of code were generated by ChatGPT so there's probably issues with them. Still, they might provide a good reference point.

breuerfelix commented 1 year ago

thanks :) well since they now provide a json api, i will rewrite the updater anyways, its way easier to use a json api :)

problem is: i am on vacation till sunday so this fix might wait till next week... thanks for your investigation! that helped me a lot :) expect new versions next week!

RileyXX commented 1 year ago

Very cool, no problem. That's good to hear and thanks for the update! Yeah json is definitely a better approach. Nice to see they switched over. Thankfully it looks like chromedriver 114 is working fine on chrome 115 so this shouldn't be an issue for existing projects.

Enjoy your vacation!

breuerfelix commented 1 year ago

@RileyXX fixed it :) it now uses the new api, but i rewrote the whole script :D chatgpt is not the best option for this kind of task :D

but thanks for suggesting the new api / repo, that was helpful since i didnt have to research it myself :)

could you test the new 115 version? if some version in between was missing, just let me know and i will update it. from now on the pipeline should catch new versions and publish them asap!

if it doesnt work, let me know and i will fix it!

RileyXX commented 1 year ago

Yeah ChatGPT can be hit or miss.

Glad the API info was useful.

I just tested the latest chromedriver-py release (115) and all seems to be working good on my projects. Nicely done and thanks for the quick fix! 👍

breuerfelix commented 1 year ago

awesome, thanks for the fast feedback!