Flaresolverr package and module do not work

[akho]

Describe the bug

The new flaresolverr package and module do not work. Flaresolverr never starts listening on its port.

Also, the test included with that package hangs. As far as I can understand, ofborg timed out running it in the original pr #324894, and running nix-build -A nixosTests.flaresolverr on current nixpkgs hangs after

machine # [   24.880934] flaresolverr[821]: 2024-08-06 17:44:20 INFO     Platform: Linux-6.6.44-x86_64-with-glibc2.39
machine # [   24.882897] flaresolverr[821]: 2024-08-06 17:44:20 INFO     Chrome / Chromium path: /nix/store/i3jc1r0mqpk9pr4y7qsvsdxb3y6c819x-chromium-127.0.6533.88/bin/chromium
machine # [   26.772615] flaresolverr[821]: 2024-08-06 17:44:22 INFO     Chrome / Chromium major version: 127
machine # [   26.774883] flaresolverr[821]: 2024-08-06 17:44:22 INFO     Launching web browser...

As machine.wait_for_open_port(8888) in the test never succeeds.

So it seems that the test was written, but not actually run before merging...

Steps To Reproduce

Steps to reproduce the behavior:

1. Add to configuration.nix:

      services.flaresolverr.enable = true;

2. Run

> curl --fail https://localhost:8191/v1                                                
curl: (7) Failed to connect to localhost port 8191 after 0 ms: Couldn't connect to server

Alternatively, run nix-build -A nixosTests.flaresolverr on current nixpkgs.

Expected behavior

Flaresolverr listening on port 8191, solving captchas.

Notify maintainers

@paveloom @K900

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

> nix-shell -p nix-info --run "nix-info -m"                                        
 - system: `"x86_64-linux"`
 - host os: `Linux 6.10.1, NixOS, 24.11 (Vicuna), 24.11.20240802.d049530`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.5`
 - nixpkgs: `/nix/store/r8nhgnkxacbnf4kv8kdi8b6ks3k9b16i-source`

---

Add a :+1: reaction to issues you find important.

[K900]

No issues here (though on aarch64).

[paveloom]

So it seems that the test was written, but not actually run before merging...

It was. The checks on the last set of commits timed out on the rebuild of Chromium, since @K900 added a build from the source. You can see successful checks on earlier commits (e.g., here).

Also, I've been running FlareSolverr since I created the PR, which was exactly a month ago.

The issue you are experiencing exists since the update of Chromium to version 127, and is actually related to the Selenium's Python library getting stuck at the creation of the base class here. I've created https://github.com/NixOS/nixpkgs/pull/332066 to update Selenium in Nixpkgs, but the issue still persists at that build. Switching back to Chromium 126 worked fine at the time of testing.

I haven't really investigated it beyond that point, yet. That said, I'm pretty sure it's an upstream issue.

[akho]

Thank you both. Sorry for not trusting the process.

[paveloom]

A tiny bit of insight after applying the changes from https://github.com/SeleniumHQ/selenium/pull/14354:

$ GLOBAL_DEFAULT_TIMEOUT=5 python src/flaresolverr.py
2024-08-07 22:45:12 INFO     FlareSolverr 3.3.21
2024-08-07 22:45:12 INFO     Testing web browser installation...
2024-08-07 22:45:12 INFO     Platform: Linux-6.10.2-x86_64-with-glibc2.39
2024-08-07 22:45:12 INFO     Chrome / Chromium path: /nix/store/qljylnjpb42m42rw5y8y68xd87jrlnd5-chromium-127.0.6533.72/bin/chromium
2024-08-07 22:45:12 INFO     Chrome / Chromium major version: 127
2024-08-07 22:45:12 INFO     Launching web browser...
2024-08-07 22:45:28 ERROR    Error starting Chrome: HTTPConnectionPool(host='localhost', port=54891): Read timed out. (read timeout=5.0)
Traceback (most recent call last):
  File "/home/paveloom/Playground/flaresolverr-env/FlareSolverr/src/utils.py", line 320, in get_user_agent
    driver = get_webdriver()
             ^^^^^^^^^^^^^^^
  File "/home/paveloom/Playground/flaresolverr-env/FlareSolverr/src/utils.py", line 223, in get_webdriver
    return driver
           ^^^^^^
UnboundLocalError: cannot access local variable 'driver' where it is not associated with a value

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/paveloom/Playground/flaresolverr-env/FlareSolverr/src/flaresolverr.py", line 108, in <module>
    flaresolverr_service.test_browser_installation()
  File "/home/paveloom/Playground/flaresolverr-env/FlareSolverr/src/flaresolverr_service.py", line 73, in test_browser_installation
    user_agent = utils.get_user_agent()
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paveloom/Playground/flaresolverr-env/FlareSolverr/src/utils.py", line 326, in get_user_agent
    raise Exception("Error getting browser User-Agent. " + str(e))
Exception: Error getting browser User-Agent. cannot access local variable 'driver' where it is not associated with a value

[paveloom]

See also https://github.com/FlareSolverr/FlareSolverr/issues/1318.

[Redhawk18]

I'm actually getting the same issue as well

[Redhawk18]

Use this docker container if your having a problem with this, how they handle flaresolverr is kinda embarrassing. But use this docker tag until they fix it.

https://github.com/FlareSolverr/FlareSolverr/pull/1300#issuecomment-2381408420

[lunik1]

I have this too for the module, but the flaresolverr/flaresolverr container is working.

[ostrolucky]

Can't we pin the nix package to older chromium version, until flaresolverr works with newer one? Better than having broken package.

[paveloom]

@ostrolucky Unfortunately, there is no easy way to "pin" it in nixpkgs. One would have to maintain a derivation for an older version of the browser. You can pin nixpkgs itself in your flake, though, and pull chromium from there.

[Redhawk18]

Libraries are often pinned to major version, maybe the same could be done with chrome? A lot of programs use chromium in some way so maybe this could be an ideal solution.

[Redhawk18]

However maybe pinned is the wrong name, they have many packages of different versions.