[BUG] Update fails unreproducably in 40 - 60% of cases with messages "Invalid search result range" and "DOM agent hasn't been enabled"

knipknap commented 8 months ago

✔️ Expected Behaviour

Publishing ads should work without timeout and without tracebacks.

🐞 Actual Behaviour

When publishing updates I am seeing that it fails in about 30% to 50% of attempts. Meaning that it stalls and eventually the bot exits with a timeout. In the browser I see that some fields are not filled, for example the title or the price. The output contains many such lines:

args ({'code': -32000, 'message': 'Invalid search result range'},) kwargs {}
args ({'code': -32000, 'message': "DOM agent hasn't been enabled"},) kwargs {}

If I try to republish the same ad, eventually it always works. But it's quite unreliable. Most of the time it just fails with a timeout, but sometimes also with this traceback:

[ERROR] Traceback (most recent call last):
  File "kleinanzeigen_bot/__main__.py", line 9, in <module>
  File "kleinanzeigen_bot/__init__.py", line 828, in main
  File "asyncio/base_events.py", line 684, in run_until_complete
  File "kleinanzeigen_bot/__init__.py", line 89, in run
  File "kleinanzeigen_bot/__init__.py", line 480, in publish_ads
  File "kleinanzeigen_bot/__init__.py", line 526, in publish_ad
  File "kleinanzeigen_bot/__init__.py", line 714, in __set_shipping_options
  File "kleinanzeigen_bot/web_scraping_mixin.py", line 341, in web_click
  File "nodriver/core/element.py", line 392, in click
  File "nodriver/core/element.py", line 799, in flash
AttributeError: 'NoneType' object has no attribute 'center'

📋 Steps to Reproduce

Try to publish any ad. Doesn't matter if it's new or an update.
Note that it often fails.

📺 What browsers are you seeing the problem on? (if applicable)

Chrome

💻 What operating systems are you seeing the problem on? (if applicable)

Linux

📃 Relevant log output (if applicable)


| | _| | ___(_)_ __   __ _ _ __  _______(_) __ _  ___ _ __      | |__   ___ | |_
| |/ / |/ _ \ | '_ \ / _` | '_ \|_  / _ \ |/ _` |/ _ \ '_ \ ____| '_ \ / _ \| __|
|   <| |  __/ | | | | (_| | | | |/ /  __/ | (_| |  __/ | | |____| |_) | (_) | |_
|_|\_\_|\___|_|_| |_|\__,_|_| |_/___\___|_|\__, |\___|_| |_|    |_.__/ \___/ \__|
                                           |___/
                         https://github.com/Second-Hand-Friends/kleinanzeigen-bot

[INFO] Logging to [/home/xyz/Documents/Unterlagen/Ebay/aa-klbot/kleinanzeigen-bot.log]...
[INFO] App version: 2024+d7fec9e
[INFO] Python version: 3.12.1 (main, Dec  8 2023, 05:40:51) [GCC 11.4.0]
[INFO] Loading config from [/home/xyz/Documents/Unterlagen/Ebay/aa-klbot/config.yaml]...
[INFO]  -> found 146 categories
[INFO] Searching for ad config files...
[INFO]  -> found 24 ad config files
[INFO] Start fetch task for the ad(s) with the id(s):
[INFO] 2708239090
[...loading ads removed...]
[INFO] Creating Browser session...
[INFO]  -> Chrome binary location: /usr/bin/google-chrome
[INFO] Using existing browser process at 127.0.0.1:9222
[INFO] New Browser session is ws://127.0.0.1:9222/devtools/browser/e587bae3-be02-4f3d-8093-7a366b4167cd
[INFO] Checking if already logged in...
[INFO] Already logged in as [REMOVED]. Skipping login.
[INFO] Processing 1/1: '12V Fan 7x7x2,5cm' from [.../ad_2680531550.yaml]...
args ({'code': -32000, 'message': 'Invalid search result range'},) kwargs {}
args ({'code': -32000, 'message': "DOM agent hasn't been enabled"},) kwargs {}
args ({'code': -32000, 'message': 'Invalid search result range'},) kwargs {}
args ({'code': -32000, 'message': "DOM agent hasn't been enabled"},) kwargs {}
args ({'code': -32000, 'message': 'Invalid search result range'},) kwargs {}
args ({'code': -32000, 'message': "DOM agent hasn't been enabled"},) kwargs {}
[INFO] Deleting ad '12V Fan 7x7x2,5cm' if already present...
[INFO] Publishing ad '12V Fan 7x7x2,5cm'...
[INFO]  ... pausing for 2268 ms ...
[INFO]  ... pausing for 2479 ms ...
[INFO]  ... pausing for 2207 ms ...
[INFO]  ... pausing for 2388 ms ...
[INFO]  ... pausing for 1632 ms ...
args ({'code': -32000, 'message': 'Invalid search result range'},) kwargs {}
args ({'code': -32000, 'message': "DOM agent hasn't been enabled"},) kwargs {}
args ({'code': -32000, 'message': 'Invalid search result range'},) kwargs {}
args ({'code': -32000, 'message': "DOM agent hasn't been enabled"},) kwargs {}
args ({'code': -32000, 'message': 'Invalid search result range'},) kwargs {}
args ({'code': -32000, 'message': "DOM agent hasn't been enabled"},) kwargs {}
args ({'code': -32000, 'message': 'Invalid search result range'},) kwargs {}
args ({'code': -32000, 'message': "DOM agent hasn't been enabled"},) kwargs {}
args ({'code': -32000, 'message': 'Invalid search result range'},) kwargs {}
args ({'code': -32000, 'message': "DOM agent hasn't been enabled"},) kwargs {}
[INFO]  ... pausing for 2481 ms ...
[INFO]  ... pausing for 2349 ms ...
[INFO]  ... pausing for 2238 ms ...
[INFO]  -> found 1 image
[INFO]  -> uploading image [.../ad_2680531550__img1.jpg]
[INFO]  ... pausing for 2161 ms ...
[ERROR] TimeoutError: Condition not met within 20 seconds
[68124] Failed to execute script '__main__' due to unhandled exception!
successfully removed temp profile /tmp/uc_f1qs0mf1

Relevant config parts:

browser:
  arguments:
    - --remote-debugging-port=9222
  binary_location: /usr/bin/google-chrome
  extensions: []
  use_private_window: false
  user_data_dir: '/home/xyz/.config/google-chrome'
  profile_name: 'Profile 2'

Code of Conduct

[X] I agree to follow this project's Code of Conduct

provinzio commented 6 months ago

Is this still an issue for you? I recently published quite a few ads and haven't seen this error.

My browser config looks like this

# browser configuration
browser:
  # https://peter.sh/experiments/chromium-command-line-switches/
  arguments:
    # https://stackoverflow.com/a/50725918/5116073
    - --disable-dev-shm-usage
    - --no-sandbox
    # --headless
    # --start-maximized
  binary_location: # path to custom browser executable, if not specified will be looked up on PATH
  extensions: [] # a list of .crx extension files to be loaded
  use_private_window: true
  user_data_dir: "" # see https://github.com/chromium/chromium/blob/main/docs/user_data_dir.md
  profile_name: "...profile name..."

How do you installed the application and how do you run it?

sebthom commented 6 months ago

Closing issue for now. If the issue still persists with the latest version, feel free to re-open the ticket and provide an updated error message/stacktrace.

Second-Hand-Friends / kleinanzeigen-bot