search

Second-Hand-Friends / kleinanzeigen-bot

A dilligent command line tool to publish ads on kleinanzeigen.de
GNU Affero General Public License v3.0
222 stars 47 forks source link

[BUG] No HTML element found #328

Closed markus2801 closed 3 days ago

markus2801 commented 3 months ago

✔️ Expected Behaviour

Reuploading ads with "kleinanzeigen-bot publish --ads=all --verbose"

🐞 Actual Behaviour

Starts with opening browser, logging in, then crashes while filling the form for the first ad

📋 Steps to Reproduce

Try to republish an ad on Windows & Chrome

📺 What browsers are you seeing the problem on? (if applicable)

Chrome, Microsoft Edge

💻 What operating systems are you seeing the problem on? (if applicable)

Windows

📃 Relevant log output (if applicable)

2024-08-27 19:54:29,485 [INFO] App version: 2024+3154005
2024-08-27 19:54:29,485 [INFO] Python version: 3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)]
2024-08-27 19:54:29,487 [DEBUG] Loading [kleinanzeigen_bot.resources.config_defaults.yaml]...
2024-08-27 19:54:29,491 [INFO] Loading config from [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\config.yaml]...
2024-08-27 19:54:29,495 [DEBUG] Loading categories from [kleinanzeigen_bot.resources.categories.yaml]...
2024-08-27 19:54:29,515 [INFO]  -> found 145 categories
2024-08-27 19:54:29,515 [INFO] Searching for ad config files...
2024-08-27 19:54:29,523 [INFO]  -> found 7 ad config files
2024-08-27 19:54:29,523 [DEBUG] Loading [kleinanzeigen_bot.resources.ad_fields.yaml]...
2024-08-27 19:54:29,526 [INFO] Loading ad from [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\downloaded-ads\ad_2840581133\ad_2840581133.yaml]...
2024-08-27 19:54:29,530 [INFO]  -> SKIPPED: ad [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\downloaded-ads\ad_2840581133\ad_2840581133.yaml] was last published 0 days ago. republication is only required every 7 days
2024-08-27 19:54:29,530 [INFO] Loading ad from [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\downloaded-ads\ad_2840581693\ad_2840581693.yaml]...
2024-08-27 19:54:29,533 [INFO]  -> SKIPPED: ad [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\downloaded-ads\ad_2840581693\ad_2840581693.yaml] was last published 0 days ago. republication is only required every 7 days
2024-08-27 19:54:29,534 [INFO] Loading ad from [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\downloaded-ads\ad_2840582281\ad_2840582281.yaml]...
2024-08-27 19:54:29,540 [INFO] Loading ad from [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\downloaded-ads\ad_2840583653\ad_2840583653.yaml]...
2024-08-27 19:54:29,545 [INFO] Loading ad from [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\downloaded-ads\ad_2840584365\ad_2840584365.yaml]...
2024-08-27 19:54:29,550 [INFO] Loading ad from [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\downloaded-ads\ad_2840585016\ad_2840585016.yaml]...
2024-08-27 19:54:29,565 [INFO] Loading ad from [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\downloaded-ads\ad_2840585812\ad_2840585812.yaml]...
2024-08-27 19:54:29,571 [INFO] Loaded 5 ads
2024-08-27 19:54:29,572 [INFO] Creating Browser session...
2024-08-27 19:54:29,700 [INFO]  -> Chrome binary location: C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe
2024-08-27 19:54:29,702 [INFO] -> Setting chrome prefs [C:\Users\marku\AppData\Local\Temp\uc_jatyjjrg\Default\Preferences]...
2024-08-27 19:54:29,704 [INFO] starting
    executable :C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe

arguments:
--remote-allow-origins=*
    --no-first-run
    --no-service-autorun
    --no-default-browser-check
    --homepage=about:blank
    --no-pings
    --password-store=basic
    --disable-infobars
    --disable-breakpad
    --disable-component-update
    --disable-backgrounding-occluded-windows
    --disable-renderer-backgrounding
    --disable-background-networking
    --disable-dev-shm-usage
    --disable-features=IsolateOrigins,site-per-process
    --disable-session-crashed-bubble
    --user-data-dir=C:\Users\marku\AppData\Local\Temp\uc_jatyjjrg
    --disable-features=IsolateOrigins,site-per-process
    --disable-session-crashed-bubble
    --disable-crash-reporter
    --disable-domain-reliability
    --disable-sync
    --no-experiments
    --disable-search-engine-choice-screen
    --disable-features=MediaRouter
    --use-mock-keychain
    --test-type
    --host-resolver-rules="MAP connect.facebook.net 127.0.0.1, MAP securepubads.g.doubleclick.net 127.0.0.1, MAP www.googletagmanager.com 127.0.0.1"
    -inprivate
    --remote-debugging-host=127.0.0.1
    --remote-debugging-port=54958
2024-08-27 19:54:29,710 [INFO] created process with pid 26852 
2024-08-27 19:54:30,317 [INFO] enabling autodiscover targets
2024-08-27 19:54:30,338 [INFO] New Browser session is ws://127.0.0.1:54958/devtools/browser/a5ab156e-4ae1-4167-8039-df268ea40395
2024-08-27 19:54:30,338 [INFO] Checking if already logged in...
2024-08-27 19:54:30,339 [DEBUG]  -> Opening [https://www.kleinanzeigen.de]...
2024-08-27 19:54:37,433 [INFO] Opening login page...
2024-08-27 19:54:37,434 [DEBUG]  -> Opening [https://www.kleinanzeigen.de/m-einloggen.html?targetUrl=/]...
2024-08-27 19:54:42,697 [INFO] Logging in as [XXXXXXXXXX]...
2024-08-27 19:54:42,871 [DEBUG]  ... pausing for 1402 ms ...
2024-08-27 19:54:44,490 [DEBUG]  ... pausing for 1085 ms ...
2024-08-27 19:54:45,711 [INFO]  ... pausing for 2426 ms ...
2024-08-27 19:54:52,359 [INFO] Handling GDPR disclaimer...
2024-08-27 19:55:02,960 [INFO] Processing 1/5: 'Druckregelventil CommonRail Ford Peugeot Citroen VEMO V22-11-0017' from [C:\Users\marku\Documents\Kleinanzeigen\Kleinanzeigen Bot\downloaded-ads\ad_2840582281\ad_2840582281.yaml]...
2024-08-27 19:55:05,126 [INFO] Deleting ad 'Druckregelventil CommonRail Ford Peugeot Citroen VEMO V22-11-0017' if already present...
2024-08-27 19:55:05,126 [DEBUG]  -> Opening [https://www.kleinanzeigen.de/m-meine-anzeigen.html]...
2024-08-27 19:55:05,127 [DEBUG]   => skipping, [https://www.kleinanzeigen.de/m-meine-anzeigen.html] is already open
2024-08-27 19:55:05,265 [DEBUG]  -> HTTP POST [https://www.kleinanzeigen.de/m-anzeigen-loeschen.json?ids=2840582281]...
2024-08-27 19:55:05,465 [INFO]  ... pausing for 2337 ms ...
2024-08-27 19:55:07,811 [INFO] Publishing ad 'Druckregelventil CommonRail Ford Peugeot Citroen VEMO V22-11-0017'...
2024-08-27 19:55:07,812 [DEBUG]  -> effective ad meta:
2024-08-27 19:55:07,817 [DEBUG]  -> Opening [https://www.kleinanzeigen.de/p-anzeige-aufgeben-schritt2.html]...
2024-08-27 19:55:10,582 [INFO]  ... pausing for 1778 ms ...
2024-08-27 19:55:12,492 [INFO]  ... pausing for 1755 ms ...
2024-08-27 19:55:14,391 [INFO]  ... pausing for 2039 ms ...
2024-08-27 19:55:16,573 [DEBUG]  ... pausing for 1451 ms ...
2024-08-27 19:55:18,145 [DEBUG]  -> Opening [https://www.kleinanzeigen.de/p-kategorie-aendern.html#?path=210l1892/223l1892]...
2024-08-27 19:55:18,667 [DEBUG]  ... pausing for 1072 ms ...
2024-08-27 19:55:19,754 [DEBUG] Found 1 special attributes
2024-08-27 19:55:19,755 [DEBUG] Setting special attribute [art_s] to [ersatz_reparaturteile]...
2024-08-27 19:55:30,622 [DEBUG] Attribute field 'art_s' is not of kind dropdown, trying to input as plain text...
2024-08-27 19:55:35,880 [DEBUG] Attribute field 'art_s' is not of kind plain text, trying to input as radio button...
2024-08-27 19:55:41,169 [DEBUG] Attribute field 'art_s' is not of kind radio button.
2024-08-27 19:55:41,170 [DEBUG] Closing Browser session...
2024-08-27 19:55:41,178 [INFO] terminated browser with pid 26852 successfully
2024-08-27 19:55:41,183 [ERROR] Traceback (most recent call last):
  File "kleinanzeigen_bot\__init__.py", line 725, in __set_category
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 545, in web_select
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 284, in web_await
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 277, in web_await
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 310, in web_check
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 392, in web_find
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 285, in web_await
TimeoutError: No HTML element found using XPath '//select[contains(@id, 'art_s')]' within 5 seconds.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "kleinanzeigen_bot\__init__.py", line 729, in __set_category
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 446, in web_input
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 363, in web_find
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 285, in web_await
TimeoutError: No HTML element found with ID 'art_s' within 5 seconds.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "kleinanzeigen_bot\__init__.py", line 733, in __set_category
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 341, in web_click
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 392, in web_find
  File "kleinanzeigen_bot\web_scraping_mixin.py", line 285, in web_await
TimeoutError: No HTML element found using XPath '//*[contains(@id, 'art_s')]/option[@value='ersatz_reparaturteile']' within 5 seconds.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "kleinanzeigen_bot\__main__.py", line 9, in <module>
  File "kleinanzeigen_bot\__init__.py", line 902, in main
  File "asyncio\base_events.py", line 684, in run_until_complete
  File "kleinanzeigen_bot\__init__.py", line 93, in run
  File "kleinanzeigen_bot\__init__.py", line 484, in publish_ads
  File "kleinanzeigen_bot\__init__.py", line 516, in publish_ad
  File "kleinanzeigen_bot\__init__.py", line 736, in __set_category
TimeoutError: Failed to set special attribute [art_s]

2024-08-27 19:55:41,185 [INFO] terminated browser with pid 26852 successfully

Code of Conduct

sebthom commented 3 days ago

Please download the latest version. Then ensure category is set to "210/223/ersatz_reparaturteile" or "Auto, Rad & Boot > Autoteile & Reifen > Ersatz- & Reparaturteile" and ensure that under special_attributes no art_s entry exists.