search

Second-Hand-Friends / kleinanzeigen-bot

A dilligent command line tool to publish ads on kleinanzeigen.de
GNU Affero General Public License v3.0
222 stars 46 forks source link

[BUG] JSONDecodeError: Expecting ',' delimiter: line 1 column 17 (char 16) #318

Closed dbcesar closed 4 months ago

dbcesar commented 4 months ago

✔️ Expected Behaviour

I expected to download all ads.

🐞 Actual Behaviour

It fails with JSONDecodeError: Expecting ',' delimiter: line 1 column 17 (char 16) on the first ad.

📋 Steps to Reproduce

Download the executable or install it from source.

Run pdm run app download or kleinanzeigen-bot download

It could be the issue happens with a particular ad, so here the link to it: https://www.kleinanzeigen.de/s-anzeige/babybodys-von-carter-s-groesse-40-46/2809810622-22-20

📺 What browsers are you seeing the problem on? (if applicable)

Chrome

💻 What operating systems are you seeing the problem on? (if applicable)

Linux

📃 Relevant log output (if applicable)

Start fetch task for your unsaved ads! [DEBUG] -> Opening [https://www.kleinanzeigen.de/s-anzeige/babybodys-von-carter-s-groesse-40-46/2809810622-22-20]... [INFO] ... pausing for 1740 ms ... [INFO] Deleting current folder of ad... [INFO] New directory for ad created at downloaded-ads/ad_2809810622. [INFO] Extracting information from ad with title "Babybodys von Carter's Größe 40/46" [DEBUG] Closing Browser session... [INFO] terminated browser with pid 391121 successfully [ERROR] Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/home/diego/workspace/kleinanzeigen-bot/kleinanzeigen-bot/src/kleinanzeigen_bot/main.py", line 9, in kleinanzeigen_bot.main(sys.argv) File "/home/diego/workspace/kleinanzeigen-bot/kleinanzeigen-bot/src/kleinanzeigen_bot/init.py", line 864, in main nodriver.loop().run_until_complete(bot.run(args)) File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/home/diego/workspace/kleinanzeigen-bot/kleinanzeigen-bot/src/kleinanzeigen_bot/init.py", line 114, in run await self.download_ads() File "/home/diego/workspace/kleinanzeigen-bot/kleinanzeigen-bot/src/kleinanzeigen_bot/init.py", line 823, in download_ads await ad_extractor.download_ad(ad_id) File "/home/diego/workspace/kleinanzeigen-bot/kleinanzeigen-bot/src/kleinanzeigen_bot/extract.py", line 49, in download_ad info = await self._extract_ad_page_info(new_base_dir, ad_id) File "/home/diego/workspace/kleinanzeigen-bot/kleinanzeigen-bot/src/kleinanzeigen_bot/extract.py", line 236, in _extract_ad_page_info info['special_attributes'] = await self._extract_special_attributes_from_ad_page() File "/home/diego/workspace/kleinanzeigen-bot/kleinanzeigen-bot/src/kleinanzeigen_bot/extract.py", line 295, in _extract_special_attributes_from_ad_page special_attributes = json.loads(special_attributes_fixed_str) File "/usr/lib/python3.10/json/init.py", line 346, in loads return _default_decoder.decode(s) File "/usr/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.10/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 17 (char 16)

[INFO] terminated browser with pid 391121 successfully successfully removed temp profile /tmp/uc_b2zs3r1m

Code of Conduct

dbcesar commented 4 months ago

I added some debug print here and I found this output for the special_attributes_str and special_attributes_fixed_str, respectively. 

[INFO] versand_s:t|color_s:bunt|groesse_s:44|condition_s:new|type_s:shirts_tops|art_s:unisex
[INFO] Attributes fixed
[INFO] {"versand_s":"t"|"color_s":"bunt"|"groesse_s":"44"|"condition_s":"new"|"type_s":"shirts_tops"|"art_s":"unisex"}

Which I fixed with adding just after the regex substitution the following: special_attributes_fixed_str = special_attributes_fixed_str.replace("|", ",")

dbcesar commented 4 months ago

Ok, my fix is enough to download the items, but publishing them is not working. Here I'm not sure if it's something wrong with the importing mechanism (essentially, my changes) or if there's something else wrong.

The issue now is that it fails to set special attributes:

[DEBUG] Found 6 special attributes
[DEBUG] Setting special attribute [versand_s] to [t]...
[DEBUG] Attribute field 'versand_s' is not of kind dropdown, trying to input as plain text...
[DEBUG] Attribute field 'versand_s' is not of kind plain text, trying to input as radio button...
[DEBUG] Attribute field 'versand_s' is not of kind radio button.

...a lot of backtrace... 

  File "/home/diego/workspace/kleinanzeigen-bot/kleinanzeigen-bot/src/kleinanzeigen_bot/__init__.py", line 512, in publish_ad
    await self.__set_category(ad_file, ad_cfg)
  File "/home/diego/workspace/kleinanzeigen-bot/kleinanzeigen-bot/src/kleinanzeigen_bot/__init__.py", line 698, in __set_category
    raise TimeoutError(f"Failed to set special attribute [{special_attribute_key}]") from ex

TimeoutError: Failed to set special attribute [versand_s]
dbcesar commented 4 months ago

It seems the issue I've just had is related to: https://github.com/Second-Hand-Friends/kleinanzeigen-bot/issues/281

Raketenphysik commented 4 months ago

Confirmed! Same issue on my side. Let me know if any further information would be helpful.

julius-gun commented 4 months ago

same issue here

provinzio commented 4 months ago

@sebthom This issue is resolved by the merged PR :)

dbcesar commented 4 months ago

Thanks @provinzio!