search

Second-Hand-Friends / kleinanzeigen-bot

A dilligent command line tool to publish ads on kleinanzeigen.de
GNU Affero General Public License v3.0
197 stars 43 forks source link

Update Categories - fixed pipeline #308

Open provinzio opened 3 months ago

provinzio commented 3 months ago

Issue #, if available: replaces https://github.com/Second-Hand-Friends/kleinanzeigen-bot/pull/274

Description of changes: fixed the pipeline of the other PR by converting the encoding of categories.yaml-file to utf-8

I removed old category.yaml file and replace with the complete category list, I scraped from kleinanzeigen.de. These should be all categories as of March 2024. I created a category tree therefore all leaf categories are included in the yaml-file (with name and path).

I haven't tested the categeories, just fixed the pipeline.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

sebthom commented 3 months ago

I don't think this works. Categories are currently expected to be a flat structure, not a tree.

I am not too big of a fan of the category update like this anyways as it may breaks existing ads.

provinzio commented 3 months ago

I have actually no idea how the categories work, but I just did some quick regex search an replace magic to flatten the structure.

Hope it helps.

provinzio commented 3 months ago

Ah ok, now I see your point... I'll reformat the categories.yaml

provinzio commented 3 months ago

Ok, I added all missing categories from https://github.com/Second-Hand-Friends/kleinanzeigen-bot/pull/274 at the bottom.

I haven't sorted them in, because that could be too much of a pain.

I was quite random when changing the keys for new duplicated entries. Please have look and tell me what you think about it.

Easter egg: Reciever has multiple different typos but is never written without typos (reciver, receiver)

Vel-San commented 2 months ago

@provinzio Good approach!

However, the bot is still unable to publish ads even with this PR. I am still getting random errors for Art e.g.

[ERROR] TimeoutError: Failed to set special attribute [art_s]

For category

special_attributes:
  art_s: sonstige
  condition_s: like_new
  versand_s: f

I think the problem is with the download option, where the category is not saved like the ones you have updated. It requires mapping for it to work properly.