[Bug]: WAS - Wolfsburger Abfallwirtschaft u. Straßenreinigung did no longer collects the waste appointments

dp20eic commented 5 months ago

I Have A Problem With:

A specific source

What's Your Problem

Since February 8th, 2024, I no longer see any appointments displayed in the calendar and therefore no more appointments are shown to me in the automations.

my configuration

# Abfallkallender
waste_collection_schedule:
  sources:
    - name: was_wolfsburg_de
      args:
        city: my district
        street: my street
  fetch_time: "04:00"
  day_switch_time: "10:00"
sensor:
  # ------- Garbage Collection -------
  # next collection
  - platform: waste_collection_schedule
    name: "Nächste Abholung"
    value_template: '{{value.types|join(", ")}}{% if value.daysTo == 0 %} Heute{% elif value.daysTo == 1 %} Morgen{% else %} in {{valu
  # Nächste Bioabfall Abholung
  - platform: waste_collection_schedule
    name: "Bioabfall"
    value_template: '{{value.types|join(", ")}}{% if value.daysTo == 0 %} Heute{% elif value.daysTo == 1 %} Morgen{% else %} in {{valu
    types:
      - Bioabfall
  # Nächste gelber-Sack Abholung
  - platform: waste_collection_schedule
    name: "Gelber Sack"
    value_template: '{{value.types|join(", ")}}{% if value.daysTo == 0 %} Heute{% elif value.daysTo == 1 %} Morgen{% else %} in {{valu
    types:
      - Gelber Sack
  # Nächste Papierabfall Abholung
  - platform: waste_collection_schedule
    name: "Altpapier"
    value_template: '{{value.types|join(", ")}}{% if value.daysTo == 0 %} Heute{% elif value.daysTo == 1 %} Morgen{% else %} in {{valu
    types:
      - Altpapier
  # Nächste Restabfall-Abholung
  - platform: waste_collection_schedule
    name: "Restabfall "
    value_template: '{{value.types|join(", ")}}{% if value.daysTo == 0 %} Heute{% elif value.daysTo == 1 %} Morgen{% else %} in {{valu
    types:
      - Restabfall

Source (if relevant)

No response

Logs

No response

Relevant Configuration

I saw that WAS changed the website, that could be the reason that the data can no longer be found.

Checklist Source Error

[X] Use the example parameters for your source (often available in the documentation) (don't forget to restart Home Assistant after changing the configuration)
[X] Checked that the website of your service provider is still working
[X] Tested my attributes on the service provider website (if possible)
[X] I have tested with the latest version of the integration (master) (for HACS in the 3 dot menu of the integration click on "Redownload" and choose master as version)

Checklist Sensor Error

[X] Checked in the Home Assistant Calendar tab if the event names match the types names (if types argument is used)

Required

[X] I have searched past (closed AND opened) issues to see if this bug has already been reported, and it hasn't been.
[X] I understand that people give their precious time for free, and thus I've done my very best to make this problem as easy as possible to investigate.

ppk2000 commented 5 months ago

Hello,

Yes, the WAS website has changed. Here are the new URL:

Gelber Sack: r = requests.post( "https://was-wolfsburg.de/wp-content/themes/astra-child/search_gelbersack/search.php", data={"loc": "MY_CITY"}, headers={"content-type": "application/x-www-form-urlencoded; charset=UTF-8"}, )

Bioabfall,Restabfall,Altpapier: r = requests.post( "https://was-wolfsburg.de/wp-content/themes/astra-child/search_restabfall/search.php", data={"loc": "MY_STREET"}, headers={"content-type": "application/x-www-form-urlencoded; charset=UTF-8"}, )

dp20eic commented 5 months ago

Yes, the WAS website has changed. Here are the new URL:

Hi, ok, I try to change it local at was_wolfsburg_de.py and for Gelber Sack it works, but for the rest not.

VG

ppk2000 commented 5 months ago

You can do a list like the Gelber Sack.

I´ve done :

r = requests.post( "https://was-wolfsburg.de/wp-content/themes/astra-child/search_restabfall/search.php", data={"loc": "YOUR_STREET"}, headers={"content-type": "application/x-www-form-urlencoded; charset=UTF-8"}, )

    entries = []
    match = re.findall(r"(\d{2})\.(\d{2})\.(\d{4})", r.text)
    match2 = re.findall(r"Bioabfall|Restabfall|Altpapier", r.text)
    myit = iter(match2)
    for m in match:
        date = datetime.date(day=int(m[0]), month=int(m[1]), year=int(m[2]))
        entries.append(Collection(date, next(myit)))
   return entries

VG

dp20eic commented 5 months ago

Hi,

Yes, if I delete "entries = []" it works, but I need to change the city and street in the main source and couldn't configure it in "configuration.yaml, ..." like before and the icons are no more correctly. I'm not good at programming, but I think the source code needs to be completely rewritten so that everything can be used as before. Your preparatory work can contribute very well to this.

VG

Dark-Tower-Coder commented 5 months ago

I would also be very happy about an adjustment.

boehser-enkel commented 4 months ago

Me too (also where is the ics for the dates?)

dp20eic commented 4 months ago

Me too (also where is the ics for the dates?)

Good morning, I don't understand that, it has nothing to do with the integration, the integration queries the data directly on the WAS page. If you want to save ICS data into a calendar and read it out, then WAS has to have an ICS calendar provide, or you enter the appointments yourself into a calendar, and then you have to make your adjustments in the .yaml, which comes from the calendar you created. As I know, there is no ICS file anymore you can download from the WAS page.

VG Bernd

boehser-enkel commented 4 months ago

Me too (also where is the ics for the dates?)

Good morning, I don't understand that, it has nothing to do with the integration, the integration queries the data directly on the WAS page. If you want to save ICS data into a calendar and read it out, then WAS has to have an ICS calendar provide, or you enter the appointments yourself into a calendar, and then you have to make your adjustments in the .yaml, which comes from the calendar you created. As I know, there is no ICS file anymore you can download from the WAS page.

VG Bernd

The ics is coming

dp20eic commented 4 months ago

Moin,

Today I was offered an update of the hacs_waste_collection_schedule, unfortunately I found that the error was not fixed :( Ok, I change it again at original Source and wait if it changed next update.

VG

boehser-enkel commented 4 months ago

Moin,

Today I was offered an update of the hacs_waste_collection_schedule, unfortunately I found that the error was not fixed :) Ok, I change it again at original Source and wait if it changed next update.

VG

Maybe i have an error in there. Could you upload your complete was_wolfsburg_de.py please?

dp20eic commented 4 months ago

Maybe i have an error in there. Could you upload your complete was_wolfsburg_de.py please?

Good morning,

I took the code from @ppk2000 and just adapted it to my street and city, so where ...={"loc": "MY_CITY"} is written, the district comes in, e.g. E.g.: Barnstorf or Laagberg. With ...={"loc": "YOUR_STREET"} then just the appropriate street without house number, e.g. E.g.: Barnstorfer Weg or Heinrich-Nordhoff-Straße. As I wrote above, the entries = [] have to be removed for leftover, organic or waste paper, then my code looks like this

import datetime
import re

import requests
from waste_collection_schedule import Collection  # type: ignore[attr-defined]
from waste_collection_schedule.service.ICS import ICS

TITLE = "Wolfsburger Abfallwirtschaft und Straßenreinigung"
DESCRIPTION = "Source for waste collections for WAS-Wolfsburg, Germany."
URL = "https://was-wolfsburg.de"
TEST_CASES = {
    "Barnstorf": {"city": "Barnstorf", "street": "Bahnhofspassage"},
    "Sülfeld": {"city": "Sülfeld", "street": "Bärheide"},
}

ICON_MAP = {
    "Gelber Sack": "mdi:recycle",
    "Bioabfall": "mdi:leaf",
    "Restabfall": "mdi:trash-can",
    "Altpapier": "mdi:file-document-outline",
}

CHARACTER_MAP = {
    ord("ü"): "u",
    ord("ö"): "o",  # doesn't appear to be needed
    ord("ä"): "a",  # doesn't appear to be needed
}

class Source:
    def __init__(self, city: str, street: str):
        self._city = city.translate(CHARACTER_MAP)
        self._street = street.translate(CHARACTER_MAP)
        self._ics = ICS()

    def fetch(self):
        # fetch "Gelber Sack"
        args = {"g": self._city}
        r = requests.post(
            "https://was-wolfsburg.de/wp-content/themes/astra-child/search_gelbersack/search.php",
            data={"loc": "MY_CITY"},
            headers={"content-type": "application/x-www-form-urlencoded; charset=UTF-8"},
        )
        entries = []
        match = re.findall(r"(\d{2})\.(\d{2})\.(\d{4})", r.text)
        for m in match:
            date = datetime.date(day=int(m[0]), month=int(m[1]), year=int(m[2]))
            entries.append(
                Collection(date, "Gelber Sack", icon=ICON_MAP["Gelber Sack"])
            )

        # fetch remaining collections
        args = {"k": self._street}
        r = requests.post(
            "https://was-wolfsburg.de/wp-content/themes/astra-child/search_restabfall/search.php",
            data={"loc": "MY_STREET"},
            headers={"content-type": "application/x-www-form-urlencoded; charset=UTF-8"},
        )
        match = re.findall(r"(\d{2})\.(\d{2})\.(\d{4})", r.text)
        match2 = re.findall(r"Bioabfall|Restabfall|Altpapier", r.text)
        myit = iter(match2)
        for m in match:
            date = datetime.date(day=int(m[0]), month=int(m[1]), year=int(m[2]))
            entries.append(Collection(date, next(myit)))
        return entries

VG

dp20eic commented 3 months ago

Good morning,

I would like to bring this error up again, it would be nice if a developer would take a look at it and fix it.

With kind regards Bernd

FelixArndt commented 3 months ago

Yeah, would be awesome if someone can fix this :-)

ppk2000 commented 3 months ago

The URLs works fine for me. Maybe the problem are last changes in the integration??? Sorry but I use my own implementation and i can´t help.

slashphotos commented 1 month ago

Hey!

Is there any update to this issue? I can't get it to work. Neither the master nor this code snippet are working in homeassistant or in my python environment. If I try to access the URL by hand, I get {"status":true,"count":0,"string":null} as a response.

Kind regards

dp20eic commented 1 month ago

Moin,

If I try to access the URL by hand, I get Das ist normal, da Du ja kein Ziel mit angegeben hast, im Python Skript, kommen ja noch
data={"loc": "MY_CITY"},
headers={"content-type": "application/x-www-form-urlencoded; charset=UTF-8"},
Oder
data={"loc": "MY_STREET"},
headers={"content-type": "application/x-www-form-urlencoded; charset=UTF-8"},
Die, die Suche vervollständigen.

Sag mal, wo es klemmt, dann kann ich vielleicht unterstützen.

VG Bernd

mampfes / hacs_waste_collection_schedule