Open exbachi opened 6 months ago
I can confirm the issue. f_id_kommune: 908 isn't working any longer, too. (also AWB Göppingen) Seems to do not fetch any data, due to missing parameter.
I can also confirm the issue, but the parameter timeperiod is completly different from the normal request. It seems to be a new qraphql based widget?!?
I can also confirm the issue, but the parameter timeperiod is completly different from the normal request. It seems to be a new qraphql based widget?!?
For me the parameter was part of the GET parameters directly in the URL like the other parameters, too
f_id_kommune 911 is also not working. AWB Göppingen as well.
Can confirm the same for f_id_kommune 913. @exbachi Tried to get it manually working but no luck. Can you post your current workaround? Thanks!
f_id_kommune: 902 is also not working. AWB Göppingen as well.
f_id_kommune: 898 is also not working. AWB Göppingen as well.
Can confirm the same for f_id_kommune 913. @exbachi Tried to get it manually working but no luck. Can you post your current workaround? Thanks!
I used the developer tools in chrome to get the request to download the ICS: https://api.abfall.io/?key=XXXX&mode=export&idhousenumber=XXXX&wastetypes=20,17,59,18,19,60&timeperiod=20240101-20241231&showinactive=false&type=ics
You can create your own URL here and get it with the dev tools: https://awb-gp.de/abfallabholung/abfuhrtermine Then you can use the URL as an ICS source as waste scheduler source
The wohle AWG Göttingen ist out of order ...
894 = Adelberg 895 = Aichelberg 896 = Albershausen 897 = Bad Boll 898 = Bad Ditzenbach 899 = Bad Überkingen 900 = Birenbach 901 = Böhmenkirch 902 = Börtlingen 903 = Deggingen 904 = Donzdorf 905 = Drackenstein 906 = Dürnau 907 = Ebersbach 908 = Eislingen 909 = Eschenbach 910 = Gammelshausen 911 = Geislingen 912 = Gingen 913 = Göppingen 914 = Gruibingen 915 = Hattenhofen 916 = Heiningen 917 = Hohenstadt 918 = Kuchen 919 = Lauterstein 920 = Mühlhausen 921 = Ottenbach 922 = Rechberghausen 923 = Salach 924 = Schlat 925 = Schlierbach 926 = Süßen 927 = Uhingen 928 = Wangen 929 = Wäschenbeuren 930 = Wiesensteig 931 = Zell u. A.
I used the developer tools in chrome to get the request to download the ICS: https://api.abfall.io/?key=XXXX&mode=export&idhousenumber=XXXX&wastetypes=20,17,59,18,19,60&timeperiod=20240101-20241231&showinactive=false&type=ics
You can create your own URL here and get it with the dev tools: https://awb-gp.de/abfallabholung/abfuhrtermine Then you can use the URL as an ICS source as waste scheduler source
Xes, but it is not a solution here in this case!
The wohle AWG Göttingen ist out of order ...
894 = Adelberg 895 = Aichelberg 896 = Albershausen 897 = Bad Boll 898 = Bad Ditzenbach 899 = Bad Überkingen 900 = Birenbach 901 = Böhmenkirch 902 = Börtlingen 903 = Deggingen 904 = Donzdorf 905 = Drackenstein 906 = Dürnau 907 = Ebersbach 908 = Eislingen 909 = Eschenbach 910 = Gammelshausen 911 = Geislingen 912 = Gingen 913 = Göppingen 914 = Gruibingen 915 = Hattenhofen 916 = Heiningen 917 = Hohenstadt 918 = Kuchen 919 = Lauterstein 920 = Mühlhausen 921 = Ottenbach 922 = Rechberghausen 923 = Salach 924 = Schlat 925 = Schlierbach 926 = Süßen 927 = Uhingen 928 = Wangen 929 = Wäschenbeuren 930 = Wiesensteig 931 = Zell u. A.
I used the developer tools in chrome to get the request to download the ICS: https://api.abfall.io/?key=XXXX&mode=export&idhousenumber=XXXX&wastetypes=20,17,59,18,19,60&timeperiod=20240101-20241231&showinactive=false&type=ics
You can create your own URL here and get it with the dev tools: https://awb-gp.de/abfallabholung/abfuhrtermine Then you can use the URL as an ICS source as waste scheduler source
Xes, but it is not a solution here in this case!
It is the workaround for fabssor to get the dates back working in HA, not a solution for the Problem
I investigated a little bit more and got abfall.io working again for Göppingen.
It seems to be that the API got changed at least for Göppingen. I tried my code for ALBA Berlin but this seems to be still using the old API. Also can be seen on the ALBA Berlin website
The funny thing is, the old API works still for Göppingen, but there are no EVENTS in the ICS file. This is always empty :(
I investigated a little bit more and got abfall.io working again for Göppingen. It seems to be that the API got changed at least for Göppingen. I tried my code for ALBA Berlin but this seems to be still using the old API. Also can be seen on the ALBA Berlin website....
@fabssor @Wilkware : Does that mean you got 'waste-collection-schedule' back to working, or just being able to export working data to ICS? For me the problem wasn't getting data into the HA calendar, this could be done manually, but my cards with the integration aren't working anymore, so in my opinion the API isn't working for Goeppingen anymore.
I got waste Collection Integration working again by migrating the abfall_io Python Script to the new API. I can later Share my Modifications.
Sadly the New API is Not supported by all communes. So my changes work for Göppingen but not for Berlin.
this is my modified python script which seems to be working and seems not to break the old API:
import datetime
import logging
import re
from html.parser import HTMLParser
import requests
from waste_collection_schedule import Collection # type: ignore[attr-defined]
from waste_collection_schedule.service.AbfallIO import SERVICE_MAP
from waste_collection_schedule.service.ICS import ICS
TITLE = "Abfall.IO / AbfallPlus"
DESCRIPTION = (
"Source for AbfallPlus.de waste collection. Service is hosted on abfall.io."
)
URL = "https://www.abfallplus.de"
COUNTRY = "de"
def EXTRA_INFO():
return [{"title": s["title"], "url": s["url"]} for s in SERVICE_MAP]
TEST_CASES = {
"Waldenbuch": {
"key": "8215c62763967916979e0e8566b6172e",
"f_id_kommune": 2999,
"f_id_strasse": 1087,
# "f_abfallarten": [50, 53, 31, 299, 328, 325]
},
"Landshut": {
"key": "bd0c2d0177a0849a905cded5cb734a6f",
"f_id_kommune": 2655,
"f_id_bezirk": 2655,
"f_id_strasse": 763,
# "f_abfallarten": [31, 17, 19, 218]
},
"Schoenmackers": {
"key": "e5543a3e190cb8d91c645660ad60965f",
"f_id_kommune": 3682,
"f_id_strasse": "3682adenauerplatz",
"f_id_strasse_hnr": "20417",
# "f_abfallarten": [691,692,696,695,694,701,700,693,703,704,697,699],
},
"Freudenstadt": {
"key": "595f903540a36fe8610ec39aa3a06f6a",
"f_id_kommune": 3447,
"f_id_bezirk": 22017,
"f_id_strasse": 22155,
},
"Ludwigshafen am Rhein": {
"key": "6efba91e69a5b454ac0ae3497978fe1d",
"f_id_kommune": "5916",
"f_id_strasse": "5916abteistrasse",
"f_id_strasse_hnr": 33,
},
"Traunstein": {
"key": "279cc5db4db838d1cfbf42f6f0176a90",
"f_id_kommune": "2911",
"f_id_strasse": "2374",
},
"AWB Limburg-Weilburg": {
"key": "0ff491ffdf614d6f34870659c0c8d917",
"f_id_kommune": 6031,
"f_id_strasse": 621,
"f_id_strasse_hnr": 872,
"f_abfallarten": [27, 28, 17, 67],
},
"ALBA Berlin": {
"key": "9583a2fa1df97ed95363382c73b41b1b",
"f_id_kommune": 3227,
"f_id_strasse": 3475,
"f_id_strasse_hnr": 185575,
},
"Göppingen": {
"key": "f35bd08b1d18d9c81fcdee75dbcce5d3",
"idhousenumber": 7074,
"wastetypes": [20,17,59,18,19,60]
},
}
_LOGGER = logging.getLogger(__name__)
MODUS_KEY = "d6c5855a62cf32a4dadbc2831f0f295f"
HEADERS = {"user-agent": "Mozilla/5.0 (xxxx Windows NT 10.0; Win64; x64)"}
# Parser for HTML input (hidden) text
class HiddenInputParser(HTMLParser):
def __init__(self):
super().__init__()
self._args = {}
@property
def args(self):
return self._args
def handle_starttag(self, tag, attrs):
if tag == "input":
d = dict(attrs)
if d["type"] == "hidden":
self._args[d["name"]] = d["value"]
class Source:
def __init__(
self,
key,
f_id_kommune = None,
f_id_strasse = None,
f_id_bezirk=None,
f_id_strasse_hnr=None,
f_abfallarten=[],
idhousenumber = None, # New API
wastetypes = None, # New API
):
self._key = key
self._kommune = f_id_kommune
self._bezirk = f_id_bezirk
self._strasse = f_id_strasse
self._strasse_hnr = f_id_strasse_hnr
self._abfallarten = f_abfallarten # list of integers
self._idhousenumber = idhousenumber
self._wastetypes = wastetypes
self._ics = ICS()
def fetch(self):
if self._idhousenumber == None:
# get token
params = {"key": self._key, "modus": MODUS_KEY, "waction": "init"}
r = requests.post("https://api.abfall.io", params=params, headers=HEADERS)
# add all hidden input fields to form data
# There is one hidden field which acts as a token:
# It consists of a UUID key and a UUID value.
p = HiddenInputParser()
p.feed(r.text)
args = p.args
args["f_id_kommune"] = self._kommune
args["f_id_strasse"] = self._strasse
if self._bezirk is not None:
args["f_id_bezirk"] = self._bezirk
if self._strasse_hnr is not None:
args["f_id_strasse_hnr"] = self._strasse_hnr
for i in range(len(self._abfallarten)):
args[f"f_id_abfalltyp_{i}"] = self._abfallarten[i]
args["f_abfallarten_index_max"] = len(self._abfallarten)
args["f_abfallarten"] = ",".join(map(lambda x: str(x), self._abfallarten))
now = datetime.datetime.now()
date2 = now.replace(year=now.year + 1)
args["f_zeitraum"] = f"{now.strftime('%Y%m%d')}-{date2.strftime('%Y%m%d')}"
params = {"key": self._key, "modus": MODUS_KEY, "waction": "export_ics"}
# get csv file
r = requests.post(
"https://api.abfall.io", params=params, data=args, headers=HEADERS
)
else:
params = {"key": self._key, "mode": "export", "type": "ics", "showinactive": "false"}
now = datetime.datetime.now()
params["timeperiod"] = f"{now.strftime('%Y0101')}-{now.strftime('%Y1231')}"
params["idhousenumber"] = self._idhousenumber
params["wastetypes"] = ','.join(str(wastetype) for wastetype in self._wastetypes)
r = requests.get(
"https://api.abfall.io", params=params, headers=HEADERS
)
# parse ics file
r.encoding = "utf-8" # requests doesn't guess the encoding correctly
ics_file = r.text
# Remove all lines starting with <b
# This warning are caused for customers which use an extra radiobutton
# list to add special waste types:
# - AWB Limburg-Weilheim uses this list to select a "Sonderabfall <city>"
# waste type. The warning could be removed by adding the extra config
# option "f_abfallarten" with the following values [27, 28, 17, 67]
html_warnings = re.findall(r"\<b.*", ics_file)
if html_warnings:
ics_file = re.sub(r"\<br.*|\<b.*", "\\r", ics_file)
# _LOGGER.warning("Html tags removed from ics file: " + ', '.join(html_warnings))
dates = self._ics.convert(ics_file)
entries = []
for d in dates:
entries.append(Collection(d[0], d[1]))
return entries
To make it work you need to supply the new parameters idhousenumber
and wastetypes
.
idhousenumber
is the same number as f_id_strasse
. wastetypes
should be self explanatory.
An example configuration could look like this:
sources:
- name: abfall_io
args:
key: keykeykeykeykey
idhousenumber: 1234456
wastetypes: [20,17,59,18,19]
Now only the wizzard needs also to be adapted.
Just explored a little bit more. As already mentioned there seems to be a graphql API (https://studio.apollographql.com/public/Abfallplus/variant/v1/home). The entry point of this should be the following query:
query GetCities($query: String) {
cities(query: $query) {
id
name
idHouseNumber
districtsCount
appointmentsSupported
}
}
This will return all cities in the commune with a unique ID.
For Authentication the Header X-Abfallplus-Api-Key
must be set. For Göppingen it is the value 7a4730955599a7e7c0ceb39c548b6abbae874dc7e6a0afc5
. This key seems to be tied to the commune (here Göppingen).
The API-Key can be get via GET https://api.abfall.io/?key=f35bd08b1d18d9c81fcdee75dbcce5d3&modus=d6c5855a62cf32a4dadbc2831f0f295f&waction=init
, which will responde with a JSON object.
The other communes will reponsed here with a form. So maybe they are migrating commune for commune to the new GraphQL based API?
I Have A Problem With:
A specific source
What's Your Problem
My waste scheduler source for f_id_kommune: 899 stopped working a few days ago. After some tests to verify my configuration, I realised that the request from generated from the awb-gp.de adds a timeperiod=20240101-20241231 parameter. Without this, the ICS has no values For the moment I added the generated URL as ICS source and it works now. But the Source abfall_io seems to be broken without this timeperion parameter (which cant be set in the configuration)
Source (if relevant)
No response
Logs
Relevant Configuration
Checklist Source Error
Checklist Sensor Error
Required