Closed Hartger87 closed 1 year ago
What I can see this is a API not used by other waste collectors. But I will have a look if I can implement this new API
Another customer of RMN here, just wanting to let you know the custom_component is highly appreciated and I hope you will be able to implement this one.
Keep up the great work!
It is the same API as in issue #378 I'm trying to reverse engineer the Android app to see if I can access the API, but this will take some time.
Doesn't look like the same issue. I get the following message in the logs:
2023-03-02 15:06:34.751 ERROR (MainThread) [custom_components.afvalbeheer.API] Error occurred while fetching data: JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
This is the afvalbeheer part in configuration.yaml
afvalbeheer: !include include.d/afvalbeheer.yaml
This is the file include.d/afvalbeheer.yaml (redacted my address details)
---
wastecollector: RMN
resources:
- restafval
- gft
- papier
postcode: 34****
streetnumber: ***
printwastetypes: 1
upcomingsensor: 1
dateformat: '%d-%m-%Y' # (optional)
dateonly: 1 # (optional)
name: "RMN Nieuwegein Doorslag" # (optional)
nameprefix: 1 # (optional)
builtinicons: 0 # (optional)
dutch: 0 # (optional)
@dajomas It is the same issue. The RMN and the BAR waste collectors are broken right now. The both started using a new API service, which I'm not yet familiar with.
here some results api
{
"actionResult": null,
"changes": {
"2814749778847455": {
"rel": {
"value": "Address"
},
"name": {
"value": "Address"
},
"href": {
"value": "https://21qubz.mendixcloud.com/rest/meldingenapi/v3/adressen?huisnummer=148&postcode=3734BL&huisletter=&toevoeging="
}
}
},
"commits": [
"7318349395320724",
"7318349395320722",
"7318349395320723",
"7318349395320720",
"7318349395320721",
"2814749778847457",
"4222124651127712",
"2814749778847456",
"7318349395320719",
"2814749778847458"
],
"committedObjectsOmitted": false,
"deletes": [],
"instructions": [
{
"args": {
"FormTarget": "content",
"FormWidth": 0,
"FormPath": "Burger_Applicatie/Landing_RMN_MeerdereAdressen.page.xml",
"FormUrl": "/p/rmni/landing/adressen/{Id}",
"FormResizable": true,
"FormHeight": 0,
"FormName": "Burger_Applicatie.Landing_RMN_MeerdereAdressen",
"FormParameters": {
"$Resultset_Address": "4222124651127712"
},
"FormTitle": "Selecter uw adres"
},
"target": "system",
"type": "open_form"
}
],
"newpersistable": [],
"objects": [
{
"attributes": {
"Burger_Applicatie.AanvraagActie_APIV3_Adres_selected": {
"value": null
},
"aid": {
"value": null
},
"Burger_Applicatie.Adres_Resultset_Address": {
"value": "4222124651127712"
},
"VolledigAdres": {
"value": null
},
"Burger_Applicatie.FractieAPIV3_Adres_selected": {
"value": null
},
"Postcode": {
"value": "3734BL"
},
"Huisnummer": {
"value": "148"
},
"Letter": {
"value": "E"
},
"Toevoeging": {
"value": null
},
"_selfURL": {
"value": "https://21qubz.mendixcloud.com/rest/meldingenapi/v3/adressen/206321157929798558/groups"
},
"Straat": {
"value": "Dolderseweg 148E, 3734BL Den Dolder"
}
},
"guid": "7318349395320724",
"hash": "SaB9WMCEjrsXTlUKSHOWq6gE7m6c6CE4/nxbJJ1MB/E=",
"objectType": "Burger_Applicatie.Adres"
},
{
"attributes": {
"Burger_Applicatie.AanvraagActie_APIV3_Adres_selected": {
"value": null
},
"aid": {
"value": null
},
"Burger_Applicatie.Adres_Resultset_Address": {
"value": "4222124651127712"
},
"VolledigAdres": {
"value": null
},
"Burger_Applicatie.FractieAPIV3_Adres_selected": {
"value": null
},
"Postcode": {
"value": "3734BL"
},
"Huisnummer": {
"value": "148"
},
"Letter": {
"value": "C"
},
"Toevoeging": {
"value": null
},
"_selfURL": {
"value": "https://21qubz.mendixcloud.com/rest/meldingenapi/v3/adressen/206321157929798556/groups"
},
"Straat": {
"value": "Dolderseweg 148C, 3734BL Den Dolder"
}
},
"guid": "7318349395320722",
"hash": "3OIwEAdbZZGj9KcXYn13W9QyDbjHF/g9vyzP6Q/g670=",
"objectType": "Burger_Applicatie.Adres"
},
{
"attributes": {
"Burger_Applicatie.AanvraagActie_APIV3_Adres_selected": {
"value": null
},
"aid": {
"value": null
},
"Burger_Applicatie.Adres_Resultset_Address": {
"value": "4222124651127712"
},
"VolledigAdres": {
"value": null
},
"Burger_Applicatie.FractieAPIV3_Adres_selected": {
"value": null
},
"Postcode": {
"value": "3734BL"
},
"Huisnummer": {
"value": "148"
},
"Letter": {
"value": "D"
},
"Toevoeging": {
"value": null
},
"_selfURL": {
"value": "https://21qubz.mendixcloud.com/rest/meldingenapi/v3/adressen/206321157929798557/groups"
},
"Straat": {
"value": "Dolderseweg 148D, 3734BL Den Dolder"
}
},
"guid": "7318349395320723",
"hash": "xeJkfnsHhnNQ3cVcHkzBRCtsWqr767EVhcjVkZ4PU5A=",
"objectType": "Burger_Applicatie.Adres"
},
{
"attributes": {
"Burger_Applicatie.AanvraagActie_APIV3_Adres_selected": {
"value": null
},
"aid": {
"value": null
},
"Burger_Applicatie.Adres_Resultset_Address": {
"value": "4222124651127712"
},
"VolledigAdres": {
"value": null
},
"Burger_Applicatie.FractieAPIV3_Adres_selected": {
"value": null
},
"Postcode": {
"value": "3734BL"
},
"Huisnummer": {
"value": "148"
},
"Letter": {
"value": "A"
},
"Toevoeging": {
"value": null
},
"_selfURL": {
"value": "https://21qubz.mendixcloud.com/rest/meldingenapi/v3/adressen/206321157929795014/groups"
},
"Straat": {
"value": "Dolderseweg 148A, 3734BL Den Dolder"
}
},
"guid": "7318349395320720",
"hash": "SBquvtTDFF3qxxHOabGGADbtVtiujbBsfuEATT9+kj0=",
"objectType": "Burger_Applicatie.Adres"
},
{
"attributes": {
"Burger_Applicatie.AanvraagActie_APIV3_Adres_selected": {
"value": null
},
"aid": {
"value": null
},
"Burger_Applicatie.Adres_Resultset_Address": {
"value": "4222124651127712"
},
"VolledigAdres": {
"value": null
},
"Burger_Applicatie.FractieAPIV3_Adres_selected": {
"value": null
},
"Postcode": {
"value": "3734BL"
},
"Huisnummer": {
"value": "148"
},
"Letter": {
"value": "B"
},
"Toevoeging": {
"value": null
},
"_selfURL": {
"value": "https://21qubz.mendixcloud.com/rest/meldingenapi/v3/adressen/206321157929798555/groups"
},
"Straat": {
"value": "Dolderseweg 148B, 3734BL Den Dolder"
}
},
"guid": "7318349395320721",
"hash": "bddipqgbNCHq2WiUl2MDKqzEdruyoLYJnLTg9mhRrQU=",
"objectType": "Burger_Applicatie.Adres"
},
{
"attributes": {
"_type": {
"value": "GET"
},
"name": {
"value": "Categorie selecteren"
},
"_return": {
"value": false
},
"rel": {
"value": "Groups"
},
"Burger_Applicatie.link_helper": {
"value": null
},
"Burger_Applicatie.link_resultSet": {
"value": "4222124651127712"
},
"href": {
"value": null
},
"createdDate": {
"readonly": true,
"value": 1677774553869
},
"active": {
"value": false
}
},
"guid": "2814749778847457",
"hash": "M1/OkqG9qWagqNKVz1H1KLGOHAnNjmygdYuqRrYxSIY=",
"objectType": "Burger_Applicatie.link"
},
{
"attributes": {
"Latitude": {
"value": "0"
},
"Longitude": {
"value": "0"
},
"ophaallocatie": {
"value": null
},
"BetalingsproviderId": {
"value": null
},
"VrijeTekst": {
"value": null
},
"adres": {
"value": null
}
},
"guid": "4222124651127712",
"hash": "zuOc7F0MTKvZSa54v9Q4F9EyF6RSyDI+WbLZrlii0J8=",
"objectType": "Burger_Applicatie.Resultset_Address"
},
{
"attributes": {
"_type": {
"value": "GET"
},
"name": {
"value": "Adres selecteren"
},
"_return": {
"value": false
},
"rel": {
"value": "default"
},
"Burger_Applicatie.link_helper": {
"value": null
},
"Burger_Applicatie.link_resultSet": {
"value": "4222124651127712"
},
"href": {
"value": "default"
},
"createdDate": {
"readonly": true,
"value": 1677774553869
},
"active": {
"value": false
}
},
"guid": "2814749778847456",
"hash": "Nc/Osowx7qmkk9njw0R7FfRY8ft5AY7NKP2+883Zkb0=",
"objectType": "Burger_Applicatie.link"
},
{
"attributes": {
"Burger_Applicatie.AanvraagActie_APIV3_Adres_selected": {
"value": null
},
"aid": {
"value": null
},
"Burger_Applicatie.Adres_Resultset_Address": {
"value": "4222124651127712"
},
"VolledigAdres": {
"value": null
},
"Burger_Applicatie.FractieAPIV3_Adres_selected": {
"value": null
},
"Postcode": {
"value": "3734BL"
},
"Huisnummer": {
"value": "148"
},
"Letter": {
"value": null
},
"Toevoeging": {
"value": null
},
"_selfURL": {
"value": "https://21qubz.mendixcloud.com/rest/meldingenapi/v3/adressen/206321157929772869/groups"
},
"Straat": {
"value": "Dolderseweg 148, 3734BL Den Dolder"
}
},
"guid": "7318349395320719",
"hash": "d/MecQCPCsVQk/Cq3/QbULkj6SdI3VB0NjkYxdiEx6E=",
"objectType": "Burger_Applicatie.Adres"
},
{
"attributes": {
"_type": {
"value": "GET"
},
"name": {
"value": "Proces selecteren"
},
"_return": {
"value": false
},
"rel": {
"value": "Templates"
},
"Burger_Applicatie.link_helper": {
"value": null
},
"Burger_Applicatie.link_resultSet": {
"value": "4222124651127712"
},
"href": {
"value": null
},
"createdDate": {
"readonly": true,
"value": 1677774553869
},
"active": {
"value": false
}
},
"guid": "2814749778847458",
"hash": "NjcLslJD5tZah7DCRxbStPihviJZ/D0+OOVokX4d+9g=",
"objectType": "Burger_Applicatie.link"
}
],
"resets": {}
}
Try to find an address on the RMN webportal that won't give you any popups for wrong address. You're asking for information on an address that can't be inserted in the app and gives an error on the website.
I have the same problem,really appreciate all the work!
Also RMN customer. Every effort is appreciated!
Follow. thanks for the effort!
I have some sad news: I'm unable to find an open API we can use. Neither in the website, IOS or Android app.
The website uses a locked down API, which I'm not (yet) able to get past the authentication. Maybe anyone here has better luck?
I send a request to RMN for further info on how to obtain the information. As I have a Tizen OS screen, I am fully dependent on the info that can be collected by HA to display the info since RMN does not have an App for Tizen systems
Will keep you posted
I send a request to the provider of the new system. It is using mendix to build the service. Didn't get any response.
Got a respond and are not planning to open up the api for retrieving garbage collecting. The clients should ask for this. So my next mail goes to RMN.
Can you copy/paste your mail? Then we can send something with greater numbers, I am really missing the API!
I have sent a mail to RMN for access to their API
I received an answer from RMN, basically telling me that currently, they don't have an API and it is unclear if one will come anytime soon, as they are focussing on the most common use of their citizen portal.
I think we need to spam them. If we create a template mail and everyone is going to mail them it will be the common use ;) to have an open api.
There are also other collectors using the same system (https://21south.nl/21qubz/ is the developer. Maybe we could add them so the main developer could open it to all. It should be the same code as it is the same backend
For now I created my own local.calendar and manualy put in the dates for this years collections. Not ideal, but a workable solution for now.
I received an answer from RMN, basically telling me that currently, they don't have an API and it is unclear if one will come anytime soon, as they are focussing on the most common use of their citizen portal.
So, while RMN was in compliance, they just started being not-compliant towards the PSI guideline on Open Data (NL: "Open Data Richtlijn")? Wow!
If so, local governments (municipalities) is to be contacted/spam , not (just) RMN.
I am having the same issues, but with an other waste collector named ACV, After an update the dates turned to unknown, and getting the same list index out of range error.
I received an answer from RMN, basically telling me that currently, they don't have an API and it is unclear if one will come anytime soon, as they are focussing on the most common use of their citizen portal.
So, while RMN was in compliance, they just started being not-compliant towards the PSI guideline on Open Data (NL: "Open Data Richtlijn")? Wow!
If so, local governments (municipalities) is to be contacted/spam , not (just) RMN.
RMN is a commercial party. It’s the local government that’s not complaint. In my case the city of Soest.
RMN is a commercial party. It’s the local government that’s not complaint. In my case the city of Soest.
Ik heb inmiddels ook de vraag uitgezet bij RMN, en gerefereerd aan deze en vergelijkbare GH issues en de wetgeving. Even het antwoord afwachten.
Gelukkig (...) is RMN als bedrijf in eigendom van de 6 gemeenten. . ....zes gemeenten zijn eigenaar van RMN en vertegenwoordigd in het algemeen en dagelijks bestuur van RMN...
Zo googlend naar de mogelijkheden van Mendix, zou een API ermee realiseren niet al te moeilijk moeten zijn (?)
Over het verantwoordelijk zijn als gemeente te zorgen dat de open data er is, zal vast veel te doen zijn komende jaren.
I've also asked them on twitter. They responded this morning that they've asked the developers, but that it might cost additional money 👀 https://twitter.com/rmn_nl/status/1638796533578584064?s=19
I also send them a message:
Dag,
Onlangs hebben jullie een nieuw platform in gebruik genomen voor de afvalkalender. Helaas zijn er een paar functies verdwenen die voor mij heel waardevol waren:
- het vorige platform had de mogelijkheid om de afvalkalender in mijn eigen kalender te laden, waardoor altijd in mijn agenda stond welke container er buiten moet.
- het vorige platform had ook een open API, waarmee ik deze informatie in mijn domotica-systeem kon verwerken.
Het nieuwe platform (voor zover ik kan zien gemaakt met behulp van 21QUBZ) ondersteunt dat helaas niet. De API's waar de app gebruik van maakt, zijn gesloten en niet herbruikbaar. Dat is mijns inziens niet in lijn met een Open Data Richtlijn, die volgens mij op deze data van toepassing is.
Mijn vraag is daarom wanneer deze informatie wel op een herbruikbare manier beschikbaar komt voor mij (en met mij anderen die hier gebruik van maken).
Alvast bedankt voor uw reactie.
Met vriendelijke groet,
Feel free to re-use. :)
I also send them a message:
Dag, Onlangs hebben jullie een nieuw platform in gebruik genomen voor de afvalkalender. Helaas zijn er een paar functies verdwenen die voor mij heel waardevol waren: - het vorige platform had de mogelijkheid om de afvalkalender in mijn eigen kalender te laden, waardoor altijd in mijn agenda stond welke container er buiten moet. - het vorige platform had ook een open API, waarmee ik deze informatie in mijn domotica-systeem kon verwerken. Het nieuwe platform (voor zover ik kan zien gemaakt met behulp van 21QUBZ) ondersteunt dat helaas niet. De API's waar de app gebruik van maakt, zijn gesloten en niet herbruikbaar. Dat is mijns inziens niet in lijn met een Open Data Richtlijn, die volgens mij op deze data van toepassing is. Mijn vraag is daarom wanneer deze informatie wel op een herbruikbare manier beschikbaar komt voor mij (en met mij anderen die hier gebruik van maken). Alvast bedankt voor uw reactie. Met vriendelijke groet,
Feel free to re-use. :)
did :-)
RMN has the data publicly available but I'm not too tech savvy to actually reverse engineer the whole thing. On https://21burgerportaal.mendixcloud.com/#/rmn/landing/ you can enter your postal code and house number and afterwards you can view the calendar. I've been able to copy-paste the requests in Postman and get a response but I can't seem to figure out which request I have to send exactly and what the request entails as to interpret it to a date when certain gargage is collected. Maybe someone with more knowledge of web development and reverse engineering can look into this? Perhaps we can make a proof of concept that posts the right requests, parse them and shows the dates?
I tried, but it seems not possible to use the websites API. I also had a look at the ios and android app, but no luck there either.
I did some digging using the Android app. I have succefully been able to retrieve data using the following commands:
Step 1: We need to register the app/device
curl -X POST "https://www.googleapis.com/identitytoolkit/v3/relyingparty/signupNewUser?key=AIzaSyA6NkRqJypTfP-cjWzrZNFJzPUbBaGjOdk"
Which will return a IDtoken (valid for one hour), and refresh token.
{
"kind": "identitytoolkit#SignupNewUserResponse",
"idToken": "_IDTOKEN_",
"refreshToken": "_REFRESH_TOKEN_",
"expiresIn": "3600",
"localId": "XXXXXX"
}
Step 2: We can refresh our idToken with the refresh token (which will not expire)
curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' --data 'grant_type=refresh_token&refresh_token=_REFRESH_TOKEN_' "https://securetoken.googleapis.com/v1/token?key=AIzaSyA6NkRqJypTfP-cjWzrZNFJzPUbBaGjOdk"
Which will return a new IDtoken (valid for one hour), and refresh token.
{
"access_token": "_IDTOKEN_",
"expires_in": "3600",
"token_type": "Bearer",
"refresh_token": "_REFRESH_TOKEN_",
"id_token": "_ID_TOKEN_",
"user_id": "XXX",
"project_id": "XXX"
}
Step 3: We can use the IDToken to retrieve a address ID by using the postal code and housenumber
curl -H "authorization: _IDTOKEN_" "https://europe-west3-burgerportaal-production.cloudfunctions.net/exposed/organisations/138204213564933597/address?zipcode=1234AB&housenumber=1"
[{"addressId":"_ADDRESSID_","zipcode":"1234AB","street":"Xtraat","city":"XCity","housenumber":1,"municipalityId":"XXX","latitude":XXXX,"longitude":XXXX}]
Step 4: We can use the IDToken and ADDRESSID to retrieve the calendar
curl -H "authorization: _IDTOKEN_" https://europe-west3-burgerportaal-production.cloudfunctions.net/exposed/organisations/138204213564933597/address/_ADDRESSID_/calendar
[
{
"year": 2023,
"month": 12,
"day": 13,
"collectionDate": "2023-12-11T00:00:00.000Z",
"fraction": "GFT",
"placementPeriod": "",
"placementDescription": "",
"uuid": "XXXGFT"
},
{
"year": 2023,
"month": 12,
"day": 19,
"collectionDate": "2023-12-14T00:00:00.000Z",
"fraction": "REST",
"placementPeriod": "",
"placementDescription": "",
"uuid": "XXXREST"
},
............
]
Awesome, great work! @pippyn Is this something you can work with?
I think this should work. I verified the proof of concept and it works for RMN. I'll need to find the organisation IDs for the other waste collectors and verify that it works for them too.
Great work @tomlissen !
I'll need to find the organisation IDs for the other waste collectors and verify that it works for them too.
If we have other providers with an app in the android playstore, I could use the current setup for retrieving the organizations (and see if they use a differen "key")
Just leave a note with the name of the collectors
Edit:
I just installed the BAR App. The initial deviceregistration key is the same.
Bar organisation id: 138204213564933497
Difference is only in the last three digits
Just leave a note with the name of the collectors
According to this page, the other collectors are:
I now have a working version of the sensor. You can test this beta if you want. It works for RMN as well as BAR. But I need some more addresses to get fraction mapping correct. Can you all help me out?
You can use 3435AM, number 15 for RMN
3401 PV 92
Op do 6 apr. 2023 14:27 schreef Pippyn @.***>:
I now have a working version of the sensor. It works for RMN as well as BAR. But I need some more addresses to get fraction mapping correct. Can you all help me out?
— Reply to this email directly, view it on GitHub https://github.com/pippyn/Home-Assistant-Sensor-Afvalbeheer/issues/382#issuecomment-1498982475, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4USDKAJQRU5A4GPX5G3SWDW72ZBTANCNFSM6AAAAAAVNDRHOA . You are receiving this because you authored the thread.Message ID: @.*** com>
Maybe share private data in DM instead of thread!
Maybe share private data in DM instead of thread!
Sadly, there seems to be no DM in Github
You can DM me on the HA forum if you like. This is me
Just leave a note with the name of the collectors
According to this page, the other collectors are:
- Zaanstad
- Renewi
- Maassluis
- Spullenbak
It seems all of these do not use the same api. So far I could only find BAR and RMN using this specific api.
OK bummer, thank you for checking @tomlissen
Just installed V5.2.1 (you capitalized the V here ;-) ) and I got the dates! GREAT JOB!
However, I think there are no icons provided by the wastecollector. (So I had to set builtinicons to 1)
Thanks very much, it WORKS! :-)
Thanks a lot. I am using RMN and can confirm dates are available again.
Thanks a lot!
Op do 6 apr. 2023 16:51 schreef Taner Paca @.***>:
Thanks a lot. I am using RMN and can confirm dates are available again.
— Reply to this email directly, view it on GitHub https://github.com/pippyn/Home-Assistant-Sensor-Afvalbeheer/issues/382#issuecomment-1499198966, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4USDKDTWFOAL5VLBFPRFHTW73J7JANCNFSM6AAAAAAVNDRHOA . You are receiving this because you authored the thread.Message ID: @.*** com>
Thanks @tomlissen and @pippyn ! :D
Really great job @tomlissen of finding the "api" we can use and @pippyn of implementing it so fast. I think you made a lot of us very happy!!
Hi,
Since last monday no upcoming events are showing in HA.
I didn't made any change, but I know RMN is switched to an other calender.
Adress in my neigbourhood is 3401AD 16
Hopefully you can help