Republic Services data on HA calendar does not match calendar on source site

[SpencerDub]

The schedule on the Republic Services website displays my waste pickup accurately when I submit my address: solid waste and yard waste every week, and recycling every other week.

I have added this basic configuration to my HA installation:

waste_collection_schedule:
  sources:
    - name: republicservices_com
      args:
        street_address: "[my address]"

The calendar created by this add-on, however, displays wildly incorrect data that barely resembles the schedule on the source website: nothing appears weekly, and instead, there is one week when solid waste is listed twice (but no yard waste), followed by a week that lists solid waste and recycle (but still no yard waste). Then begins a period where every other week simply has no listing at all.

Is this an issue with my configuration somehow, or an issue with the source?

I recognize troubleshooting this is more difficult without my home address, but I hope it's understandable why I would not post that here. I'm open to trying workarounds (say, using testing addresses) for the sake of troubleshooting if they're suggested to me.

[5ila5]

I don't think it your configuration testing our test cases (see below) shows only one entry (only the next)

test cases

TEST_CASES = {
    "Scott Country Clerk": {"street_address": "101 E Main St, Georgetown, KY 40324"},
    "Branch County Clerk": {"street_address": "31 Division St. Coldwater, MI 49036"},
    "Contract Collection": {"street_address": "8957 Park Meadows Dr, Elk Grove, CA 95624"},
    "Residential Collection": {"street_address": "117 Roxie Ln, Georgetown, KY 40324"},
}

test:

Testing source republicservices_com ...
  found 1 entries for Scott Country Clerk
    2023-07-13 : Solid Waste [mdi:trash-can]
  found 1 entries for Branch County Clerk
    2023-07-17 : Solid Waste [mdi:trash-can]
  found 8 entries for Contract Collection
    2023-07-18 : Recycle [mdi:leaf]
    2023-07-25 : Recycle [mdi:recycle]
    2023-08-08 : Recycle [mdi:recycle]
    2023-08-22 : Recycle [mdi:recycle]
    2023-09-05 : Recycle [mdi:recycle]
    2023-09-19 : Recycle [mdi:recycle]
    2023-10-03 : Recycle [mdi:recycle]
    2023-07-18 : Solid Waste [mdi:trash-can]
  found 1 entries for Residential Collection
    2023-07-13 : Solid Waste [mdi:trash-can]

the one entry for test 1,2 and 4 seems to be at least correct (next pickup) I couldn't confirm test 3 (calendar did not appear) (might be due to the fact that I have to use TOR to prevent being Geo-blocked) It looks like for some types they only provide one Pickup date in the nextServiceDays argument of the response from https://www.republicservices.com/api/v1/publicPickup

This needs deeper investigation.

[dt215git]

Seems like every time we sort out an issue with Republic Services, another one surfaces...

@5ila5: The discrepancy in what the website calendar displays, and the number of collections the json returns is consistent with what I've seen previously. The 3rd test case has never displayed a website calendar as far as I can remember, despite the address returning collection dates in the json for that address.

The script should result in HA displaying what the json contains (and should adjust for public holidays), but I don't think it's going to replicate the website calendar unless someone works out what other scripting the website uses.

@SpencerDub: Is there a nearby street/address in the same town that shows similar behaviour? That might help with troubleshooting. Would also be good to know if your HA instance is using the scripts obtained via a HACS install, or the latest version in this github repo. Don't think that'd make a difference to what you're seeing, but would ensure we're all looking at the same code.

[StevenGFX]

My source is only showing the next pickup.

sources:
  - name: republicservices_com
    calendar_title: Trash Pickup
    args:
      street_address: !secret home_address
    customize:
      - type: Solid Waste
        alias: Trash Pickup

[dt215git]

@StevenGFX: That's not totally unexpected. The script should return the dates provided directly from the website's backend. Those often only cover the next scheduled collection, so I would assume the response only contained the single date you're seeing the the HA calendar. There seems to be some further scripting running on the website that generates the longer-term schedule being displayed in the website's calendar.

[StevenGFX]

@dt215git Thank you for the response. That's about what I assumed what was going on and that would make sense to me. I wasn't sure if this was expected, or if it was possibly related to this issue.

[vogles]

Hi, I just found this integration and was confused about the difference between what I saw in the republic services website vs HA. So I spent way too long staring at decompiled JS to try and make sense of it. I think I figured it out, though. I'm not particularly good at JS, so I hope I'm not too far from the target here.

In src_app_modules_layout_content-wrapper_content-wrapper_main_ts.blahblahblah.js there is a class that I found by searching for thursdayPickups. I searched for that, because it's part of the response JSON for the publicPickup endpoint, and it just so happens to be when my trash gets picked up. There are a couple of interesting functions here: calcEventDates, getDaysOfWeek, and getDaysKeys. If I understand this correctly, it is basically mapping out the week for each pickup type (trash, recycle, yard), if "[weekday]Pickups" is > 0. It does seem to use nextServiceDays, but I can't really understand how it's used. Once the calendar has been built up, there's a call to adjustHolidayPickups which does what the method name says. I've attached the decompiled class that I'm talking about. I hope it helps. republic_services_calendar_decompiled.txt