USA-MA20483-X only 12 days of data

[karog]

For all 68 stationIDs I use, today /schedules with a modified date of 2014-11-24T15:08:59Z (about 90 minutes ago) have a start date of 2014-11-24 and end date of 2014-12-07 (as it should) but the programs only go through the end of 2014-12-05 (ie an end date of 2014-12-06). Here is an example for stationID 20431 (CBS HD) showing the last program and the metadata:

    {
        "programID": "EP014145320767",
        "airDateTime": "2014-12-05T23:30:00Z",
        "duration": 1800,
        "md5": "0Eclw5nJGzCBYHIspKq1Dw",
        "new": true,
        "audioProperties": [
            "cc",
            "stereo"
        ],
        "videoProperties": [
            "hdtv"
        ]
    }
],
"metadata": {
    "modified": "2014-11-24T15:08:59Z",
    "md5": "RdL8paqVi109MY3HuMMeGQ",
    "startDate": "2014-11-24",
    "endDate": "2014-12-07",
    "days": 13
}

[karog]

Today this problem has occurred again except that 43 of my 68 channels got only 12 days of data and 25 got all 13 days of data. Looking at one from each group, the modified datetimes for the respective stationIDs were comparable, within 20 minutes of each other.

There definitely seems to be a sporadic issue of the 13th day of data getting missed. And it appears to be precisely one day of data. When the last day is missing, the final programs have starttimes just before the end of the 12th day by less than or equal to the program duration.

[rkulagowski]

Can you send me one each example of the 12 days / 13 days stationIDs (just the stationID number) so that I can check the server, and the raw data used to generate them? I'd like to get this figured out, so I won't rotate the schedules for now.

[karog]

One with 13 days is stationID 20431 and one with 12 days is stationID 58452, both of which I just checked.

[rkulagowski]

This appears to be an upstream issue; the raw data is also ending at the 12/13 day time period as you found. I will open a ticket with our upstream, and will consider updating the server-side code to include the "actual" number of days in the data rather than the number which was requested, because I have no guarantee that our upstream will be able to fix the issue, and I don't want to change the ceiling to "12 days" from "13" if we can avoid it.

[karog]

Thanks. I don't want to see days reduced to 12 either.

Before this recent service change, the old mythfilldatabase and schedulesdirect used to give me 16 days which was perfect. That would give me full data for the current day 2 weeks out plus partial for the next day (after time zone adjustment). For example, on Wed I would get full Wed in 2 weeks plus partial Thu so I could easily see future weekly program recurrences.

What is the reason that only 13 days are available now when 16 were available before? Just curious and wanting 16 if that were possible.

[rkulagowski]

The upstream data now only goes out 14 days, and even 14 days included "fringe" and partial schedules, which is why we had backed off to 13 days. Now it appears that even 13 days isn't complete, which is why I need to open a ticket. I will let you know what I find out.

[karog]

I know you have been working on this. I ran into the station purged issue someone reported the other day somewhat before I saw it reported. Seeing your answer to that report I let it go. No problem.

But I thought I would give you a new data point done today 2014-12-07 17:00:00Z (approx). For stationID 20431 schedule, the metadata reported 12 days of data which matches the start and end dates in the metadata but I actually got all 13 days. Below see the tail of the schedule json where the last program is at the end of the first day after the endDate:

    {
        "programID": "EP014145320777",
        "airDateTime": "2014-12-19T23:30:00Z",
        "duration": 1800,
        "md5": "KEks1aDThLHE/mxN8lBi9Q",
        "new": true,
        "audioProperties": [
            "cc",
            "stereo"
        ],
        "videoProperties": [
            "hdtv"
        ]
    }
],
"metadata": {
    "modified": "2014-12-07T19:50:27Z",
    "md5": "B+Jh+uGHuVQIQXZt27HqdQ",
    "startDate": "2014-12-07",
    "endDate": "2014-12-19",
    "days": 12
}

}

[rkulagowski]

API-20141201 now provides per-stationID schedules per-day, up to 30 days into the future; you no longer need to request 2, 4 or 13 days.