SchedulesDirect / JSON-Service

Code related to download, slice-and-dice and generation of JSON into database.
36 stars 5 forks source link

Feature Request: Include IDs in JSON instead of having to parse from filenames in zip #3

Closed cwchapma closed 11 years ago

cwchapma commented 11 years ago

For example, when getting lineups for a headend, the headend id is in the filename '0005410.json.txt':

Request:  {"randhash":"xxxx","request":["0005410","PC:K2J1T2"],"object":"lineups","action":"get","api":20130502}
Response: {"response":"OK","code":200,"serverID":"AWS-micro.1","message":"file available at URL.","filename":"2013-05-03T21:13:19Z.headends.json.zip","URL":"https:\/\/s3.amazonaws.com\/schedulesdirect\/lineup\/40a1deac1748cd6f3c9b6d04773e927b.zip","datetime":"2013-05-03T21:13:19Z"}

0005410.json.txt: {"name":"Rogers Cable Inc.","location":"Ottawa","deviceTypes":["X","Analog"],"X":{"map":[{"stationID":14786,"channel":"001"}, ...

This is far simpler to parse if the headend id is included in the json:

0005410.json.txt: {"headend":"0005410","name":"Rogers Cable Inc.","location":"Ottawa","deviceTypes":["X","Analog"],"X":{"map":[{"stationID":14786,"channel":"001"}, ...

I think the name and location are just repeats of what's in the headend data so they could be removed if the headend id is included.

rkulagowski commented 11 years ago

That's easy enough for cable headends. What about Over-the-air and free-to-air? headend:"PC:60030" headend:"FTA"

iShBuu commented 11 years ago

I agree with this.

Just put headend:"{whatever file is named}"

I'm not sure what the file naming scheme is for OTA, but i would think the stationID as an Int would be most useful. Perhaps as a separate field in the file stationID:"0005410" or stationID:"5410" I personally consume them all and convert to ints and store in my db that way.

rkulagowski commented 11 years ago

Over-the-air uses "PC" for postal code.

StationIDs are sent as strings because I Have Plans For Them. You can cast to int locally if you want, but don't assume that they will stay as int's forever.

If I don't hear objections, then on 2013-05-15 I will add "headend":"{foo}" into the json object. The location and other information (even though it's redundant) will remain in place for the time being.

foo will be "PC:{whatever}", "FTA", or the name of the headend.

cwchapma commented 11 years ago

That makes sense. I think the "id" should be the same as what you get from the get headends command. For OTA it's "PC:00454". I haven't tried FTA.

Note: this also applies to "get programs".

Now the file named sched_10036.json.txt contains:

[
   {
      "programID":"EP013413450057",
      ...
   },
   {
      "programID":"EP013413450076",
      ...
   }
]

I think it should be:

{
   "stationID":"10036",
   programs:[
      {
         "programID":"EP013413450057",
         ...
      },
      {
         "programID":"EP013413450076",
         ...
       }
   ]
}

And again, the same idea for "get schedules".

The whole idea being that you don't have to parse anything from the filename. Ideally that filename isn't even included (it should be redundant). The zip is just a transfer mechanism and the data contained in it should be self described.

rkulagowski commented 11 years ago

Programs on the beta server have been converted and now include the programID in the object:

{"programID":"EP000000060029","md5":"5\/HNizWKAPhKptvuN\/6Mmg","descriptionLanguage":"English","colorCode":"Color","sourceType":"Network","showType":"Series","episodeTitle150":"Post-Matrimonial Depression","syndicatedEpisodeNumber":"507","alternateSyndicatedEpisodeNumber":"35","originalAirDate":"1988-10-15","castAndCrew":["Actor:Gorden Kaye","Actor:Carmen Silvera","Actor:Rose Hill","Actor:Vicki Michelle","Actor:Kirsten Cooke"],"genres":["Sitcom"],"titles":{"title120":"'Allo 'Allo!","title10":"'Allo!"},"descriptions":{"description255":"In the aftermath of Rene's wedding, Denise escapes and the cafe is attacked.","description60":"The Colonel orders Helga to find the paintings.","description40":"Helga ordered to find paintings.","seriesDescription":"A cafe owner attempts to repatriate escapees during World War II."}}

rkulagowski commented 11 years ago

Beta server, sched_10002.json.txt:

{"stationID":"10002","programs":[{"programID":"SH009123340000","md5":"d3mRT3bnB2JCE1VHzEU6nA","airDateTime":"2013-05-14T00:00:00Z","duration":3600,"cc":true,"stereo":true},{"programID":"SH006701200000","md5":"WknOC87aDs6hCT8l83dT+A","airDateTime":"2013-05-14T01:00:00Z","duration":3600,"cc":true,"stereo":true},

rkulagowski commented 11 years ago

Beta server has been converted; all json objects for programs and schedules should now include metadata within the object.

Unless I hear that there are still issues, this ticket will be closed as completed on 2013-05-16.