sqlite3.connect fails @ libschedulesdirect/cache.py line 13

[r0m30]

Your described use case is exactly the same as mine. Trying this I get sqlite3.OperationalError: unable to open database file

Is there something I should be doing to set up the initial database? The options don't include an option for the DB path just the output path.

Sorry if this is just an I don't know python question but that may be a lot of it.

Thanks for doing this, I've tried to make a perl add-on to run the xmltv projects dd_na grabber but the dependencies are out of control.

[astrilchuk]

I'm working on a new version of sd2xmltv which is just about ready. It supports the latest SchedulesDirect API 20141201. It also supports basic lineup management. I'll try to replicate your issue and push it to github within the next few days.

The sqlite database should automatically create itself though that's not a feature that I've tested in a long time. There is also no lineup management in the current version meaning you must have already added lineups to your account.

[r0m30]

Great! I saw that there was no management so I setup my headend on the SD website.

Waiting pseudo calmly.☺ On May 25, 2015 16:22, "astrilchuk" notifications@github.com wrote:

I'm working on a new version of sd2xmltv which is just about ready. It supports the latest SchedulesDirect API 20141201. It also supports basic lineup management. I'll try to replicate your issue and push it to github within the next few days.

The sqlite database should automatically create itself though that's not a feature that I've tested in a long time. There is also no lineup management in the current version meaning you must have already added lineups to your account.

— Reply to this email directly or view it on GitHub https://github.com/astrilchuk/sd2xmltv/issues/1#issuecomment-105333941.

[astrilchuk]

I've merged and pushed the latest code to the master branch. Please give it a try. Once you have the xmltv.xml output file, you can socat it to tvheadend like so (adjust the path to xmltv.sock as needed):

cat xmltv.xml | sudo socat - unix-connect:/home/hts/.hts/tvheadend/epggrab/xmltv.sock

This could be added to a shell script and called as a cron job.

[r0m30]

Hi Thanks for the new version!

You have Canada hardcoded as the country, so I changed that. It doesn't seem to be picking up my existing headend configuration, I'm wondering it that has something to do with the json vs soap servers. So I went ahead and used manage to add a lineup and tried again.

I'm getting a message: Lineup responses updated. See http://forums.schedulesdirect.org/viewtopic.php?f=17&t=2645&p=8379 for details. but I'm guessing you have seen this.

It's running, but slowly, this could be because I'm testing on a RPi2 running off a SD card. The CPU consumption isn't high but I have limited tools in the openelec image I'm currently using to see where the problem lies. I did up the logging level to debug and the response from the server came back quickly and now it just seems to be loading the sdcache.db with the program information.

At the rate it's going it's going to take several hours to cache the ~9K shows that are in the full headend lineup.

I only watch the network OTA channels so it would be nice to be able to limit the data to those if that is possible.

Thanks again!

[astrilchuk]

Hi,

Canada hardcoded? Are there any other countries I should be including? ;-) Hehe...I'll fix that so you can supply the country.

I think SchedulesDirect were planning on copying everyone's subscribed lineups from the old API (20140530) to the new API (20141201) on May 30th. That's why you had to re-add them.

I wouldn't recommend running with the logging level set to DEBUG unless absolutely necessary. That logging level outputs a huge amount of information and will slow down the process considerably.

Regarding limiting the channels - that was going to be my very next task :-). If, in the mean time, you'd like to limit the channels, look in sd2xmltv.py around line 63. You'll see a commented out self.callsign_whitelist = [...]. You can add the callsigns of only the channels you want there.

Once sdcache.db is initially populated, the subsequent runs should be much faster (don't forget to switch back to INFO logging level).

You could also supply a --days parameter on the command line to limit the number of schedule days it is loading. You can start with --days 1, --days 2, then --days 3...etc to limit the amount of data it needs to initially cache. I currently default to 14 days - though SchedulesDirect may supply more days than that on particular channels.

Any other suggestions you have would be greatly appreciated. I'd like to post detailed instructions on how to setup sd2xmltv on the tvheadend forum so your feedback is very helpful.

[r0m30]

I've heard there are one or two countries outside North America but you can't believe everything you read on the internet :-)

I was just upping the logging level to see what was going on, not planning on leaving it there.

I'll have a look at the white-listing.

The performance problems are definitely related to the db being on the sd card, changing the db to :memory: reduced the runtime to under 3 minutes user time and under 4 minutes real time for 66 channels and over 9K programs. That does kind of defeat the purpose of the whole json api though.

So here are my suggestions:

1) Finish the white-listing implementation. 2) tighten up the user input parsing, I hit a couple of array out of bounds errors when I input an incorrect response. 3) Add a serialize/de-serialize to memory function to the table, I think this is an implementation http://stackoverflow.com/questions/3850022/python-sqlite3-load-existing-db-file-to-memory/10856450#10856450 but my python foo is weak. This will be important to the RPi users. 4) XMLTV compatibility, this could be a simple shell script like this: http://forum.kodi.tv/showthread.php?tid=203011&pid=1783361#pid1783361
This would allow the user soft link the script as tv_grab_sd2xmltv into a directory in the path so TVH would present it in the epg/grabber configuration. OpenElec doesn't have socat and it's nc is from busybox and I can't get it to work with TVH.

[astrilchuk]

I have modified the way programs are inserted into the database. They are now inserted as a batch (I thought that was how it was already working). It should be much faster now - though still limited by the write speed on your sd card. I've pushed the changes to github if you'd like to give it a try.

I run Openelec on a Raspberry Pi with Tvheadend running on a second Pi with Raspbian - though all of my development and testing is performed on a desktop.

I'm pretty sure I had nc working on openelec at one point. I'll see if I can recall how I managed that.

Thanks for your feedback - I've added 1, 2, and 4 as github issues. I'll try to get to them as soon as I can.

[r0m30]

The white listing works great, just need a UI for the people who want an appliance. The new insert strategy takes the run time down to ~= an in memory table so suggestion 3 isn't needed anymore.

I'm going to see about setting this up completely on OE 5.95.1/RPi 2 tonight, but my initial testing makes me think it will work well. That means I will probably make the tv_grab script, if I get it working would you want a pull request?

Thanks for all the effort.

[astrilchuk]

I would gladly take the tv_grab script pull request!

Thanks again for your input - much appreciated.

[r0m30]

I put together a simple hardcoded one, I'll put together a better one this weekend.

All seems to be working, just seem to be having some stability problems with the tvh client and can't figure out how to schedule a series record from the client.

[astrilchuk]

Regarding the series record, by 'client' are you referring to the tvheadend or kodi interface?

[r0m30]

The kodi interface, the web interface for tvheadend has a record series button but even it doesn't have new episodes only as an option.

---

Mike On May 29, 2015 7:22 PM, "astrilchuk" notifications@github.com wrote:

Regarding the series record, by 'client' are you referring to the tvheadend or kodi interface?

— Reply to this email directly or view it on GitHub https://github.com/astrilchuk/sd2xmltv/issues/1#issuecomment-106976281.

[r0m30]

The softlink in the TVH bin directory doesn't survive a reboot :-( Looks like it will require that it's packaged as an add-on for kodi for complete integration.