search

bas-t / ffdecsawrapper

FFdecsa empowered softcam for MythTV
GNU General Public License v3.0
17 stars 9 forks source link

ffdecsawrapper support? #24

Closed zaphod-muc closed 10 years ago

zaphod-muc commented 10 years ago

Dear Bas-T,

first of all, kudos for your work! I am using it to decrypt HD channels on Astra using a purchased smart card - so I am using it to see what I have paid for on the hardware I have chosen for the task - that's how life is supposed to be.

I have problems thought, quite frequent drop-outs leading to skipped frames (more than a second missing, whole dialogues sometimes) or block artifacts. (vmalloc = 256M) and I am blank on where to start and look for a solution or even error analysis.

Now I am aware that here is not the place for support questions - can you recommend a forum where I find help?

Thanks in advance, and thanks for your work done, great job!

Zaphod

bas-t commented 10 years ago

Hi Zaphod,

I don't know of any forum where you can get adequate help. In fact, the lack of adequate help caused me to take over the old sasc-ng, so there's an upside to it after all.

First of all, try a couple of FTA channels without using FFdesawrapper. The errors you are describing are most likely caused by a bad signal (could be one of these: bad dish/lnb, bad coax cable/connectors or even bad dvb hardware in your PC) FFdecsawrapper just descrambles the signal as-is. Another (slight) possibility: Oscam does not perform right. Look at its logs to get some insight. One more: I/O problems. 'top' can tell you a lot more

Secondly: if your CPU is up to it, use a amd64 kernel. So you won't have to deal with vmalloc problems.

Best of luck! zaphod-muc schreef op 17-06-14 11:07:

Dear Bas-T,

first of all, kudos for your work! I am using it to decrypt HD channels on Astra using a purchased smart card - so I am using it to see what I have paid for on the hardware I have chosen for the task - that's how life is supposed to be.

I have problems thought, quite frequent drop-outs leading to skipped frames (more than a second missing, whole dialogues sometimes) or block artifacts. (vmalloc = 256M) and I am blank on where to start and look for a solution or even error analysis.

Now I am aware that here is not the place for support questions - can you recommend a forum where I find help?

Thanks in advance, and thanks for your work done, great job!

Zaphod

— Reply to this email directly or view it on GitHub https://github.com/bas-t/ffdecsawrapper/issues/24.

fe31nz commented 10 years ago

I was having the same problem as Zaphod (or at least the same symptoms) with sasc-ng, before I found ffdecsawrapper and switched to that. The workaround I had come up with originally for sasc-ng was just to record one program at once and preferably not at the same time as any other recording. That allowed me to make good recordings often enough that I could get by as most of the programs on SkyTV in New Zealand are transmitted at least three times in a week and I could re-record whenever I got a bad recording.

Unfortunately, when I switched to ffdecsawrapper nothing changed, and still nothing changed when I changed to using a cardreader with Oscam instead of an external NEWCAMD server, so I eventually found the time to do some serious experimenting to see if I could find out what was happening. By the time I got around to this, MythTV had been updated and now produced "Gap" error messages in mythbackend.log at the end of each recording where I had problems, which helped a lot in tracking the problem. I found that I could never record three or more ffdecsawrapper programs simultaneously, and recording two at once significantly increased the probability of gaps occurring, to the point of making it more likely than not. If I was recording other DVB-T programs at the same time as an ffdecsawrapper one, that also increased the probablity of gaps, but not as much as simultaneous ffdecsawrapper recordings. And it turned out that any program I was recording at 15:10-15:15 every day always had multiple gaps during that period. That time period is when my MythTV EPG update is done, resulting in very heavy database use by the scheduler and hence very heavy disk I/O on the system drive. So I wondered if the problem related to disk I/O. I had already tried high CPU priority for ffdecsawrapper, and that did not work, but now I modified the /etc/init.d/ffdecsawrapper startup script and /etc/default/ffdecsawrapper to add a new IONICE option and tried some experiments with that. Setting it to increased "best-effort" ionice levels seemed to help somewhat, so I eventually tried setting it to "real-time:5", and that got rid of most of my recording gaps. I do still get gaps occasionally, without any particular pattern I can see, but it is only once a fortnight instead or so instead of daily, and I now regularly record two ffdecsawrapper programs at once, even when I also have multiple DVB-T recordings going on at the same time. Recording three ffdecsawrapper programs at once still seems to cause problems, and as I still get occasional recordings with gaps, I believe that using the real-time:5 IONICE setting is just managing to hide the problem rather than fix it properly, so I would love to figure out what the real cause is.

With the current setup, I have checked a couple of times to see if programs that get gaps are being recorded to the system drive, and hence more vulnerable to the periods of heavy database access, but they were being written to other drives.

My system is not at all underpowered for the job:

Asus M5A97 Evo motherboard AMD FX-4100 3.6 GHz quad core 8 Gibytes RAM 3 x DVB-T tuners TBS 5922 DVB-S2 USB tuner TeVii S470 DVB-S2 PCIe tuner Omnikey 3121 USB card reader System drive: Hitachi Deskstar 7K3000 (3 Tbytes, 7200 rpm) Other recording drives: Hitachi Deskstar 7K3000 Seagate ST4000VN000 (4 Tbyte, 5900 rpm red=NAS drive) 2 x WDC WD30EZRX (3 Tbyte green) WDC WD40EZRX (4 Tbyte green) Mythbuntu 12.04 MythTV 0.27+fixes

My system partition with the MySQL database on it is EXT3. All the recording partitions are JFS.

SkyTV NZ uses Videoguard encryption.

I have put copies of my modified files on my web server:

http://www.jsw.gen.nz/mythtv/ffdecsawrapper (the /etc/init.d script) http://www.jsw.gen.nz/mythtv/ffdecsawrapper_default (the /etc/default file)

If Zaphod's problem is the same as mine, using those files might be a usable workaround for him too.

bas-t commented 10 years ago

Still strange, looking at your specs. Mine aren't that impressive, nevertheless I can record up to 16 HD shows with overlap at the same time. (using FFdecsawrapper, Oscam and Mythtv)

I'm closing this ticket now. This is not a support forum for general PVR setup problems. Feel free to open a new ticket if you can point me to a particular bug in FFdecsawrapper that causes your problems.

Stephen Worthington schreef op 17-06-14 13:55:

I was having the same problem as Zaphod (or at least the same symptoms) with sasc-ng, before I found ffdecsawrapper and switched to that. The workaround I had come up with originally for sasc-ng was just to record one program at once and preferably not at the same time as any other recording. That allowed me to make good recordings often enough that I could get by as most of the programs on SkyTV in New Zealand are transmitted at least three times in a week and I could re-record whenever I got a bad recording.

Unfortunately, when I switched to ffdecsawrapper nothing changed, and still nothing changed when I changed to using a cardreader with Oscam instead of an external NEWCAMD server, so I eventually found the time to do some serious experimenting to see if I could find out what was happening. By the time I got around to this, MythTV had been updated and now produced "Gap" error messages in mythbackend.log at the end of each recording where I had problems, which helped a lot in tracking the problem. I found that I could never record three or more ffdecsawrapper programs simultaneously, and recording two at once significantly increased the probability of gaps occurring, to the point of making it more likely than not. If I was recording other DVB-T programs at the same time as an ffdecsawrapper one, that also increased the probablity of gaps, but not as much as simultaneous ffdecsawrapper recordings. And it turned out that any program I was recording at 15:10-15:15 every day always h ad multiple gaps during that period. That time period is when my MythTV EPG update is done, resulting in very heavy database use by the scheduler and hence very heavy disk I/O on the system drive. So I wondered if the problem related to disk I/O. I had already tried high CPU priority for ffdecsawrapper, and that did not work, but now I modified the /etc/init.d/ffdecsawrapper startup script and /etc/default/ffdecsawrapper to add a new IONICE option and tried some experiments with that. Setting it to increased "best-effort" ionice levels seemed to help somewhat, so I eventually tried setting it to "real-time:5", and that got rid of most of my recording gaps. I do still get gaps occasionally, without any particular pattern I can see, but it is only once a fortnight instead or so instead of daily, and I now regularly record two ffdecsawrapper programs at once, even when I also have multiple DVB-T recordings going on at the same time. Recording three ffdecsawrapper programs at once still seems to cause problems, and as I still get occasional recordings with gaps, I believe that using the real-time:5 IONICE setting is just managing to hide the problem rather than fix it properly, so I would love to figure out what the real cause is.

With the current setup, I have checked a couple of times to see if programs that get gaps are being recorded to the system drive, and hence more vulnerable to the periods of heavy database access, but they were being written to other drives.

My system is not at all underpowered for the job:

Asus M5A97 Evo motherboard AMD FX-4100 3.6 GHz quad core 8 Gibytes RAM 3 x DVB-T tuners TBS 5922 DVB-S2 USB tuner TeVii S470 DVB-S2 PCIe tuner Omnikey 3121 USB card reader System drive: Hitachi Deskstar 7K3000 (3 Tbytes, 7200 rpm) Other recording drives: Hitachi Deskstar 7K3000 Seagate ST4000VN000 (4 Tbyte, 5900 rpm red=NAS drive) 2 x WDC WD30EZRX (3 Tbyte green) WDC WD40EZRX (4 Tbyte green) Mythbuntu 12.04 MythTV 0.27+fixes

My system partition with the MySQL database on it is EXT3. All the recording partitions are JFS.

SkyTV NZ uses Videoguard encryption.

I have put copies of my modified files on my web server:

http://www.jsw.gen.nz/mythtv/ffdecsawrapper (the /etc/init.d script) http://www.jsw.gen.nz/mythtv/ffdecsawrapper_default (the /etc/default file)

If Zaphod's problem is the same as mine, using those files might be a usable workaround for him too.

— Reply to this email directly or view it on GitHub https://github.com/bas-t/ffdecsawrapper/issues/24#issuecomment-46297304.

bas-t commented 10 years ago

BTW, should it not be Mythtv that gets the ionice setting? After all, Mythtv does most of the I/O

zaphod-muc commented 10 years ago

Hello Tycho, Stephen,

into the blue I have changed the ionice settings, but it did not change it. I watched the logs while watching TV, and I saw during the hickup (many seconds image freeze - 20 secs according to the logs), in ffdesawrapper.log:

Jun 17 16:44:23.184 CAM(core.auStats): EMM packet load average (1/4/10min) 17 18 19 pks/s Jun 17 16:44:28.420 CSA: Got command(2): E idx: 2 pid: 0 key: 9d3b...cc Jun 17 16:44:49.288 CAM(core.net): socket: select timed out (20000 ms) Jun 17 16:44:49.288 CAM(cardclient.core): recv error. reconnecting... Jun 17 16:44:49.288 CAM(cardclient.newcamd): failed to read message length Jun 17 16:44:49.288 CAM(cardclient.newcamd): unexpected server response (code 0) Jun 17 16:44:49.288 CAM(cardclient.core): client Newcamd (192.168.0.1:10002) ECM failed (20011 ms) Jun 17 16:44:49.289 CAM(cardclient.core): cc-loop Jun 17 16:44:49.289 CAM(core.ecm): 0.1: filter flush (elapsed 20012) Jun 17 16:44:49.289 CAM(core.ecm): 0.1: lost sync (period 6998, elapsed 26731) Jun 17 16:44:49.288 CAM(general.error): action logger 0/0 read: Buffer overflow Jun 17 16:44:49.288 CAM(core.net): connecting to 192.168.0.1:10002/tcp (192.168.0.1) Jun 17 16:44:49.292 CAM(cardclient.login): Newcamd: CaID=1830 admin=1 srvUA=0000000048BCB112 provider 008011/0000000048BC0000 000000/0000000048BC0000 003411/0000000048BC0000 Jun 17 16:44:49.583 CAM(cardclient.core): Tue Jun 17 16:44:49 2014: lagged cw 14456 ms (Newcamd) Jun 17 16:44:49.583 CSA: Got command(2): O idx: 2 pid: 0 key: 309f...97 Jun 17 16:44:49.583 CAM(general.error): action ecmhandler 0/0 read: Buffer overflow Jun 17 16:44:49.880 CSA: Got command(2): E idx: 1 pid: 0 key: 1ea6...cc Jun 17 16:44:49.880 CSA: Got command(2): O idx: 1 pid: 0 key: afb0...09 Jun 17 16:44:49.880 CAM(core.ecm): 0.1: correct key found

and at the same time, in oscam.log:

2014/06/17 16:44:32 9806140 r HDplus [nagra] Camstate: 11 04 20 2014/06/17 16:44:33 0 add client job action 31 queue length 1 mythtv 2014/06/17 16:44:33 9806140 r HDplus [nagra] cardreader_do_checkhealth: reader->card_status = 2, ret = 1 2014/06/17 16:44:34 0 add client job action 31 queue length 1 mythtv 2014/06/17 16:44:34 9806140 r HDplus [nagra] cardreader_do_checkhealth: reader->card_status = 2, ret = 1 2014/06/17 16:44:49 0 --- Skipped 14 duplicated log lines --- 2014/06/17 16:44:49 980E928 c [OSCAM-WORK] new event 1 occurred on fd 15 after 20010 ms inactivity 2014/06/17 16:44:49 980E928 c nmr(): len=0, errno=0 2014/06/17 16:44:49 980E928 c nmr: 1 return 0 2014/06/17 16:44:49 980E928 c mythtv disconnected from 192.168.0.1 2014/06/17 16:44:49 980E928 c thread B747FB40 ended! 2014/06/17 16:44:49 0 s [OSCAM] new event 1 occurred on fd 6 after 56 ms inactivity 2014/06/17 16:44:49 0 s start client thread action 25 2014/06/17 16:44:49 980B640 c data from add_job action=25 client c anonymous 2014/06/17 16:44:49 980B640 c client connected to 10002 port 2014/06/17 16:44:49 980B640 c nmr(): len=2, errno=0 2014/06/17 16:44:49 980B640 c nmr: autodetect: newcamd525 used 2014/06/17 16:44:49 980B640 c received 47 bytes from client 2014/06/17 16:44:49 980B640 00 00 E0 00 2A 6D 79 74 68 74 76 00 24 31 24 61 2014/06/17 16:44:49 980B640 62 63 64 65 66 67 68 24 55 71 71 72 41 6A 38 32 2014/06/17 16:44:49 980B640 4B 73 36 41 57 72 6C 37 39 6C 63 4A 79 31 00 2014/06/17 16:44:49 980B640 c account->usr=mythtv

So it does look like a loss of communication between ffdesawrapper and OSCAM, something that might be caused by OSCAM.

I found a post regarding deactivating the ECM cache, a first try looks promising... I will find a forum for OSCAM support (digitaleliteboard, maybe), and be quiet on the thread here...

Thanks for your replies!

Zaphod

fe31nz commented 10 years ago

MythTV works just fine with its default ionice settings. It records from my DVB-T tuners with no problems, and also from unencrypted DVB-S, although I have only done that for tests as all the unencrypted DVB-S channels here are also available on DVB-T in higher quality. I have had it recording from at least six DVB-T channels at once with overlaps bringing it up to 12 for serveral minutes, and that when I had only three drives to record to and was using my old motherboard with only a dual core AMD and 2 Gibytes of RAM. It is only ffdecsawrapper that seems to need the I/O priority boost. I have also not had to touch the priorities of Oscam or pcscd. I did wonder if it might have been log traffic from ffdecsawrapper that was the problem, but increasing or decreasing the level of logging did not seem to affect things. I know little about the internals of ffdecsawrapper, but I did think that, in order to do the decryption, it needed to process all the packets for the channels being recorded, and that needs to use I/O. Not necessarily disk I/O, but I was under the impression that in Linux the ionice setting controlled all I/O, not just disk I/O. I could easily be wrong about that as I have never done any research on it.