braice / MuMuDVB

A DVB IPTV streaming software
http://mumudvb.braice.net/
GNU General Public License v2.0
215 stars 133 forks source link

Relatively high CPU usage while decrypting channels #157

Closed anphsw closed 7 years ago

anphsw commented 7 years ago

On FTA TPs it'ok, 1-2%, but if I enable scam decrypt (BISS), CPU USAGE goes to 20%, but it's not decryption penalty, I tried also astra-sm in same setup, and it consumes no more than 5% of CPU.

here is config: mumudvb:

` card=0 tuning_timeout=0 timeout_no_diff=0

lnb_type=universal pol=v freq=12522 srate=27500

autoconfiguration=full autoconf_radios=0 autoconf_scrambled=1

unicast=1 port_http=8070

multicast_ipv4=0 multicast_ipv6=0

dont_send_scrambled=1 playlist_ignore_dead=1

dvr_buffer_size=1024

scam_support=1 ring_buffer_default_size=262144 oscam: [global] nice = -1 WaitForCards = 0

logfile = stdout; syslog

pidfile = /var/run/oscam.pid lb_savepath = /var/run/oscam/lbstat

[anticasc] enabled = 0

[dvbapi] enabled = 1 au = 1 boxtype = pc user = mumudvb pmt_mode = 4 request_mode = 1

[reader] label = static_biss protocol = constcw device = /etc/oscam/biss.key group = 1 `

perf-top while using mumudvb: ` PerfTop: 2554 irqs/sec kernel:33.5% exact: 0.0% [4000Hz cycles], (all, 2 CPUs)

42.64%  libdvbcsa.so.1.0.1  [.] dvbcsa_bs_block_decrypt_batch
14.52%  libdvbcsa.so.1.0.1  [.] dvbcsa_bs_stream_cipher_batch
 5.19%  [kernel]            [k] rtl8169_interrupt
 3.29%  [dvb_core]          [k] dvb_dmx_swfilter_section_copy_dump
 1.31%  [kernel]            [k] sysenter_past_esp
 1.05%  libdvbcsa.so.1.0.1  [.] dvbcsa_bs_stream_transpose_out
 1.03%  [kernel]            [k] _raw_spin_lock
 0.91%  libpthread-2.22.so  [.] pthread_mutex_lock
 0.90%  mumudvb             [.] main
 0.87%  [dvb_core]          [k] dvb_dmx_swfilter_packet
 0.81%  libpthread-2.22.so  [.] __pthread_mutex_unlock_usercnt
 0.78%  [kernel]            [k] __copy_to_user_ll
 0.68%  [vdso]              [.] __kernel_vsyscall
 0.68%  [ip_tables]         [k] ipt_do_table
 0.66%  [kernel]            [k] _raw_spin_lock_irqsave
 0.58%  libdvbcsa.so.1.0.1  [.] dvbcsa_bs_block_transpose_in

astra config:

!/usr/bin/env astra-sm

package.path = "/etc/dvb/scripts-sm/?.lua";

require("stream") require("playlist")

-- 55E, LNB master

adapter_single = dvb_tune({ type = "S", adapter = 0, tp = "12522:V:27500", lnb = "9750:10600:11700", -- Ku universal })

make_channel({ name = "Kanal Disney +2", input = { "dvb://adapter_single#pnr=2&biss=XXXXXXXXXXXXXXXX", "file:///usr/lib/matras_sd.ts#loop" }, output = { "http://0.0.0.0:8070/bysid/2" }, })

make_channel({ name = "U +2", input = { "dvb://adapter_single#pnr=3", "file:///usr/lib/matras_sd.ts#loop" }, output = { "http://0.0.0.0:8070/bysid/3" }, })

make_channel({ name = "GTRK Groznyj", input = { "dvb://adapter_single#pnr=4", "file:///usr/lib/matras_sd.ts#loop" }, output = { "http://0.0.0.0:8070/bysid/4" }, })

make_channel({ name = "TV 41", input = { "dvb://adapter_single#pnr=120", "file:///usr/lib/matras_sd.ts#loop" }, output = { "http://0.0.0.0:8070/bysid/120" }, })

http_server({ port = 9100, route = {{ "/playlist.m3u", playlist_request }} }) `

perf-top while using astra: ` PerfTop: 156 irqs/sec kernel:31.4% exact: 0.0% [4000Hz cycles], (all, 2 CPUs)

42.11%  libdvbcsa.so.1.0.1  [.] dvbcsa_bs_block_decrypt_batch
13.44%  libdvbcsa.so.1.0.1  [.] dvbcsa_bs_stream_cipher_batch
 4.54%  [kernel]            [k] rtl8169_interrupt
 4.29%  libdvbcsa.so.1.0.1  [.] dvbcsa_bs_stream_transpose_out
 2.57%  [dvb_core]          [k] dvb_dmx_swfilter_packet
 1.98%  [tbs_pcie_dvb]      [k] tbs6902_pcie_irq
 1.45%  [dvb_core]          [k] dvb_dmx_swfilter_packets
 0.89%  libdvbcsa.so.1.0.1  [.] dvbcsa_bs_block_transpose_in
 0.80%  libdvbcsa.so.1.0.1  [.] dvbcsa_bs_block_transpose_out
 0.79%  [kernel]            [k] _raw_spin_lock_irqsave
 0.77%  [kernel]            [k] sysenter_past_esp
 0.72%  [kernel]            [k] _raw_spin_lock
 0.71%  [kernel]            [k] irq_entries_start
 0.54%  [tbs_pcie_dvb]      [k] adapter_tasklet
 0.49%  astra-sm            [.] __module_stream_send

`

Same libdvbcsa calls, but huge diffirence: 2554 vs 156 irq/s. Is there way to tune mumudvb config to reduce usage?

braice commented 7 years ago

Hello

I can only do an educated guess from here. But before that do we have any source code from astra sm

I guess they send the packets to libdvbsa in bigger chunks but would be good to check on a piece of code.

Otherwise they also may use different config parameters but as I didn't wrote this code it is hard to say more without digging further.

Brice

On Jan 23, 2017 01:22, "anphsw" notifications@github.com wrote:

On FTA TPs it'ok, 1-2%, but if I enable scam decrypt (BISS), CPU USAGE goes to 20%, but it's not decryption penalty, I tried also astra-sm in same setup, and it consumes no more than 5% of CPU.

here is config: mumudvb: `# 55E LNB master

card=0 tuning_timeout=0 timeout_no_diff=0

lnb_type=universal pol=v freq=12522 srate=27500

autoconfiguration=full autoconf_radios=0 autoconf_scrambled=1

unicast=1 port_http=8070

multicast_ipv4=0 multicast_ipv6=0

dont_send_scrambled=1 playlist_ignore_dead=1

dvr_buffer_size=1024

scam_support=1 ring_buffer_default_size=262144 oscam:[global] nice = -1 WaitForCards = 0

logfile = stdout; syslog

pidfile = /var/run/oscam.pid lb_savepath = /var/run/oscam/lbstat

[anticasc] enabled = 0

[dvbapi] enabled = 1 au = 1 boxtype = pc user = mumudvb pmt_mode = 4 request_mode = 1

[reader] label = static_biss protocol = constcw device = /etc/oscam/biss.key group = 1 perf-top while using mumudvb: PerfTop: 2554 irqs/sec kernel:33.5% exact: 0.0% [4000Hz cycles], (all, 2 CPUs)

42.64% libdvbcsa.so.1.0.1 [.] dvbcsa_bs_block_decrypt_batch 14.52% libdvbcsa.so.1.0.1 [.] dvbcsa_bs_stream_cipher_batch 5.19% [kernel] [k] rtl8169_interrupt 3.29% [dvb_core] [k] dvb_dmx_swfilter_section_copy_dump 1.31% [kernel] [k] sysenter_past_esp 1.05% libdvbcsa.so.1.0.1 [.] dvbcsa_bs_stream_transpose_out 1.03% [kernel] [k] _raw_spin_lock 0.91% libpthread-2.22.so [.] pthread_mutex_lock 0.90% mumudvb [.] main 0.87% [dvb_core] [k] dvb_dmx_swfilter_packet 0.81% libpthread-2.22.so [.] pthread_mutex_unlock_usercnt 0.78% [kernel] [k] __copy_to_user_ll 0.68% [vdso] [.] kernel_vsyscall 0.68% [ip_tables] [k] ipt_do_table 0.66% [kernel] [k] _raw_spin_lock_irqsave 0.58% libdvbcsa.so.1.0.1 [.] dvbcsa_bs_block_transpose_in

astra config:#!/usr/bin/env astra-sm

package.path = "/etc/dvb/scripts-sm/?.lua";

require("stream") require("playlist")

-- 55E, LNB master

adapter_single = dvb_tune({ type = "S", adapter = 0, tp = "12522:V:27500", lnb = "9750:10600:11700", -- Ku universal })

make_channel({ name = "Kanal Disney +2", input = { "dvb://adapter_single#pnr=2&biss=XXXXXXXXXXXXXXXX", "file:///usr/lib/matras_sd.ts#loop" }, output = { "http://0.0.0.0:8070/bysid/2" }, })

make_channel({ name = "U +2", input = { "dvb://adapter_single#pnr=3", "file:///usr/lib/matras_sd.ts#loop" }, output = { "http://0.0.0.0:8070/bysid/3" }, })

make_channel({ name = "GTRK Groznyj", input = { "dvb://adapter_single#pnr=4", "file:///usr/lib/matras_sd.ts#loop" }, output = { "http://0.0.0.0:8070/bysid/4" }, })

make_channel({ name = "TV 41", input = { "dvb://adapter_single#pnr=120", "file:///usr/lib/matras_sd.ts#loop" }, output = { "http://0.0.0.0:8070/bysid/120" }, })

http_server({ port = 9100, route = {{ "/playlist.m3u", playlist_request }} }) perf-top while using astra: PerfTop: 156 irqs/sec kernel:31.4% exact: 0.0% [4000Hz cycles], (all, 2 CPUs)

42.11% libdvbcsa.so.1.0.1 [.] dvbcsa_bs_block_decrypt_batch 13.44% libdvbcsa.so.1.0.1 [.] dvbcsa_bs_stream_cipher_batch 4.54% [kernel] [k] rtl8169_interrupt 4.29% libdvbcsa.so.1.0.1 [.] dvbcsa_bs_stream_transpose_out 2.57% [dvb_core] [k] dvb_dmx_swfilter_packet 1.98% [tbs_pcie_dvb] [k] tbs6902_pcie_irq 1.45% [dvb_core] [k] dvb_dmx_swfilter_packets 0.89% libdvbcsa.so.1.0.1 [.] dvbcsa_bs_block_transpose_in 0.80% libdvbcsa.so.1.0.1 [.] dvbcsa_bs_block_transpose_out 0.79% [kernel] [k] _raw_spin_lock_irqsave 0.77% [kernel] [k] sysenter_past_esp 0.72% [kernel] [k] _raw_spin_lock 0.71% [kernel] [k] irq_entries_start 0.54% [tbs_pcie_dvb] [k] adapter_tasklet 0.49% astra-sm [.] __module_stream_send

`

Same libdvbcsa calls, but huge diffirence: 2554 vs 156 irq/s. Is there way to tune mumudvb config to reduce usage?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/braice/MuMuDVB/issues/157, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUUD1reQxcFxvaH1Pu-AgAGHfPklVV8ks5rVEc-gaJpZM4Lqsvk .

anphsw commented 7 years ago

Astra-sm code is complicated, but i think we can start digging from https://gitlab.com/berdyansk/astra-sm/blob/master/src/stream/softcam/decrypt.c

anphsw commented 7 years ago

I suddenly noticed that decrypted channels have huge bitrate, while others is normal on the same TP. I analyzed FTA and decrypted streams and what i got: This is normal stream: Mar 21 00:39:12: INFO: PAT: tsid: 9 Mar 21 00:39:12: INFO: PAT: pid: 42 PMT pnr: 3 Mar 21 00:39:12: INFO: PAT: crc32: 0xEA8CAE66 Mar 21 00:39:12: INFO: PMT: pnr: 3 Mar 21 00:39:12: INFO: PMT: pid: 41 PCR Mar 21 00:39:12: INFO: AUDIO: pid: 40 type: MPEG-2 audio, ISO/IEC 13818-3 (0x04) Mar 21 00:39:12: INFO: VIDEO: pid: 41 type: MPEG-4 AVC/H.264, ISO/IEC 14496-10 (0x1B) Mar 21 00:39:12: INFO: PMT: crc32: 0x1ED890CF Mar 21 00:39:12: INFO: CAT: CAS: caid: 0x0632 pid: 7502 Mar 21 00:39:12: INFO: SDT: tsid: 9 Mar 21 00:39:12: INFO: SDT: sid: 3 Mar 21 00:39:12: INFO: SDT: Service: U +2 Mar 21 00:39:12: INFO: SDT: Provider: GAZPROM SS Mar 21 00:39:12: INFO: SDT: crc32: 0x400E25F3 Mar 21 00:39:13: INFO: Bitrate: 2629 Kbit/s Mar 21 00:39:14: INFO: Bitrate: 2640 Kbit/s

This is decrypted (softcam) stream: Mar 21 00:39:05: INFO: PAT: tsid: 9 Mar 21 00:39:05: INFO: PAT: pid: 39 PMT pnr: 2 Mar 21 00:39:05: INFO: PAT: crc32: 0xDA998432 Mar 21 00:39:05: INFO: CAT: CAS: caid: 0x0632 pid: 7502 Mar 21 00:39:05: INFO: PMT: pnr: 2 Mar 21 00:39:05: INFO: PMT: pid: 38 PCR Mar 21 00:39:05: INFO: PMT: CAS: caid: 0x2600 pid: 8191 Mar 21 00:39:05: INFO: AUDIO: pid: 37 type: MPEG-1 audio, ISO/IEC 11172-3 (0x03) Mar 21 00:39:05: INFO: VIDEO: pid: 38 type: MPEG-4 AVC/H.264, ISO/IEC 14496-10 (0x1B) Mar 21 00:39:05: INFO: PMT: crc32: 0xADD30267 Mar 21 00:39:05: INFO: SDT: tsid: 9 Mar 21 00:39:05: INFO: SDT: sid: 2 Mar 21 00:39:05: INFO: SDT: Service: Kanal Disney +2 Mar 21 00:39:05: INFO: SDT: Provider: GAZPROM SS Mar 21 00:39:05: INFO: SDT: crc32: 0xE0285326 Mar 21 00:39:06: INFO: Bitrate: 50331 Kbit/s Mar 21 00:39:07: INFO: Bitrate: 50005 Kbit/s Mar 21 00:39:08: INFO: Bitrate: 50342 Kbit/s

But 50mbits it's whole TP, not single stream! I saved stream to file and analyzed. It have huge amount of trashed packets. I think trashed packets it's the rest of TP bandwith, incorrectly decrypted (it should drop this traffic instead!) by mumudvb and passed to output.

anphsw commented 7 years ago

This valgrind warning may be releated to this bug: ==23391== Use of uninitialised value of size 4 ==23391== at 0x488D4C2: dvbcsa_bs_block_decrypt_batch (in /usr/lib/libdvbcsa.so.1.0.1) ==23391== by 0x488D1E9: dvbcsa_bs_decrypt (in /usr/lib/libdvbcsa.so.1.0.1) ==23391== by 0x13A967: decsathread_func (scam_decsa.c:232) ==23391== by 0x48DA41A: start_thread (in /usr/lib/libpthread-2.22.so) ==23391== by 0x49E9C0D: clone (clone.S:122) ==23391== Uninitialised value was created by a stack allocation ==23391== at 0x488D3C6: dvbcsa_bs_block_decrypt_batch (in /usr/lib/libdvbcsa.so.1.0.1)

anphsw commented 7 years ago

Before patch: Mar 22 13:57:11: INFO: PAT: tsid: 20 Mar 22 13:57:11: INFO: PAT: pid: 82 PMT pnr: 15 Mar 22 13:57:11: INFO: PAT: crc32: 0x54B8FD4E Mar 22 13:57:11: INFO: PMT: pnr: 15 Mar 22 13:57:11: INFO: PMT: pid: 81 PCR Mar 22 13:57:11: INFO: PMT: CAS: caid: 0x2600 pid: 8191 Mar 22 13:57:11: INFO: PMT: descriptor: 0x050443554549 Mar 22 13:57:11: INFO: AUDIO: pid: 80 type: MPEG-2 audio, ISO/IEC 13818-3 (0x04) Mar 22 13:57:11: INFO: VIDEO: pid: 81 type: MPEG-4 AVC/H.264, ISO/IEC 14496-10 (0x1B) Mar 22 13:57:11: INFO: DATA: pid: 442 type: Private PES, ISO/IEC 13818-1 (0x06) Mar 22 13:57:11: INFO: DATA: descriptor: 0x45041C02F3D3 Mar 22 13:57:11: INFO: DATA: pid: 1001 type: User private (0x86) Mar 22 13:57:11: INFO: PMT: crc32: 0xD36989C Mar 22 13:57:11: INFO: SDT: tsid: 20 Mar 22 13:57:11: INFO: SDT: sid: 15 Mar 22 13:57:11: INFO: SDT: Service: CHE+7 Mar 22 13:57:11: INFO: SDT: Provider: GAZPROM SS Mar 22 13:57:11: INFO: SDT: crc32: 0x243B8549 Mar 22 13:57:12: INFO: Bitrate: 25267 Kbit/s Mar 22 13:57:13: INFO: Bitrate: 25312 Kbit/s Mar 22 13:57:14: INFO: Bitrate: 25432 Kbit/s Mar 22 13:57:15: INFO: Bitrate: 25411 Kbit/s

after patch: Mar 22 14:03:15: INFO: PAT: tsid: 20 Mar 22 14:03:15: INFO: PAT: pid: 82 PMT pnr: 15 Mar 22 14:03:15: INFO: PAT: crc32: 0x54B8FD4E Mar 22 14:03:15: INFO: SDT: tsid: 20 Mar 22 14:03:15: INFO: SDT: sid: 15 Mar 22 14:03:15: INFO: SDT: Service: CHE+7 Mar 22 14:03:15: INFO: SDT: Provider: GAZPROM SS Mar 22 14:03:15: INFO: SDT: crc32: 0x243B8549 Mar 22 14:03:15: INFO: PMT: pnr: 15 Mar 22 14:03:15: INFO: PMT: pid: 81 PCR Mar 22 14:03:15: INFO: PMT: CAS: caid: 0x2600 pid: 8191 Mar 22 14:03:15: INFO: PMT: descriptor: 0x050443554549 Mar 22 14:03:15: INFO: AUDIO: pid: 80 type: MPEG-2 audio, ISO/IEC 13818-3 (0x04) Mar 22 14:03:15: INFO: VIDEO: pid: 81 type: MPEG-4 AVC/H.264, ISO/IEC 14496-10 (0x1B) Mar 22 14:03:15: INFO: DATA: pid: 442 type: Private PES, ISO/IEC 13818-1 (0x06) Mar 22 14:03:15: INFO: DATA: descriptor: 0x45041C02F3D3 Mar 22 14:03:15: INFO: DATA: pid: 1001 type: User private (0x86) Mar 22 14:03:15: INFO: PMT: crc32: 0xD36989C Mar 22 14:03:16: INFO: Bitrate: 3745 Kbit/s Mar 22 14:03:17: INFO: Bitrate: 3777 Kbit/s

anphsw commented 7 years ago

Do we need to send unencrypted packets to scam_new_packet()? I think checking if packet is encrypted or not will reduce cpu usage for a little.

braice commented 7 years ago

I have merged your patch

Thank you very much

Brice

2017-03-22 9:38 GMT-04:00 anphsw notifications@github.com:

Do we need to send unencrypted packets to scam_new_packet()? I think checking if packet is encrypted or not will reduce cpu usage for a little.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/braice/MuMuDVB/issues/157#issuecomment-288400458, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUUD3sX_dn3LBbdDtMlGZ-3RmAQebiVks5roSRYgaJpZM4Lqsvk .