Bazarr consume a lot of ressources for scans (15% on a i7 7700)

[moust4ki]

Describe the bug Bazarr using a lot of CPU (15% on a i7 7700) To Reproduce Steps to reproduce the behavior: On each scan (every 3hours) it consume a lot of cpu and as I have a large library mainly hosted on Google via rclone, it take around 1h. So every 2h I have Bazarr using 15-20% cpu for an hour.

Expected behavior Less ressources needed? Screenshots https://i.imgur.com/6ODktdV.png

Software (please complete the following information):

• Bazarr: 0.8.1
• Radarr version 0.2.0.1358
• Sonarr version 2.0.0.5338
• OS: unRAID

Additional context I ask 2 subs for each media.

[halali]

If you using gdrive i suggest disable scan for embedded subs. If you will have still same issue come back with debug log.

[moust4ki]

Hello,

Thanks for the reply. Here is a debug log during a spike (actually Bazarr is consuming 18% CPU since 2 hours now).

https://we.tl/t-zfLrQWZ2DV

And here are my Bazarr settings: https://imgur.com/a/NgRrpbE

Feel free to ask if you need anything.

Cheers

[halali]

Your log is full of errors with read file, so probably you have some issues with your rclone mount, some bad settings or you got 1 day ban on gdrive for api hammering.

edit: Have you good pathmappings for movies? because series seems work good, only movies have errors with read.

and also you can desable subscene provider, reqire anticaptcha provider to work, and require usename and password, so another error you got every search

[moust4ki]

I have no issue watching anything, so I don't think its an API ban. Are you talking about those errors? 22/09/2019 00:02:28|ERROR |root |BAZARR Error trying to get video information for this file: /rclone/tv_shows/Dexter/Season 5/flhd-dexters05e09.mkv|'Traceback (most recent call last):\n File "/app/bazarr/bazarr/get_subtitle.py", line 64, in get_video\n hash_from=hash_from)\n File "/app/bazarr/bazarr/../libs/subzero/video.py", line 58, in parse_video\n skip_hashing=skip_hashing, hash_from=hash_from)\n File "/app/bazarr/bazarr/../libs/subliminal_patch/core.py", line 529, in scan_video\n video = Video.fromguess(path, guessed_result)\n File "/app/bazarr/bazarr/../libs/subliminal/video.py", line 93, in fromguess\n return Episode.fromguess(name, guess)\n File "/app/bazarr/bazarr/../libs/subliminal/video.py", line 170, in fromguess\n raise ValueError(\'Insufficient data to process the guess\')\nValueError: Insufficient data to process the guess'|

Isn't it error about not enough data to identify a file ? (title ?)

[halali]

you have many kind of errors, this filenames is realy bad... also this error 22/09/2019 04:09:27|ERROR |root |BAZARR Error trying to get video information for this file: /rclone/tv_shows/Dirty Money/Season 1/dirty.money.2018.s01e02.720p.web.x264-strife.mkv|'Traceback (most recent call last):\n File "/app/bazarr/bazarr/get_subtitle.py", line 70, in get_video\n refine_from_ffprobe(original_path, video)\n File "/app/bazarr/bazarr/get_subtitle.py", line 951, in refine_from_ffprobe\n data = parser.parseFfprobe(path)\n File "/app/bazarr/bazarr/../libs/pyprobe/pyprobe.py", line 40, in parseFfprobe\n fdict = self._executeFfprobe(inputFile)\n File "/app/bazarr/bazarr/../libs/pyprobe/pyprobe.py", line 71, in _executeFfprobe\n outputJson = self._executeParser(self._ffprobe, commandArgs, inputFile)\n File "/app/bazarr/bazarr/../libs/pyprobe/pyprobe.py", line 184, in _executeParser\n "Error occurred during execution - " + e.output\nIOError: Error occurred during execution - '| and this 22/09/2019 04:08:13|ERROR |root |BAZARR Error trying to detect language for this subtitles file: /rclone/tv_shows/The Flash (2014)/Season 02/The.Flash.2014.S02E07.720p.HDTV-DIMENSION.srt You should try to delete this subtitles file manually and ask Bazarr to download it again.|'Traceback (most recent call last):\n File "/app/bazarr/bazarr/list_subtitles.py", line 88, in store_subtitles\n text = text.decode(encoding.original_encoding)\n File "/usr/lib/python2.7/encodings/cp1254.py", line 15, in decode\n return codecs.charmap_decode(input,errors,decoding_table)\nUnicodeDecodeError: \'charmap\' codec can\'t decode byte 0x9d in position 788: character maps to <undefined>'|

and also this 22/09/2019 10:25:04|ERROR |subliminal_patch.core |Unexpected error in provider 'subscene': Traceback (most recent call last): File "/app/bazarr/bazarr/../libs/subliminal_patch/core.py", line 162, in list_subtitles_provider results = self[provider].list_subtitles(video, provider_languages) File "/app/bazarr/bazarr/../libs/subliminal_patch/core.py", line 98, in __getitem__ provider = provider_registry[name](**self.provider_configs.get(name, {})) File "/app/bazarr/bazarr/../libs/subliminal_patch/providers/subscene.py", line 132, in __init__ raise ConfigurationError('Username and password must be specified')ConfigurationError: Username and password must be specified|Traceback (most recent call last): File "/app/bazarr/bazarr/../libs/subliminal_patch/core.py", line 162, in list_subtitles_provider results = self[provider].list_subtitles(video, provider_languages) File "/app/bazarr/bazarr/../libs/subliminal_patch/core.py", line 98, in __getitem__ provider = provider_registry[name](**self.provider_configs.get(name, {})) File "/app/bazarr/bazarr/../libs/subliminal_patch/providers/subscene.py", line 132, in __init__ raise ConfigurationError('Username and password must be specified')ConfigurationError: Username and password must be specified|

[moust4ki]

Yes I deactivated subscene, my bad. Regarding path, they seems good. So all remaining error are because of file naming? Thats it?

Thanks again :)

[halali]

i think this errors cause your high cpu usage. Fix your file naming, the second error, is because bazarr can read some of your existing subtitles because isnt in utf-8 encoded.

[halali]

Can i close this?

[moust4ki]

Well I think I found the bug: If I have Hebrew subs, consumption goes crazy. When I deactivate them consumption is normal (~3%) so I think it is related to encoding? Do you want me to do some tests?

[halali]

Can you show me export of locale command?

[halali]

It also can be connected with provider where is searching for Hebrew subs, and when you don't search for them this provider is discarded. Can tell me what providers you have enabled?

[moust4ki]

Can you show me export of locale command?

What do you mean ?

[morpheus65535]

Some providers need to download a lot of zip/rar file from providers website and extract them to get the required information from the included subtitles. This require a lot of CPU and could explain resources consumption you have.

[moust4ki]

It also can be connected with provider where is searching for Hebrew subs, and when you don't search for them this provider is discarded. Can tell me what providers you have enabled?

OpenSubtitles Podnapisi

Some providers need to download a lot of zip/rar file from providers website and extract them to get the required information from the included subtitles. This require a lot of CPU and could explain resources consumption you have.

Yes that's possible but could this apply just to Hebrew ones? Cause without changing the provider, only deactivating Hebrew everything works flawless.

[morpheus65535]

podnapisi use zip file. Maybe those Hebrew subtitles were downloaded only from Podnapisi. I don't know if OS provide Hebrew subs at all.

[moust4ki]

At the end I still have some spikes but less frequents since I deactivated Hebrew. So I assume its related to some naming errors. I will check this and get back to you if I am able to see where it comes from. Cheers

[morpheus65535]

Thanks!