Failing to Post-Process

[raph521]

Hi,

I saw mention in the below issue thread that post-processing was failing with this mylar3 image:

https://github.com/linuxserver/docker-mylar/issues/39

I've noticed the same thing. I see the following in the log:

26-Aug-2020 19:02:20 - WARNING :: mylar.run.108 : Thread-17 : [META-TAGGER][WARNING] Make sure that you are using the comictagger included with Mylar.
26-Aug-2020 19:03:20 - ERROR   :: mylar.extract_image.146 : CP Server Thread-4 : [ERROR] Unable to properly retrieve the cover. It's probably best to re-tag this file : Unrar not installed? (rarfile.UNRAR_TOOL='unrar')

From within the container, it looks like unrar is NOT in the PATH:

$ docker exec -it mylar3 bash
root@af9d98e18c4d:/# ls
app  comics  defaults  docker-mods  downloads  home  lib      media  opt   root  sbin  sys  usr
bin  config  dev       donate.txt   etc        init  libexec  mnt    proc  run   srv   tmp  var
root@af9d98e18c4d:/# which unrar
root@af9d98e18c4d:/# find . -name unrar
./usr/lib/python3.8/site-packages/unrar
root@af9d98e18c4d:/# which unrar
root@af9d98e18c4d:/# exit
exit

I compared this to the old/current mylar image and container, where post-processing does work, and I see that unrar is in the path:

$ docker exec -it mylar bash
root@af9d98e18c4d:/# which unrar
/usr/bin/unrar
root@af9d98e18c4d:/# find . -name unrar
./usr/lib/python2.7/site-packages/unrar
./usr/bin/unrar
root@af9d98e18c4d:/# exit
exit

Could that be the issue?

[aptalca]

Good idea, that could be the issue. Can you exec into the container with docker exec -it mylar3 bash, then install the repo package with apk add --no-cache unrar and see if that works? If so, we'll add it into the image.

[raph521]

Sadly, it didn't work. The unrar message disappeared but then there was a new one that stated the version of comictagger included in mylar should be used. I tried installing comictagger myself since the old linuxserver/mylar image did that, but it didn't help.

I've rebuilt this image in my environment since the version on dockerhub hasn't been updated in a few months and there's a different error now:

27-Aug-2020 10:28:49 - WARNING :: mylar.run.257 : CP Server Thread-8 : [META-TAGGER][COMIC-TAGGER][CBR-TO-CBZ] Failed to convert cbr to cbz - check permissions on folder : /config/mylar/cache and/or the location where Mylar is trying to tag the files from.

I'm going to take a shot and see if the mylar devs know what the issue may be

[raph521]

I posted on mylar3's issues: https://github.com/mylar3/mylar3/issues/425

But managed to get it working by installing all of the requirements I found from hotio/mylar3's Dockerfile: https://github.com/hotio/docker-mylar3/blob/stable/linux-arm-v7.Dockerfile

I think it was the installation of unrar-cffi that fixed it?

[evilhero]

Just came across this thanks to @Raph521 and his posting of the issue, along with some google-fu.

The problem is that the unrar binary isn't in the path, but also that the check that's done in ComicTagger checks for rar, not unrar - I heavily modified CT in order to work with Mylar, so I wasn't even thinking of this as my local repo never had any problems, but it's also-non-docker.

So yeah, there's a PR coming into the dev side of Mylar probably tonight, and should be in the main build within the week after we can get some testing on it other than myself. But for the docker aspect going forward, as long as unrar is installed on the system, and it's in the path so the which binary can find it, Mylar will be able to store the path in the config going forward so it'll work going forward as well.

Cheers,

[raph521]

I've re-built this image using the mylar3's python3-dev branch, which has the fix mentioned by evilhero:

    mylar3:
        container_name: mylar3
        #image: lsiodev/mylar3:latest
        build:
            context: https://github.com/linuxserver/docker-mylar3.git#initial
            dockerfile: Dockerfile.armhf
            args:
                MYLAR3_RELEASE: python3-dev

@evilhero has updated mylar3's README.md with more info about rar/unrar requirements.

With his change, plus adding unrar via apk add --no-cache unrar, post-processing now works perfectly.

So I think unrar just needs to be added to this image.

[aptalca]

Excellent, thanks for confirming. I'll give this one another go and once I get it working locally, we'll publish

[barbequesauce]

Thank you @Raph521 and @aptalca !

[aptalca]

@barbequesauce @Raph521 I'm getting this error trying to start the latest version (0.4.4):

Traceback (most recent call last):
  File "/app/mylar3/Mylar.py", line 26, in <module>
    import mylar
  File "/app/mylar3/mylar/__init__.py", line 38, in <module>
    from mylar import logger, versioncheckit, rsscheckit, searchit, weeklypullit, PostProcessor, updater, helpers
  File "/app/mylar3/mylar/rsscheckit.py", line 21, in <module>
    from mylar import logger, rsscheck, helpers, auth32p
  File "/app/mylar3/mylar/rsscheck.py", line 52, in <module>
    feedparser._FeedParserMixin._start_newznab_attr = _start_newznab_attr
AttributeError: module 'feedparser' has no attribute '_FeedParserMixin'

[evilhero]

What version is the feedparser module at? Can you list your versions via pip?

We need 5.2.1 or greater in the requirements.txt file - not sure if they depreciated something in a newer version or not as hasn't been brought up as of yet by anyone.

[aptalca]

root@80979066e085:/# pip list
Package                   Version
------------------------- ----------
altgraph                  0.17
appdirs                   1.4.4
APScheduler               3.6.3
beautifulsoup4            4.9.1
CacheControl              0.12.6
certifi                   2020.4.5.1
cfscrape                  2.1.1
chardet                   3.0.4
cheroot                   8.2.1
CherryPy                  18.6.0
colorama                  0.4.3
configparser              5.0.0
contextlib2               0.6.0
distlib                   0.3.0
distro                    1.5.0
feedparser                6.0.0
html5lib                  1.0.1
idna                      2.9
jaraco.classes            3.1.0
jaraco.collections        3.0.0
jaraco.functools          3.0.1
jaraco.text               3.2.0
lockfile                  0.12.2
Mako                      1.1.3
MarkupSafe                1.1.1
more-itertools            8.5.0
msgpack                   1.0.0
natsort                   7.0.1
ordered-set               4.0.1
packaging                 20.4
pep517                    0.8.2
Pillow                    6.2.2
pip                       20.2.3
portend                   2.6
progress                  1.5
pyinstaller               4.0
pyinstaller-hooks-contrib 2020.8
pyparsing                 2.4.7
pytoml                    0.1.21
pytz                      2020.1
requests                  2.23.0
retrying                  1.3.3
setuptools                47.0.0
sgmllib3k                 1.0.0
simplejson                3.17.2
six                       1.15.0
soupsieve                 2.0.1
tempora                   4.0.0
toml                      0.10.1
tzlocal                   2.1
urllib3                   1.25.9
webencodings              0.5.1
zc.lockfile               2.0

[evilhero]

Perfect, thanks for that.

I'm guessing 6.0.0 depreciated what we're trying to use, so I'll have to figure out what it is.

Setting feedparser in requirements.txt to feedparser==5.2.1 would at least force it to use the version we built against which definitely works.

But will check regardless, as at worst we'd have to limit it to below 6.0.0 temporarily until it can be addressed properly.

[aptalca]

@evilhero looks like 6.0.0 breaks sab as well and they limited it to below 6 https://github.com/sabnzbd/sabnzbd/commit/bc4f06dd1d3ab91a57f4df053c8399010cb915fb#diff-b4ef698db8ca845e5845c4618278f29a

[evilhero]

Should be fixed now in latest master release (v0.4.5) that went live a little while ago.

Mylar specifically referenced an internal module of feedparser that was causing the error, so a quick patch to make it backwards compatible and works regardless of the version (at least until they depreciate something)

Thanks for bringing it to our attention! Cheers.

[aptalca]

Thanks for the quick fix. The latest version starts up fine and it successfully post-processes. I just had dumb luck that I ended up building the image right after feedparser was updated.

The only issue I now notice is that the web viewer only opens the first page. There is a question mark button in the top left corner, but clicking it doesn't seem to do anything. I'm assuming there is supposed to be an overlay loaded with player controls but it doesn't seem to be loaded. I can open other pages only by changing the page_num in the url.

Console errors:

jquery-2.2.4.min.js:1 Failed to load resource: the server responded with a status of 404 ()
2materialize.min.js:1 Failed to load resource: the server responded with a status of 404 ()
read_comic?ish_id=763572&page_num=0&size=high:67 Uncaught ReferenceError: $ is not defined
    at read_comic?ish_id=763572&page_num=0&size=high:67
Roboto-Regular.woff2:1 Failed to load resource: the server responded with a status of 404 ()
Roboto-Regular.woff:1 Failed to load resource: the server responded with a status of 404 ()
Roboto-Regular.ttf:1 Failed to load resource: the server responded with a status of 404 ()
/favicon.ico:1 Failed to load resource: the server responded with a status of 500 ()

Error in the log:

192.168.13.12 - - [14/Sep/2020:05:50:02] "GET /read_comic?ish_id=763572&page_num=0&size=high HTTP/1.1" 200 1277 "http://192.168.13.40:8090/comicDetails?ComicID=112161" "Mozilla/5.0 (X11; CrOS aarch64 13310.59.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.84 Safari/537.36"
192.168.13.12 - - [14/Sep/2020:05:50:03] "GET /interfaces/default/js/jquery-2.2.4.min.js HTTP/1.1" 404 465 "http://192.168.13.40:8090/read_comic?ish_id=763572&page_num=0&size=high" "Mozilla/5.0 (X11; CrOS aarch64 13310.59.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.84 Safari/537.36"
192.168.13.12 - - [14/Sep/2020:05:50:03] "GET /interfaces/default/js/materialize.min.js HTTP/1.1" 404 463 "http://192.168.13.40:8090/read_comic?ish_id=763572&page_num=0&size=high" "Mozilla/5.0 (X11; CrOS aarch64 13310.59.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.84 Safari/537.36"
192.168.13.12 - - [14/Sep/2020:05:50:03] "GET /interfaces/default/js/materialize.min.js HTTP/1.1" 404 463 "http://192.168.13.40:8090/read_comic?ish_id=763572&page_num=0&size=high" "Mozilla/5.0 (X11; CrOS aarch64 13310.59.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.84 Safari/537.36"
192.168.13.12 - - [14/Sep/2020:05:50:03] "GET /interfaces/default/fonts/roboto/Roboto-Regular.woff2 HTTP/1.1" 404 468 "http://192.168.13.40:8090/interfaces/default/css/materialize.min.css" "Mozilla/5.0 (X11; CrOS aarch64 13310.59.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.84 Safari/537.36"
192.168.13.12 - - [14/Sep/2020:05:50:03] "GET /interfaces/default/fonts/roboto/Roboto-Regular.woff HTTP/1.1" 404 466 "http://192.168.13.40:8090/interfaces/default/css/materialize.min.css" "Mozilla/5.0 (X11; CrOS aarch64 13310.59.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.84 Safari/537.36"
192.168.13.12 - - [14/Sep/2020:05:50:03] "GET /interfaces/default/fonts/roboto/Roboto-Regular.ttf HTTP/1.1" 404 465 "http://192.168.13.40:8090/interfaces/default/css/materialize.min.css" "Mozilla/5.0 (X11; CrOS aarch64 13310.59.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.84 Safari/537.36"
[14/Sep/2020:05:50:04] HTTP
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/cherrypy/_cprequest.py", line 638, in respond
self._do_respond(path_info)
File "/usr/lib/python3.8/site-packages/cherrypy/_cprequest.py", line 697, in _do_respond
response.body = self.handler()
File "/usr/lib/python3.8/site-packages/cherrypy/lib/encoding.py", line 219, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
return self.callable(*self.args, **self.kwargs)
File "/usr/lib/python3.8/site-packages/cherrypy/_cptools.py", line 175, in handle_func
handled = self.callable(*args, **self._merged_args(kwargs))
TypeError: staticfile() got multiple values for argument 'filename'

[barbequesauce]

Awesome... I'm on it..

[barbequesauce]

@aptalca the webviewer should be good to go now, following https://github.com/mylar3/mylar3/pull/490

[aptalca]

Great, thanks for the heads up. I'll close this issue once that pr hits a new release and we push it.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[aptalca]

This was fixed a while back but I forgot to close it