Docker / Magnet Issues - Githubissues

trip5 commented 4 years ago

Trying to make a docker image from python 3.7 (I've tried 3.6 but pip fails on installing transmission) - I've tried various base images including more full ubuntu images. I consistently get this error:

error

Upon viewing helpers.py file it also seems to be missing everything after line 60 so I'm wondering if either libtorrent isn't handling the magnet URI correctly, especially since this error message isn't in your source code or (b) it's a kind of redirect link and helpers.py doesn't know how to handle it.

By the way, except this, your program looks exactly what I've been searching for! (I'll use a schedule script to search for newly released / rare torrents, automated with a 'echo :d 1' pipe).

trip5 commented 4 years ago

Oh and my apologies for the screen shot. I'm also doing this through Synology Docker UI which makes building and running containers a bit of a pain but quite functional.

trip5 commented 4 years ago

My docker file looks like this (just so you get some idea of what I'm starting with):

FROM python:3.7.8-slim
EXPOSE 443
ADD Raincoat.json /root/.config/Raincoat.json
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
RUN apt-get update
RUN apt-get install apt-utils -y
RUN apt-get install python-libtorrent -y
RUN pip install raincoat-jackett
ADD init.sh /init.sh
ENTRYPOINT ["/bin/bash"]

The ADD at the beginning adds a basic json config file. PIP, when running a build doesn't seem to create that file. After that, everything seems fine... until I get to actually trying to use it.

Gabisonfire commented 4 years ago

Hey! I'm glad you like my tool :) Libtorrent is only used when your want to download the torrent files locally, and I haven't pushed it yet to pip since I got sidetracked and could not do proper testing yet. I should've put it in a separate branch.

Since you are trying to send to deluge, you are not using libtorrent and the magnet links should be sent directly to Deluge without issues. You might have a bit more info in the log file, it logs in the same place as your config file (something I'd like to change as well). Use the --verbose option and see if we can get more info.

Also could you please post your config file, make sure to obfuscate personal information.

I did not consider running it in docker, I will probably try to build the image myself and see if I get the same issues as you.

automated with a 'echo :d 1' pipe I can probably add a switch for this instead, something like --select ID

trip5 commented 4 years ago

It doesn't recognize --verbose as an argument. So... The logfile is only one line now. Not very useful, probably. I'm trying another Docker build without all the apt and libtorrent stuff.

The new dockerfile looks like this:

FROM python:3.7.8-slim EXPOSE 443 ADD Raincoat.json /root/.config/Raincoat.json RUN pip install raincoat-jackett ENTRYPOINT ["/bin/bash"]

And the docker-compose.yml looks like this (I have no idea what I'm doing here but the network has to be exposed to get PIP to work):

version: "2.3" services: base: build: context: . network: host

My Raincoat.json:

{ "jackett_apikey": "123abcjackettapikey123abc", "jackett_url": "http://127.0.0.1:9117", "jackett_indexer": "all", "description_length": 100, "exclude": "", "results_limit": 20, "client_url": "http://127.0.0.1:8111", "display": "grid", "torrent_client": "deluge", "torrent_client_username": "admin", "torrent_client_password": "password", "download_dir": "/downloads" }

I've tried using the actual IP address as well. It doesn't make any difference it seems. Jackett can still be accessed while Deluge cannot. Everything runs from the same Docker on a Synology device. Deluge is version 1.3.15 if that makes any difference (the last one that works with the Windows UI and I have no problem accessing it even on WAN). Docker can (of course) be configured to only open certain ports. I've so far been operating the container with full access to the host network - just in the meantime while I get it working... 443 needs to be exposed at first to get PIP to work but I'm assuming after that, only the 2 ports used by Jackett and Deluge should be exposed, which I'll work out later.

Gabisonfire commented 4 years ago

My bad, I added the --verbose in the unreleased (git) version.

If everything works on the same docker, you should not need to but the host network, and access the containers directly by their names. The issue there might be the listening IP of deluge (should be 0.0.0.0). Can you do a netstat in the container to see if it listens? Can you reach Deluge's Web UI?

I doubt it's related to Raincoat.

trip5 commented 4 years ago

Hi again. I've tried multiple things. Netstat shows 0.0.0.0:8111 and 0.0.0.0:8112 LISTEN (run through a Synology BASH shell).

I can get Sonarr to communicate with Deluge with localhost:8112 (no username). So I tried that. I also have QBitTorrent running and Sonarr has no issues with that. Sonarr is also exposed to the host so no messy port remapping to think of.

Anyways, I've tried all combinations trying to get the magnet passed to either deluge or qbittorrent and it fails with exactly the same error as above.

My thought is that you use a helper to get the magnet passed on to the torrent? And that's the bit that's failing... I don't know where this error would come from: "Could not fetch torrent url: No connection adapters were found for 'magnet:?xt=urn:btih:..."

If relevant, I use Deluge Client: 1.3.15 / Server: 1.3.15 / libtorrent: 1.0.11.0

Gabisonfire commented 4 years ago

Hi,

Yes the error is returned from the helpers.py file, by the request module. I'll have to check as it should support magnet links right away, I might have an idea on what's causing the issue. I'll have a look most likely tonight but in the meantime, could you try with the latest version:

pip install git+https://github.com/Gabisonfire/raincoat.git

And the "--verbose" switch, it would help me greatly :)

trip5 commented 4 years ago

Hey again. I assume I'm missing something? I don't know how Python works or how to interpret this:

Traceback (most recent call last): File "/usr/local/bin/raincoat", line 5, in from raincoat.raincoat import main File "/usr/local/lib/python3.7/site-packages/raincoat/init.py", line 1, in from .raincoat import main File "/usr/local/lib/python3.7/site-packages/raincoat/raincoat.py", line 7, in from raincoat import shared ImportError: cannot import name 'shared' from 'raincoat' (/usr/local/lib/python3.7/site-packages/raincoat/init.py)

I've tried building using Python3.7.8-buster base (since it comes with git in the image already)... is the version of python deadly important? I also tried 3.8.4-buster... same... but it does add this info:

ImportError: cannot import name 'shared' from partially initialized module 'raincoat' (most likely due to a circular import) (/usr/local/lib/python3.8/site-packages/raincoat/init.py)

Gabisonfire commented 4 years ago

Hey,

I'm using Python 3.8.3, as long as you use python 3.7 you should not have any issues.

For your error, there was a missing file I forgot to push. I just pushed it, if you try again you should not have this error. I'm still working on the issue you are having.

trip5 commented 4 years ago

Still using Python3.8.4-buster as a base (for now - it's 1GB - try slim to get it down to 200MBish later):

Using port 8112 (the WebUI):

Using port 8111 (the daemon):

I double checked by deluge config. Looks like at some point, I did put in an auth file that has a username and password properly... so that's not it. Lol.

trip5 commented 4 years ago

And for good measure, tried it will Qbittorrent on 6880 and 6881 also. Same error in the log.

Gabisonfire / raincoat

Docker / Magnet Issues #7