Problems with sending files to SABNZBD

[fant0mas]

Hello,

I seem to be having a problem with sending files to SABNZBD in the recent version of Mylar. While the logs are indicating that the send is successful, I'm not seeing any new downloads appear in SABNZBD. I've verified that the NZB API key works and this seems to be the only application that I am having problems with sending new issues. This has worked in the past and I have not made any configuration changes which makes this issue a little strange.

Below is my pastebin for my mylar.log:

http://pastebin.com/tueyCUCj

Please let me know if you have any suggestions on how to resolve this issue. Thanks!

[evilhero]

The log posted doesn't show anything being found, let alone being sent to sabnzbd. You'll have to post another log showing a send being tried.

If you initiate a search for an issue you know is on a provider, and then it fails to send, check the mylar.log file right away (or shutdown mylar in order to check the log after a fail). If it's not in mylar.log, then it could be in one of the numbered log files.

It could be related to your version of python, or if your mylar and sabnzbd aren't on the same machine (this has been acknowledged in another issue).

[fant0mas]

Sorry about that, I think I posted the wrong log earlier. I modified my original pastebin link. Does that have any additional information?

[evilhero]

Does /opt/mylar/cache exist on the same machine as death?

And what version of python are you running mylar against?

In the version of mylar that you're running, mylar attempts to send the nzb directly from the local cache directory, instead of using the url link for the provider. This will be fixed in the next development commit, but if mylar and sab are on different machines, then you'll need to wait for the fix (or you can try to symlink/map the cache directory back to a location that can be accessed by the Sabnzbd machine)

[fant0mas]

Yeah they're on different machines - Ok that explains it, I'll give that a shot and see if that fixes things. Thanks for the quick responses! :)

[TBhimdi]

I've got Mylar on my local machine and Sab and NZBMega on the same VPS. With this new commit, I'm assuming I have to check the box in Mylar that flags as different machines and enter my IP address (instead of 0.0.0.0). A few questions:

1. Does this mean I now have to open a port in my router so that Sab (on my VPS) can access my local machine? I didn't have to do this before.
2. Since Sab and NZBMega are on the same VPS, is there a way to "skip" the Mylar API like before? I'd rather not open ports on my local machine. I know Mylar does some checking/parsing now for validity, but I'd rather have it error out than have an open port. Personal preference.
3. Can I enter my Dynamic DNS domain, instead of the actual IP address of my local machine?

[evilhero]

Well I'll try to answer the questions as best I can from a theoretical standpoint since I can't verify if it would work or not:

1. You shouldn't have to open up any ports that aren't already opened. If you used any type of post-processing from sabnzbd to mylar previously than the port (default of 8090) would already be open.
2. The problem is that going back to the sendurl provider directly to sab method is that if either sab or the provider use https then there's a good chance it won't send properly. The whole downloading first and then sending directly to Sab is a way around this limitation which is primarily due to python and sni certificates which screwed everyone over. So me altering it back for one user (or maybe several), would break it again for the vast majority - at least at this stage. But like I said if you were already doing post-processing with mylar then the required port is already open anyways since the machine has to listen on 8090 for any Incoming connection. And if I let it just error out on invalid files, it would error out on everything sent (as described above). I can look into local bypasses, but not until I know that the api method being used now is working properly for everyone.
3. You can try, as long as your router accepts connections to that dynamic dns. Routing after that point would be on you because mylar would only be listening on that address, not the ip at all. As a side note, if you're using dynamic dns then you'd need to have port 8090 open already in order to flow things through (unless you're doing internal routing, or reverse proxy type stuff).

[TBhimdi]

Thanks for the quick answer. It's theoretical here as well, I'm just waiting for Mylar to do a regular search and grab and see how it goes (currently no Wanted issues). Unfortunately I'll be out of town for a few weeks so can't check until the last week of June.

1. No post-processing being done. I didn't have any ports open before, sounds like they should be fine now too then, port-wise.
2. Makes sense, wasn't aware of the Python issue, definitely don't add a workaround for a small subset of users (like me). I'll figure something out.
3. I'll give the DDNS a shot, otherwise stick with IP. I'm just worried that one day Mylar will stop working because my IP changed and the only way I'll find out is by realizing that nothing has come through in a few weeks.

I would set up Mylar on my VPS, but can't think of an easy way for it to scan the library on my local machine without going through a VPN or mounting shares, etc. I'm all ears if people have any suggestions.

[TBhimdi]

Just a quick update. I think I'm having a hard time understanding this, albeit I haven't played around with it as much as I'd like. I've shutdown Mylar for now (back up in a couple weeks when I return from travels, more testing then if needed).

I've got "difference machines" box checked, because it's true (home PC has mylar and VPS has nzbmega/sab). 0.0.0.0 obviously doesn't work when Sab receives the url (0.0.0.0 shows up), as well as entering the LAN IP for my Home PC (same thing, why would my VPS recognize my home lan ip anyway, just wanted to try it). But unfortunately entering the WAN IP in Mylar doesn't work either because then Mylar won't start up (python socket cannot be opened error, tried running as admin as well).

Assuming entering the WAN IP did work, wouldn't Sab still have to connect to my Home PC on port 8090 to Mylar, thus requiring keeping port 8090 open on my Home PC? Keep in mind, no post-processing is being done, I just want Mylar to check nzbmega and sab to download the nzb url it receives and save it locally on the VPS (I transfer/sync them manually later).

Also, another issue I found was that I didn't have the API in Mylar enabled, which is fine, BUT I didn't have an API key generated, which gave no error and silently failed. Only reason I found this out was by manually copying/pasting the url Sab was checking into Chrome and seeing an "API Key not generated" (or something similar).

[TBhimdi]

How is everybody changing their Mylar IP to their WAN address and getting it to start up? I keep getting this error and have to set it back to 0.0.0.0 to even run. This of course doesn't work when my Sab is running on an Ubuntu VPS in another state and can't download what Mylar sends it.

Portable Python 2.7.6.1 on Windows over here.

19-Jun-2015 13:04:28 - INFO    :: MAIN : Starting Mylar on http://xxx.xxx.xxx.xxx:8090/
19-Jun-2015 13:04:29 - ERROR   :: HTTPServer Thread-1 : Uncaught exception: Traceback (most recent call last):
  File "C:\Util\daemon\mylar\mylar\logger.py", line 159, in new_run
    old_run(*args, **kwargs)
  File "C:\Util\Devel\python\App\lib\threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "C:\Util\daemon\mylar\cherrypy\process\servers.py", line 187, in _start_http_thread
    self.httpserver.start()
  File "C:\Util\daemon\mylar\cherrypy\wsgiserver\__init__.py", line 1753, in start
    raise socket.error(msg)
error: No socket could be created

[evilhero]

Have you tried another port other than 8090 - or more importantly making sure that the port 8090 is open on the mylar machine? There may not be another way around it - you need to open up the port for access in all likelihood.

Or 127.0.0.1 (provided you have an entry in your hosts file to allow for that)

I can understand why the WAN wouldn't be working if you don't have the port open on the machine (it wouldn't apply for a LAN/localhost IP as it wouldn't have to connect outside to verify the open port).

[TBhimdi]

-Check via netstat if anything was using the default or different port, "just in case", all clean. -Tried a different port with WAN IP, same error. -Tried default port, WAN IP, opened 8090 on my router and forwarded it to my machine, same error. -Default port, used 127.0.0.1 and Mylar ran successfully, but just as with 0.0.0.0 and localhost, Sab on my VPS gets the URL, but not the NZB.

Here's what Sab displays for 127.0.0.1 & 0.0.0.0 & localhost (swap out the address in the URL for 0.0.0.0 and localhost, respectively):

WAIT 44 sec / Trying to fetch NZB from http://127.0.0.1:8090/api?apikey=XXXapikeyXXX&cmd=downloadNZB&nzbname=example.nzb

[evilhero]

Hmm, I wonder - if you set the IP to 0.0.0.0 (so that it listens on all IPs on that machine), and maybe if I can add in an extra option somewhere that lets you specify an external IP for mylar to be referenced from the SABnzbd url line....

[TBhimdi]

That would work. If I could enter a domain (which I have set up in my router to update with my WAN via DDNS), that would be better. If you get really bored, you could always use STUN or UPNP to check what the external IP address. That might be too much work for little gain though.

Looks like I'll have to open the Mylar port on my router to allow my VPS access to my home machine in all these solutions, even though I'm not using post-processing.

[evilhero]

Hmm..STUN might be usable enough - simple source code, easy to add into library as a module...I just have to figure out a way for users to have the option to use it or not when it comes to sending back (because if it's working as it is, there's no need to try to go external ip)

Give me abit, I'm playing with it now ;)

Actually if you're using STUN right now - can you do the following from a python prompt (assuming you did 'pip install pystun' already):

import stun
sip = "0.0.0.0"
port = 8090
nat_type, external_ip, external_port = stun.get_ip_info(sip,port)

[evilhero]

Ok, I haven't implemented STUN just yet, but did a really quick hack that you can try. Make sure you shutdown mylar, then edit the config.ini (this is after you update to the latest development commit when it gets pushed). Edit the line: host_return = "" and put in the complete url to your external IP, so as an example: host_return = http://myblender.isanin.ja:8090

Save the config.ini, and statup mylar and hopefully that will work (note you probably need to leave the mylar HOST IP set to 0.0.0.0 still). It should then start retrieving nzbs from that host_return address & IP instead of whatever the mylar HOST IP is set to.

I'll be pushing this out in the next few hours, so hopefully this will address the problem you're having. I'm also looking into STUN implementation (it looks neat), but just need to figure a clean way to implement things...

[TBhimdi]

Install pystun and ran the code up top:

Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import stun
>>> sip = "0.0.0.0"
>>> port = 8090
>>> nat_type, external_ip, external_port = stun.get_ip_info(sip,port)
>>> print nat_type, external_ip, external_port
Restric NAT xxx.xxx.xxx.xxx 8090

It displayed my correct external IP, that's the only thing I changed/hid.

I'll be out of town again this entire week for work, can test more if needed once I return. Thanks again for this.

From a quick Google, looks like pystun goes through a default list of STUN servers. My VPS is primarily an Asterisk server, so I've set up my own STUN server on it as well for my own personal uses. I'll see if it's possible to use my own server with pystun in the code above later.