Default user-agent string can cause issues with Nzb.su

[carze]

I've recently setup an instance of XDM on my Windows 7 desktop and noticed that I had a lot of trouble getting it to work with Nzb.su as my Newznab provider.

After mucking around in the logs and in the code I discovered that the default user-agent string on my desktop:

'User-Agent': 'python-requests/1.1.0 CPython/2.7.6 Windows/7'

Causes a Forbidden 403 to be returned from the API call. Switching the user-agent string to 'XDM' (or in my case I put XDM 0.23) seems to solve the issue.

I am not sure if this is just an issue in Windows, have not tried running XDM in Mac to see if the issue also occurs there.

[pannal]

https://github.com/lad1337/XDM-main-plugin-repo/issues/23

You can find a customized NZB.su plugin in my repository: https://github.com/pannal/XDM-pannal-plugin-repo

[carze]

Seems like I can't get the exact version of XDM that your plugins support. Upon loading up your repository I get the following warning:

"Plugin requires XDM version Zim 0.5.19 you have Zim 0.5.18"

I've switched my check-out of XDM to the develop branch and that didn't seem to help.

[pannal]

Well yeah, my core changes to XDM haven't been merged yet. You can try using https://github.com/pannal/XDM/tree/develop, though there may be DB merging issues.

Edit: I've just updated my NZB.su and SABnzbd plugins to support 0.5.18, as it doesn't require my core changes as AdvancedMover does.

[carze]

Thanks, will give that a shot.

[pannal]

Please check my updated comment. You don't need to switch to my forked XDM if you want my (fixed) SABnzbd and NZB.su plugins.

[pannal]

In general my SABnzbd fork should be more robust than the current available one in the core repo, you might want that, too, though it's still in development.

[carze]

Your SABnzbd fork installs fine but when I got to install the NZBsu plugin it goes through the motions of downloading and extracting the files but upon refresh I still have the option to install the plugin. Doing that just throws an error.

Going into the System Setting I am not seeing NZBsu.

EDIT: Figured it out, seems like the defaultCategoryMap dictionary was missing entries for Anime and Music and this was crashing the plugin from installing. Not sure if this is because I am using 0.5.18 and you made some changes in your version of XDM but adding those two categories resulted in the plugin loading up for me.

[pannal]

For me currently nzb.su is unavailable - can you give me the remaining category IDs so I can fix them in NZB.su plugin?

They seem to have DNS problems atm.

[carze]

Yea it looks like the site just went down. Right as I was attempting to gather the ID's for the categories.

It seems like you also need to add a user-agent string when grabbing the category ID's. I just copied and pasted them into the _gatherCategories function.

Before I had just slotted them into self by adding the headers into the Indexer class definition in base.py but not sure if this was a clean way to do it.

Will get you those category ID's as soon as the site comes back up.

[pannal]

I've just updated the plugin to not fail on unexistant categories (I wasn't too keen to gather all of those categories as default values for defaultCategoryMap for my replacement for games categories). The category ID gathering shouldn't fail, though, I've had that working while the API threw its user-agent 403 errors.

Also updated the gather-categories function to honor the new configurable user-agent config option (just for safety).

[carze]

Thanks for all your work. Appreciate you looking into this for me.

[pannal]

Updated to 0.7.23

And: no problem, I'm working on some areas of XDM to replace my stack of couchpotato/nzbdrone and to make it as usable as possible, so I will stick to fixing stuff when I have the time :)

Edit: nzb.su seems to be up again. Edit2: Please feel free to submit bugs to https://github.com/pannal/XDM-pannal-plugin-repo

[lad1337]

Honestly this is awesome ... A complet New Feature/plugin bug fix without me doing anything :)