webroot option

[nbonfire]

I'd like to put this behind a reverse proxy, so a webroot option in the config would be nice.

For example, https://myserver.com/comicstreamer would redirect through nginx to http://myserver.local:32500/comicstreamer

Right now that doesnt work.

[bonebox]

+1. This project appears abandoned so I doubt it will happen, but here's hoping.

[davide-romanini]

Project is not really dead, but proceeds very slowly. :-) I wonder if a webroot option is really necessary, maybe you can use rewrite rules for the same purpose??

[bonebox]

Glad to know someone is still working on it!

I'm really not too familiar with rewrite rules--do you have an example of how it might work for this? A Google search didn't return anything very useful.

[davide-romanini]

ProxyPass and rewrites are a lot hackish, but very useful. I'm not sure I fully understood the wanted beahviour, but at https://httpd.apache.org/docs/2.2/rewrite/proxy.html there are some informations, at least for apache. Nginx has similar support, you should refer to official documentation for details, for example: http://nginx.org/en/docs/http/converting_rewrite_rules.html, http://nginx.org/en/docs/http/ngx_http_rewrite_module.html.

[bonebox]

Thanks, yeah I've looked at all that. The ProxyPass directives work in conjunction with setting a webroot--I have them working fine with other web services that support webroot, such as Ubooquity, for example. I don't think it's possible to get it working correctly with just rewrite rules. I tried some stuff but none of it works--I can get the root page to show up (without styling, at least) but any links in ComicStreamer are based off a hard coded webroot of / so they break. That's where being able to specify the webroot in ComicStreamer comes into play.

[nbonfire]

Like bonebox said, rewrite rules alone aren't enough. They can convert an incoming url to whatever, but they can't rewrite the html going back to the client. For example, if you set up comicstreamer behind a proxy at myserver.com/comics, when you first load you will be immediately redirected to myserver.com/login (which is already used in my case so I can't just add another rewrite rule to handle that). Even if I could get past that, none of the images would work without a rewrite rule for any arbitrary uri getting redirected to /comics/whatever. So just using rewrite rules could work if comicstreamer was the only site being hosted behind that apache/nginx serverr, but if it were the only site being hosted you wouldn't need rewrite rules. Basically the minimum work that needs to be done to support this is

• a spot in the config for the webroot to be specified
• adjust templates to use the webroot in all links if specified.

The rest can be handled by rewrite rules.

[blindpet]

I would like to see webroot as well when you get around to it. SickRage uses tornado and may provide some code that you can use.

[davide-romanini]

From referenced commit 489cd1e I've added a webroot option, for example:

$ comicstreamer --webroot /comics

will expose all urls at http://localhost:32500/comics/. It's a quick and dirty fix, let me know if you find something broken eventually.

[bonebox]

Awesome, thanks!!! I'll check it out when I get a chance.

[bonebox]

I gave it a try, but haven't been able to get it to work with the same rewrite rules I'm using for my other sites. @davide-romanini did you test this and if so what did you use for rewrite rules? FYI, below is what I'm using. This works for my other sites that I have configured for reverse-proxy (with the webroot folder being different for each, of course).

ProxyPass /comics/ http://localhost:32500/comics/
ProxyPassReverse /comics/ http://localhost:32500/comics/

[nbonfire]

I won't have a chance to try this until the weekend. Did you get an error? Are the images not showing up? If the page loads, can you check the html source to see if the urls are rewritten to include the webroot?

[bonebox]

I can sort of get it to load by pointing to the local ip of my NAS like this http://x.x.x.x/comics/--but the theming is messed up and most links are broken. Adding back in the port fixes it `http://x.x.x.x:32500/comics/. It doesn't work at all for me when accessing from my domain which is the reason I need the reverse-proxy in the first place. I just get "page not found" error.

[davide-romanini]

I made a quick test with a python reverse proxy (https://github.com/metajack/tape)

  python tape -P /comics=http://localhost:32500/comics -P /static=http://localhost:32500/static

and it works. Pay attention to slashes (/), I tried three different wrong configurations before this one ;-)

[bonebox]

Yeah, I tried all different types of slashes, including that. The first thing that got me was the config; I mistakenly put just comics since the webroot setup on my other services don't require a slash I assumed it was the same. Looking again at your instructions I realized it needed to be /comics. I don't know what the problem could be, since that same setup works for my other services. I'll be interested to know if anyone else gets it working successfully.

[bonebox]

@davide-romanini out of curiosity when you say "it works", what do you mean exactly? What url are you testing to see that it works?

[bonebox]

Also, I just noticed that this partially breaks integration with Chunky comic reader (@mike-ferenduros). Even with the webroot field empty, Chunky gives a 404 when loading the ComicStreamer browser in-app. The folder navigation still works, though.

[davide-romanini]

I made only a quick test with tape as noted above. All pages and api calls worked for me. Note that also the /static path need to be reversed for correct browser display. Could you post here the relevant CS logs? You can find them in your home under .ComicStreamer/logs. Also your apache access logs could be useful.

[bonebox]

I added the reverse-proxy for /static and I can access it locally now without including the port, so that's progress, at least. I still cannot access it externally from my domain. I get a 400 Bad Request Illegal URL error. Did you test accessing it externally or just internally? BTW, what's the purpose of /static? That seems odd to also have to configure a reverse-proxy for that--is that part of the "quick and dirty fix" you mentioned?

Also, Chunky reader still does not work properly. It's almost the opposite as before--now the web browser will load, but browsing directly by folders (using the API, I'm assuming) doesn't work. Actually, the web browser doesn't really work correctly within Chunky either--once you browse a few links deep or attempt to load a comic, it stops working.

I'll post the logs here in a bit.

[bonebox]

I'm not seeing much in the ComicStreamer.log. It does appear to be seeing my attempt to access externally, because every time I try, I see this in the log:

2015-08-28 10:06:09,602 - DEBUG - 302 GET /comics/ (127.0.0.1) 2.96ms

I also noticed that (some, not all of the time) the url (https://my.domain.com/comics/) gets redirected to https://my.domain.com/login?next=%2Fcomics%2F. I do have authentication enabled, if that is related.

When accessing via Chunky, I see stuff like the below. But it doesn't appear to be logging any errors for the API. For instance when a folder or comic won't load, nothing is getting logged.

15-08-28 10:12:23,157 - DEBUG - 200 GET /comics/scanstatus?api_key=*key_removed* (127.0.0.1) 3.07ms
2015-08-28 10:12:24,267 - ERROR - Could not open static file u'/volume1/@appstore/ComicStreamer/comicstreamerlib/static/blindpet_logo_crop_small.png'
2015-08-28 10:12:24,270 - DEBUG - 304 GET /comics/folders/browse (127.0.0.1) 67.34ms
2015-08-28 10:12:24,313 - DEBUG - 304 GET /comics/folders/?&api_key=*key_removed* (127.0.0.1) 3.05ms
2015-08-28 10:12:24,327 - WARNING - 404 GET /static/blindpet_logo_crop_small.png (127.0.0.1) 10.73ms
2015-08-28 10:12:25,673 - DEBUG - 200 GET /comics/scanstatus?api_key=*key_removed* (127.0.0.1) 3.47ms
2015-08-28 10:12:31,702 - ERROR - Could not open static file u'/volume1/@appstore/ComicStreamer/comicstreamerlib/static/blindpet_logo_crop_small.png'
2015-08-28 10:12:31,705 - DEBUG - 304 GET /comics/folders/browse/0 (127.0.0.1) 57.30ms
2015-08-28 10:12:32,066 - DEBUG - 304 GET /comics/folders/0?&api_key=*key_removed* (127.0.0.1) 325.43ms
2015-08-28 10:12:32,081 - WARNING - 404 GET /static/blindpet_logo_crop_small.png (127.0.0.1) 11.83ms

[bonebox]

I noticed that when I access the web browser from its local ip (which works, as I mentioned) the log is a little different. It is:

015-08-28 10:37:35,848 - DEBUG - 200 GET /comics/ (127.0.0.1) 2047.91ms

Followed by several similar lines like this:

2015-08-28 10:37:36,147 - DEBUG - 304 GET /comics/comic/7525/thumbnail?api_key=key_removed (127.0.0.1) 8.38ms

But when accessed externally it again looks like this (mentioned in previous post):

2015-08-28 10:06:09,602 - DEBUG - 302 GET /comics/ (127.0.0.1) 2.96ms

And it's not followed by anything else in the log.

So the successful load has a 200 GET, and the failed has a 302 GET.

[davide-romanini]

Ok, it definitely requires more testing :-) I confirm that authentication wasn't tested at all so it could be a problem. Could you provide more details of your setup and apache configuration so I can try to reproduce the exact use case?

[bonebox]

I'm running it on a Synology NAS (DS414). What details would you like to know? There are several apache config files that it's using--is there something specific you're looking for?

As I previously mentioned I'm running other services with reverse-proxy just fine. Those are CouchPotato, SickRage, Mylar, Ubooquity and Deluge web client. Here is what my reverse-proxy config looks like for those:

LoadModule ssl_module modules/mod_ssl.so

NameVirtualHost *:443
<VirtualHost *:443>
  ServerName www.mydomain.com
#  SSLCipherSuite HIGH:MEDIUM
  SSLProtocol all -SSLv2
  SSLCertificateFile /usr/local/keys/ssl.cer
  SSLCertificateKeyFile /usr/local/keys/ssl.key
  SSLEngine on
  SSLProxyEngine on
  ProxyRequests Off
  ProxyVia Off
  <Proxy *>
     Order deny,allow
     Allow from all
  </Proxy>
  ProxyPass / https://localhost:5002/
  ProxyPassReverse / https://localhost:5002/

</VirtualHost>

ProxyPass /couchpotato/ http://localhost:5053/couchpotato/
ProxyPassReverse /couchpotato/ http://localhost:5053/couchpotato/

ProxyPass /mylar/ http://localhost:8090/mylar/
ProxyPassReverse /mylar/ http://localhost:8090/mylar/

ProxyPass /sickbeard/ http://localhost:8083/sickbeard/
ProxyPassReverse /sickbeard/ http://localhost:8083/sickbeard/

ProxyPass /ub/ http://localhost:2202/ub/
ProxyPassReverse /ub/ http://localhost:2202/ub/

##Deluge##
<Location /deluge>
RequestHeader append X-Deluge-Base "/deluge/"
ProxyPass http://localhost:8112
ProxyPassReverse http://localhost:8112
Order deny,allow
Deny from all
Allow from all

</Location>

[davide-romanini]

I got a chance to do some more testing:

• started up a VM with ubuntu mate
• installed stock apache2 and enabled mod_proxy
• added the following config to /etc/apache2/sites-enabled/000-default.conf:

ProxyPass /comics/ http://localhost:32500/comics/
ProxyPassReverse /comics/ http://localhost:32500/comics/

• started up comicstreamer ./comicstreamer --webroot "/comics"
• started up a firefox at http://localhost/comics/

All functionalities work as expected, at least at browser level (didn't tried with chunky). At first I also had to proxypass the `/static' path. I just commited a little change to avoid even this step and it seems to work fine.

[bonebox]

Yeah, I can access it internally--from a previous comment:

I added the reverse-proxy for /static and I can access it locally now without including the port, so that's progress, at least. I still cannot access it externally from my domain. I get a 400 Bad Request Illegal URL error. Did you test accessing it externally or just internally?

The problem is I cannot access it externally via my domain like I can with my other services, which is really the main reason for using a reverse-proxy.

[bonebox]

I updated to the latest and was able to remove the proxypass for /static as you mentioned to access locally. But I still have the same problem externally. Just to reiterate, whenever I access externally (https://domain/comics/) the link gets changed to https://domain/login?next=%2Fcomics%2F along with a page not found.

[davide-romanini]

The problem is related to authentication, I've activated it but since the cookie was already set it just didn't redirected me to the login page. I've added the missing webroot piece even to that part, hoping it finally solves your problems.

[bonebox]

That did the trick--thanks!!!

[bonebox]

Unfortunately, Chunky still doesn't work. I'm assuming something probably needs to be updated in the app to support this change. @mike-ferenduros could you please look into this when you get a chance?

[bonebox]

This is working now which is great, but one thing I noticed is I'm using https on my server and it's working ok with my other services, but when I connect to ComicStreamer using https, it doesn't use the encrypted connection. Is there something with ComicStreamer that might be preventing/overriding this?

[nbonfire]

you should be able to use a redirect rule to force https. for example, in nginx I use this in my server block:

rewrite ^ https://$server_name$request_uri? permanent;

[bonebox]

I'm already forcing it for all traffic, which is what's strange. It only does this with ComicStreamer. My other services are http as well but get redirected fine.

[bonebox]

I get a page won't load when I do that, and the other are outside and work correctly.

[weidercs]

also, if you want the pages to load correctly behind a https reverse proxy, you'll need to change the script sources to be protocol relative: example on folders.html template:

<script src="http://ajax.googleapis.com/ajax/libs/jquery/2.0.0/jquery.min.js"></script>

to:

<script src="//ajax.googleapis.com/ajax/libs/jquery/2.0.0/jquery.min.js"></script>

[bonebox]

Ah, that may explain the behavior I'm seeing then. That's something that needs to be changed within ComicStreamer, right?

[weidercs]

yes, many of the .html files in the /comicstreamerlib/templates folder have that reference, I used a command line script to change them all so I'm not sure how many...

[weidercs]

command I ran to mass replace if it helps you...

sed -i .bak 's|http://|//|g' *.html

[bonebox]

Thanks, maybe you could submit a pull request with those changes or otherwise hopefully @davide-romanini can look at making the update.

[blindpet]

I think the ideal situation is to have http replaced with a variable in all the files so it can become http or https depending on a user setting.

[weidercs]

that's what my change does... I'll work on getting a branch/pull req setup.

[weidercs]

Also, as a hack to get Chunky to work, I just ran a 2nd instance of comicstreamer using a new port and setting the webroot to ''

./comicstreamer -p 32600 --webroot ''

[bonebox]

FYI I worked with the Chunky dev to get support for this in the app. He has a working version on TestFlight, should be released soon.

[vmcosco]

@davide-romanini I have been trying to get your fork to run to take advantage of the webroot option. I cant install libunrar via pip. I am on Windows 7. Wasn't sure how to else to contact someone for assistance

[blindpet]

@vmcosco I just realized I wasn't following the new fork (my apologies @davide-romanini (I will update the tutorials to point to yours).

It looks like paver libunrar just compiles an unrar binary. On Windows you should be able to just copy unrar.exe into the libunrar folder, unless it needs to be part of the system path then create an environment variable for it.

On Linux we should be able to symlink the unrar bin

@davide-romanini can you confirm this? On linux would it be possible to just symlink libunrar/unrar to the /usr/bin/unrar on Debian?

[vmcosco]

@blindpet I copied unrar.exe to the libunrar folder and everything worked. I was able to get nginx reverse proxy to work with https://mydomain.org/comics/ from outside my network. I get "Connection Failed" in Chunky however. Has anyone had any luck getting that working?

[bonebox]

A beta version of Chunky has been out for months now that has support for this. Hit up @ChunkyReader on Twitter for a TestFlight invite.

[frameset]

No matter what I try on my server I cannot access it from a reverse apache proxy, I just get 400 Bad Request every time. I've added "/cstreamer/" as my webroot, and I've got the following in my apache2.conf file:

ProxyPass /cstreamer/ http://localhost:32500/cstreamer ProxyPassReverse /cstreamer/ http://localhost:32500/cstreamer

[bonebox]

A couple things:

Try putting a forward slash on the end of your url, like so:

ProxyPass /cstreamer/ http://localhost:325000/cstreamer/
ProxyPassReverse /cstreamer/ http://localhost:325000/cstreamer/

Also, 32500 is the default ComicStreamer port, not 325000. So unless you changed it to 325000, you'll want to fix that as well.

[frameset]

Nope, same result, 400 bad request.

On 11 April 2016 at 15:51, Jeremy Bone notifications@github.com wrote:

A couple things:

Try putting a forward slash on the end of your url, like so:

ProxyPass /cstreamer/ http://localhost:325000/cstreamer/ ProxyPassReverse /cstreamer/ http://localhost:325000/cstreamer/

Also, 32500 is the default ComicStreamer port, not 325000.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/beville/ComicStreamer/issues/24#issuecomment-208383488

[bonebox]

Can you access ComicStreamer locally? What port is it running on?

[frameset]

I'm running it on the default 32500 port. I can access it locally (though the only browser on the server I have is lynx) It is also accessible from outside the network at http://domain.name:32500.

On 11 April 2016 at 15:57, Jeremy Bone notifications@github.com wrote:

Can you access ComicStreamer locally? What port is it running on?

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/beville/ComicStreamer/issues/24#issuecomment-208385250