search

inventree / InvenTree

Open Source Inventory Management System
https://docs.inventree.org
MIT License
4.22k stars 760 forks source link

Add support for hosting on subdir #1297

Open sintech opened 3 years ago

sintech commented 3 years ago

Currently the only supported way to host inventree project is domain root. http://inventree.com/ It would be more flexible if one could host it on subdir too, e.g. http://example.com/inventree

I thought that changing STATIC_URL and MEDIA_URL will be enough, but it seems that / prefix is used in a lot of scripts and templates including redirect to /index in urls.py

SchrodingersGat commented 3 years ago

Huh, I never considered this. But yeah, that's a problem!

I'm going to start a branch with support for this. Can you help me out with testing?

For now, can you please let me know where you are coming across errors due to links not working properly?

Thanks

SchrodingersGat commented 3 years ago

Oh and also the changes that you initially made to STATIC_URL and MEDIA_URL

sintech commented 3 years ago

Sure I can help with testing. I tried to deploy inventree on my server using Apache + mod_wsgi.

I implemented 'url_prefix' config parameter (almost as you did in #1298) url_prefix: '/inventree' in config.yaml and in settings.py URL_PREFIX = CONFIG.get('url_prefix','') STATIC_URL = URL_PREFIX+'/static/' MEDIA_URL = URL_PREFIX+'/media/'

And I changed all /part to part in InvenTree/templates/js/part.js just to check it is working. I guess there are a lot of places where links should be changed to relative to work properly with subdirs.

Besides that, I had errors in inventreeCommitHash() and inventreeCommitDate() from version.py because git command was executed in root / directory. I fixed it using cwd param: return str(subprocess.check_output('git rev-parse --short HEAD'.split(),cwd=os.path.dirname(os.path.realpath(__file__))), 'utf-8').strip()

SchrodingersGat commented 3 years ago

@sintech could you please check the branch in #1298 and see what still needs work?

sintech commented 3 years ago

So partially it works but some links are still wrong.

My server configuration (apache 2.4 + mod_wsgi):

# InvenTree
Alias /inventree/static /var/www/inventree/inventree-subdir/inventree_static
<Directory /var/www/inventree/inventree-subdir/inventree_static>
 Require all denied
 Require ip <my ip>
</Directory>
Alias /inventree/media /var/www/inventree/inventree-subdir/inventree_media
<Directory /var/www/inventree/inventree-subdir/inventree_media>
 Require all denied
 Require ip <my ip>
</Directory>
WSGIProcessGroup inventree
WSGIDaemonProcess inventree python-home=/var/www/inventree/inventree-env home=/var/www/inventree/inventree-subdir/InvenTree python-path=/var/www/inventree/inventree-subdir/InvenTree
WSGIScriptAlias /inventree /var/www/inventree/inventree-subdir/InvenTree/InvenTree/wsgi.py process-group=inventree
<Directory /var/www/inventree/inventree-subdir/InvenTree/InvenTree>
   <Files wsgi.py>
       Require all denied
       Require ip <my ip>
   </Files>
</Directory>
SchrodingersGat commented 3 years ago

@sintech can you please post your settings.py file after your modifications?

sintech commented 3 years ago

Sure, here is diff:

# git diff settings.py 
diff --git a/InvenTree/InvenTree/settings.py b/InvenTree/InvenTree/settings.py
index 98b7fa8..e93abc5 100644
--- a/InvenTree/InvenTree/settings.py
+++ b/InvenTree/InvenTree/settings.py
@@ -146,9 +146,9 @@ SUBPATH_URL = get_setting(
 FORCE_SCRIPT_NAME = SUBPATH_URL

 # Web URL endpoint for served static files
-STATIC_URL = '/static/'
+STATIC_URL = 'static/'

-API_URL = '/api/'
+API_URL = 'api/'

 if SUBPATH_URL:
     STATIC_URL = os.path.join(SUBPATH_URL, STATIC_URL)
@@ -170,7 +170,7 @@ STATICFILES_DIRS = [
 STATIC_COLOR_THEMES_DIR = os.path.join(STATIC_ROOT, 'css', 'color-themes')

 # Web URL endpoint for served media files
-MEDIA_URL = '/media/'
+MEDIA_URL = 'media/'

 if SUBPATH_URL:
     MEDIA_URL = os.path.join(SUBPATH_URL, MEDIA_URL)
@@ -516,3 +516,5 @@ DBBACKUP_STORAGE_OPTIONS = {
 INTERNAL_IPS = [
     '127.0.0.1',
 ]
+
+CSRF_COOKIE_PATH = '/inventree'
rcludwick commented 3 years ago

Apologies for coming in late. Doesn't nginx/apache support transparently rewriting of urls in the reverse proxy mode?

https://serverfault.com/questions/379675/nginx-reverse-proxy-url-rewrite

That would probably be the proper way to handle this.

SchrodingersGat commented 3 years ago

@rcludwick I would like to push this problem outside of the scope of the InvenTree server itself.

I just tried configuring a nginx setup as per your example but could not get it to work. Have you had specific experience with this?

matmair commented 3 years ago

@SchrodingersGat a transparent rewrite /proxy seems like a good idea. But I would leave that to the admin of the webserver, that works with the current setup relatively easy.

SchrodingersGat commented 3 years ago

@matmair do you have some more details on this? I'd like to know that it can be made to work well by (for e.g.) nginx, so that I can close out this issue. I'd rather not have InvenTree handle this problem.

I tried to implement the url rewrite as per the suggestion of @rcludwick above.

It worked somewhat - a request for localhost:8000/subdir/part/ correctly sent me to /part/ InvenTree view.

However, the InvenTree page then requested static files from /static/ which the proxy tried to serve from localhost:8000/static/ and this then failed.

rcludwick commented 3 years ago

Yeah, it's been a while though. I'd have to do a little research. I'm vastly more familiar using traefik as a reverse proxy than nginx lately.

Basically you set up nginx to route by path or by hostname.

I have a host named 'i' that routes to the same up address and port, but is routed inside traefik to inventree.

You should also be able to route my.domain/inventree/* to rewrite the url to remove the root path.

On Wed, Jun 16, 2021, 7:53 AM Oliver @.***> wrote:

@rcludwick https://github.com/rcludwick I would like to push this problem outside of the scope of the InvenTree server itself.

I just tried configuring a nginx setup as per your example but could not get it to work. Have you had specific experience with this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/inventree/InvenTree/issues/1297#issuecomment-862399807, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAL67I4NWVJ76SIV4VI5353TTCUEDANCNFSM4XSJG5HA .

rcludwick commented 3 years ago

Apologies for taking a while. proxy_pass on nginx is the way to go here. It rewrites the url before handing it off.

https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/

Apache might do it too, but it's been around 2 decades since I last touched apache. mod_rewrite maybe? I don't know.

I would close this and add documentation on at least for nginx.

marklinmax commented 3 years ago

Sorry to jump in like that, but isn't nginx proxy_pass and url rewrite one-way? The server will effectively receive the request without the subpath, but will not redirect well since it isn't aware of the modification.

I tried configuring it like that and could reach the server but never got a proper response.

Maybe I am just doing something wrong?

So I think that both the INVENTREE_SUBPATH environment variable and nginx proxy_pass / url rewrite are needed.

rcludwick commented 3 years ago

I think this will do what you're asking for:

https://stackoverflow.com/questions/41605137/how-to-remove-a-subdirectory-from-the-url-before-passing-to-a-script

On Fri, Jul 2, 2021, 12:34 PM marklinmax @.***> wrote:

Sorry to jump in like that, but isn't nginx proxy_pass and url rewrite one-way? The server will effectively receive the request without the subpath, but will not redirect well since it isn't aware of the modification.

I tried configuring it like that and could reach the server but never got a proper response.

Maybe I am just doing something wrong?

So I think that both the INVENTREE_SUBPATH environment variable and nginx proxy_pass / url rewrite are needed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/inventree/InvenTree/issues/1297#issuecomment-873185142, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAL67I3R7FNIQICEWFJU4ADTVYBELANCNFSM4XSJG5HA .

SchrodingersGat commented 3 years ago

@marklinmax I think you are correct, I've tried both approaches in separation but I believe that as you say, INVENTREE_SUBPATH and proxy pass will need to be configured.

chrisnoisel commented 3 years ago

I've started to play with Inventree last week and stumbled on the same issue. I strongly feels it should be handled by Inventree itself like some other popular web projects, relying on url rewriting feels like a temporary workaround. I gave a try at your branch and after some light changes, it seems to work pretty good.

Do you see some issues regarding this solution being merged in master branch ?

SchrodingersGat commented 3 years ago

@chrisnoisel I'm very keen to get this correctly implemented. One potential issue I've been thinking about is the many places in the code (particularly in the javascript files) with hard-coded URLs - for example:

https://github.com/inventree/InvenTree/blob/d9c2d061cc6518b936091af262bf3130fed4f46e/InvenTree/templates/js/company.js#L40

Do these URLs work correctly in your branch?

chrisnoisel commented 3 years ago

I've yet to test it on a non-empty database but indeed I still see a lot of occurrences. I did a quick list of search patterns to have and idea of what's left : 

[^a-zA-Z0-1_-]/part/
[^a-zA-Z0-1_-]/manufacturer-part/
[^a-zA-Z0-1_-]/supplier-part/
[^a-zA-Z0-1_-]/dynamic/
[^a-zA-Z0-1_-]/common/
[^a-zA-Z0-1_-]/stock/
[^a-zA-Z0-1_-]/company/
[^a-zA-Z0-1_-]/order/
[^a-zA-Z0-1_-]/build/
[^a-zA-Z0-1_-]/auth/
[^a-zA-Z0-1_-]/login/
[^a-zA-Z0-1_-]/logout/
[^a-zA-Z0-1_-]/settings/
[^a-zA-Z0-1_-]/edit-user/
[^a-zA-Z0-1_-]/set-password/
[^a-zA-Z0-1_-]/admin/
[^a-zA-Z0-1_-]/error_log/
[^a-zA-Z0-1_-]/admin/
[^a-zA-Z0-1_-]/shell/
[^a-zA-Z0-1_-]/admin/
[^a-zA-Z0-1_-]/accounts/
[^a-zA-Z0-1_-]/index/
[^a-zA-Z0-1_-]/search/
[^a-zA-Z0-1_-]/stats/
[^a-zA-Z0-1_-]/auth/
[^a-zA-Z0-1_-]/api/
[^a-zA-Z0-1_-]/api-doc/
[^a-zA-Z0-1_-]/markdownx/
[^a-zA-Z0-1_-]/static/
[^a-zA-Z0-1_-]/media/
grep -R -f ./patterns src/

The list is big but I'm pretty sure most of it can be automated (and I guess i18n are auto-generated ?). I just need to be sure I'm doing it "the right way" as I'm pretty new to Django. For now, I expect most of the manual stuff will be switching some files from static to dynamic as I did with inventree.js

matmair commented 3 years ago

@chrisnoisel great idea. Please merge in the current master before you open a PR as there are some conflicts that need to be resolved.

rcludwick commented 3 years ago

nginx sub_filter will rewrite the urls on responses. So, I still think this is better handled outside of inventree. nginx is pretty powerful, so I prefer to move custom url behavior to the proxy whenever possible. Apologies for not finding this sooner. I have been using traefik for the most part these days instead of nginx.

http://nginx.org/en/docs/http/ngx_http_sub_module.html

It would also be able to rewrite a javascript subpath, particularly if you exposed a variable in the javascript that would be easily translated. At that point it's just a matter of giving the users a working nginx config.

The question is, is that enough? The only thing I can think of is if an email is sent with a link to the main server that won't have the subpath -- because that won't go through nginx. So in that case you'd need a "BASE_URL" variable, but only for generating emails.

matmair commented 3 years ago

@rcludwick if possible (I am nobody in this project) I would prefer to keep all magic rewriting to a minimum. If you have to change something to work there should be one place and one file to change. It can be difficult to debug this kind of stuff with rewrites (often there is more than one if you have a multi-layer enterprise network). If we already need to implement a setting, we should respect it everywhere server-side.

rcludwick commented 3 years ago

@matmair. Currently I work in a multilayer enterprise network. But I can't expect every open source project to incorporate our SSO strategy. Also in such a network, you'll typically have the resources to run a load balancer which can rewrite url's and html responses(F5, nginx, etc).

Maybe I'm wrong but it seems to me this adds surprises to JS development with a use case that most developers won't test against. As a developer, first and foremost, that's my biggest concern.

SchrodingersGat commented 3 years ago

if possible (I am nobody in this project)

image

If "second highest contributor" is "nobody" ;)

SchrodingersGat commented 3 years ago

Thanks everyone for input on this. I'll be getting back to this issue soon, and will have some time to read through your comments thoroughly.

SchrodingersGat commented 3 years ago

Ok, I attempted again in to find a way to get this working, but was not successful...

I am happy for someone to implement this, and test, but as I have no personal need for this feature I'm not going to spend any more time on it.

Some other references I came across that may be useful for anyone who wants this feature implemented:

chrisnoisel commented 3 years ago

I intend to do it. I'm also planning to convert static paths (stuff like '/part/'+pk+'/bom' ), with the dynamic DRY naming scheme. By the way, some js scripts need to compose urls with variable which excludes the {% url 'name' %} method. Is that ok for you, if the dynamic js is something like :

url = "{% url 'urlname' '0000' %}".replace('0000', jsvariable);
SchrodingersGat commented 3 years ago

I intend to do it.

Fantastic!

Is that ok for you, if the dynamic js is something like :

url = "{% url 'urlname' '0000' %}".replace('0000', jsvariable);

I'm not sure that this will work - I think that the templating engine will try to find a URL matching pk '0000' before rendering it? But give it a go!

spectre-ns commented 2 years ago

This would be amazing. We currently use a reverse proxy to use a single IP address and HTTPS connection to serve multiple services. I would love to be able to put inventree behind the apache reverse proxy.

matmair commented 2 years ago

@spectre-ns the needed changes are easy but numerous so I would not hold my breath for it. Maybe use nginx to rewrite everything - I switched to that strategy when I moved from bare metal to k8 everywhere.

Sub filters - suggested by @rcludwick - seem like a good solution. I have not tried that myself but it should work.

spectre-ns commented 2 years ago

@matmair is there an alternative way to get it running https. I could put it on a subdomain with it's own ip address?

matmair commented 2 years ago

@spectre-ns that has nothing to do with this issue or InvenTree and is handled directly by your reverse proxy or webserver. Please look up how to setup https for your respective software. I recommend to use letsencrypt and certbot - you should be supplied with a setup wizard that way.

larchen commented 2 years ago

@rcludwick Do you have an example of how you can host this on a subdirectory with traefik? I can use the stripPrefix middleware but this doesn't handle internal links/redirects. How do you get internal links/redirects to add the subdirectory prefix to the urls?

matmair commented 2 years ago

@larchen not sure the UI will work hosted on a subdir, you would have to rewrite JS code in-flight. I know that is possible in theory with enterprise reverse proxies but have yet to see a successful deployment for InvenTree.

rcludwick commented 2 years ago

@larchen You would need to do something like this:

https://serverfault.com/questions/1050668/https-nginx-reverse-proxy-url-rewrite

matmair commented 1 year ago

Related to #3901

trial-n-error commented 10 months ago

Update: The list of sub_filter was previously not complete as I found out. I updated the sub_filters

Hi,

I also would like to use the inventory management in a subpath. I understood that you tried to implement it in the code base, but several locations needs to be changed. I also saw you redo the UI and plan probably to do the implementation in a future milestone. I think this is really great and the right way to handle long term.

In the meantime I would like to share the solution using nginx. Please note that you need to build nginx for this with either a 3rd party plugin called nginx_substitutions_filter (and refine the configuration at the end) or with the builtin plugin --with-http_sub_module.

The following location block for nginx will work with current software version 0.12.10 and redirects incoming requests from the Webserver under /inventree to the python process at 127.0.0.1:8000. It handles rewrites in both directions and redirects.

Best regards, Mario

        location /inventree {
            rewrite /inventree/(.*) /$1 break;

            proxy_set_header Host $host;
            proxy_set_header Accept-Encoding "";

            sub_filter_once off;
            sub_filter_last_modified on;

            sub_filter_types text/javascript application/json;
            sub_filter ' href="/static/' ' href="/inventree/static/';
            sub_filter ' content="/static/' ' href="/inventree/static/';
            sub_filter ' href=\'/static/' ' href=\'/inventree/static/';
            sub_filter ' href="/admin/' ' href="/inventree/admin/';
            sub_filter ' href="/settings/' ' href="/inventree/settings/';
            sub_filter ' href="/accounts/' ' href="/inventree/accounts/';
            sub_filter ' href="/part/' ' href="/inventree/part/';
            sub_filter ' href="/stock/' ' href="/inventree/stock/';
            sub_filter ' href="/build/' ' href="/inventree/build/';
            sub_filter ' href="/company/' ' href="/inventree/company/';
            sub_filter ' href="/order/' ' href="/inventree/order/';
            sub_filter ' href="/stats/' ' href="/inventree/stats/';
            sub_filter ' href="/notifications/' ' href="/inventree/notifications/';
            sub_filter '.href = `/part/' '.href = `/inventree/part/';
            sub_filter ' src="/static/' ' src="/inventree/static/';
            sub_filter ' src=\'/static/' ' src=\'/inventree/static/';
            sub_filter ' src="/media/' ' src="/inventree/media/';
            sub_filter ' action="/accounts' ' action="/inventree/accounts';
            sub_filter ' action=\'/search' ' action=\'/inventree/search';
            sub_filter ' src="/js/' ' src="/inventree/js/';
            sub_filter '"/api/' '"/inventree/api/';
            sub_filter '\'/api/' '\'/inventree/api/';
            sub_filter '`/api/' '`/inventree/api/';
            # static/script/inventree/inventree.js
            sub_filter '`/about/' '`/inventree/about/';
            sub_filter '\'/stats/' '\'/inventree/stats/';
            # api/part/category/?search
            sub_filter '"url":"/part/' '"url":"/inventree/part/';
            # api/part/?search
            sub_filter '"image":"/media/' '"image":"/inventree/media/';
            sub_filter '"thumbnail":"/media/' '"thumbnail":"/inventree/media/';
            sub_filter '"thumbnail":"/static/' '"thumbnail":"/inventree/static/';
            # api/...
            sub_filter '"image":"/static/' '"image":"/inventree/static/';
            sub_filter '"url":"/stock/' '"url":"/inventree/stock/';
            sub_filter '"url":"/company/' '"url":"/inventree/company/';
            sub_filter '"url":"/build/' '"url":"/inventree/build/';
            # settings
            sub_filter 'value=\'/settings/' 'value=\'/inventree/settings/';
            sub_filter 'action=\'/settings/' 'action=\'/inventree/settings/';

            # for static files, in my setup I have this in a different location block
                sub_filter ' href=\'/part/' ' href=\'/inventree/part/';
                sub_filter '.href = `/part/' '.href = `/inventree/part/';
        sub_filter ' src="/js/' ' src="/inventree/js/';
        sub_filter '"/api/' '"/inventree/api/';
        sub_filter '\'/api/' '\'/inventree/api/';
        sub_filter '`/api/' '`/inventree/api/';
        # static/script/inventree/inventree.js
        sub_filter '`/about/' '`/inventree/about/';
        sub_filter '\'/stats/' '\'/inventree/stats/';
                sub_filter 'url = `/part/' 'url = `/inventree/part/';
                sub_filter '`/part/' '`/inventree/part/';
                sub_filter '/part/new' '/inventree/part/new';
                sub_filter 'url = `/stock/' 'url = `/inventree/stock/';
                sub_filter '`/stock/' '`/inventree/stock/';
                sub_filter '`/build/' '`/inventree/build/';
                sub_filter 'renderLink(value, \'/build/\'' 'renderLink(value, \'/inventree/build/\'';
                sub_filter '`/company/' '`/inventree/company/';
                sub_filter '`/order/' '`/inventree/order/';
                sub_filter '`/supplier-part/' '`/inventree/supplier-part/';
                sub_filter '`/manufacturer-part/' '`/inventree/manufacturer-part/';

            proxy_redirect / /inventree/;

            proxy_pass http://127.0.0.1:8000;
        }
SchrodingersGat commented 10 months ago

@trial-n-error thanks for sharing this solution. Maybe some of the other users who need this function will find it useful in the interim

trial-n-error commented 10 months ago

Sure @SchrodingersGat , I am thankful for the great piece of software and sharing it! I enjoy it a lot!

Best regards, Mario

matmair commented 8 months ago

@SchrodingersGat we probably can support this in PUI with the current code structure - is this something we also want for CUI or should we pivot it to be solely in OUI? I think this is something that could fit into 0.15.0 and I would be willing to code it if there is still intrest

SchrodingersGat commented 8 months ago

I'm happy for this to be only available for PUI