search

nginx / unit

NGINX Unit - universal web app server - a lightweight and versatile open source server that simplifies the application stack by natively executing application code across eight different programming language runtimes.
https://unit.nginx.org
Apache License 2.0
5.37k stars 323 forks source link

Nextcloud Pretty URLs #959

Open tiredofit opened 1 year ago

tiredofit commented 1 year ago

Neat project. I have been a long time fan of Nginx and PHP-FPM as a pair and decided to embark on converting many of my docker images to using Unit.

I followed the guide for Nextcloud and the configuration was easy enough to insert in. Howver, since I was using an openID Connect plugin I was locked out from being able to access the site. The reason why is that my URLs now contained index.php in the URL example host.domain.tld/index.php/apps/etc. Previously with the Nginx examples I did not see that in the URL. I have years of URLs that may be bookmarked from users who with this new insertion would invalidate all of their previous links..

In short, does anyone have a better configuration to enable these sort of pretty URLs?

ac000 commented 1 year ago

The reason why is that my URLs now contained index.php in the URL example host.domain.tld/index.php/apps/et

Do you mean that if you do

$ curl http://localhost/apps/etc

that Unit turns that into http://localhost/index.php/apps/etc?

Perhaps if you could post your config (or at least what of it you can).

tiredofit commented 1 year ago

Yes that is correct. URLs now have index.php in between the host and the "apps".

This may be long, I'm not sure if I know how to fold the config but here we are:

I grabbed the configuration for the listeners, routes, and applications from the HOWTO on the Unit website.


{
  "listeners": {
    "0.0.0.0:80": {
      "pass": "routes",
      "forwarded": {
        "protocol": "X-Forwarded-Proto",
        "source": [
          "10.0.0.0/8",
          "172.16.0.0/12",
          "192.168.0.0/16"
        ]
      }
    }
  },
  "applications": {
    "nextcloud": {
      "type": "php",
      "targets": {
        "direct": {
          "root": "/www/nextcloud/"
        },
        "index": {
          "root": "/www/nextcloud/",
          "script": "index.php"
        },
        "ocm": {
          "root": "/www/nextcloud/ocm-provider/",
          "script": "index.php"
        },
        "ocs": {
          "root": "/www/nextcloud/ocs-provider/",
          "script": "index.php"
        },
        "updater": {
          "root": "/www/nextcloud/updater/",
          "script": "index.php"
        }
      }
    }
  },
  "settings": {
    "http": {
      "header_read_timeout": 600,
      "body_read_timeout": 600,
      "send_timeout": 600,
      "idle_timeout": 600,
      "max_body_size": 10737418240,
      "discard_unsafe_fields": true,
      "log_route": false,
      "server_version": true
    }
  },
  "access_log": {
    "format": "$time_local $remote_addr $host $status $method \"$request_uri\" $body_bytes_sent \"$header_referer\" \"$header_user_agent\"",
    "path": "/var/log/unit//access.log"
  },
  "routes": [
    {
      "match": {
        "uri": [
          "/build/*",
          "/tests/*",
          "/config/*",
          "/lib/*",
          "/3rdparty/*",
          "/templates/*",
          "/data/*",
          "/.*",
          "/autotest*",
          "/occ*",
          "/issue*",
          "/indie*",
          "/db_*",
          "/console*"
        ]
      },
      "action": {
        "return": 404
      }
    },
    {
      "match": {
        "uri": [
          "/core/ajax/update.php*",
          "/cron.php*",
          "/index.php*",
          "/ocm-provider*.php*",
          "/ocs-provider*.php*",
          "/ocs/v1.php*",
          "/ocs/v2.php*",
          "/public.php*",
          "/remote.php*",
          "/status.php*",
          "/updater*.php*"
        ]
      },
      "action": {
        "pass": "applications/nextcloud/direct"
      }
    },
    {
      "match": {
        "uri": "/ocm-provider*"
      },
      "action": {
        "pass": "applications/nextcloud/ocm"
      }
    },
    {
      "match": {
        "uri": "/ocs-provider*"
      },
      "action": {
        "pass": "applications/nextcloud/ocs"
      }
    },
    {
      "match": {
        "uri": "/updater*"
      },
      "action": {
        "pass": "applications/nextcloud/updater"
      }
    },
    {
      "action": {
        "share": "/www/nextcloud/$uri",
        "fallback": {
          "pass": "applications/nextcloud/index"
        }
      }
    }
  ]
}
ac000 commented 1 year ago

Thanks!

So, just trying to create a simple reproducer...

index.php

<?php
print_r($_SERVER);
?>

unit.json

 {                                                                              
    "listeners": {                                                              
        "[::1]:8080": {                                                         
            "pass": "applications/php"                                          
        }                                                                       
    },                                                                          

    "applications": {                                                           
        "php": {                                                                
            "type": "php",                                                      
            "targets": {                                                        
                "test": {                                                       
                    "root": "/home/andrew/src/php",                             
                    "script": "index.php"                                       
                }                                                               
            }                                                                   
        }                                                                       
    }
}
$ curl http://localhost:8080/apps/etc
Array
(
    [SERVER_SOFTWARE] => Unit/1.32.0
    [SERVER_PROTOCOL] => HTTP/1.1
    [PHP_SELF] => /index.php
    [SCRIPT_NAME] => /index.php
    [SCRIPT_FILENAME] => /home/andrew/src/php/index.php
    [DOCUMENT_ROOT] => /home/andrew/src/php
    [REQUEST_METHOD] => GET
    [REQUEST_URI] => /apps/etc
    [QUERY_STRING] => 
    [REMOTE_ADDR] => ::1
    [SERVER_ADDR] => ::1
    [SERVER_NAME] => localhost
    [SERVER_PORT] => 80
    [HTTP_HOST] => localhost:8080
    [HTTP_USER_AGENT] => curl/8.0.1
    [HTTP_ACCEPT] => */*
    [REQUEST_TIME_FLOAT] => 1695779440.1456
    [REQUEST_TIME] => 1695779440
)

So /apps/etc came through unchanged. In your case it sounds like you'd see REQUEST_URI as /index.php/apps/etc, is that right?

I don't immediately see anything in your config that would be re-writing the request_uri.

tiredofit commented 1 year ago

I'll try in the AM to see if I can get those variables to output while running a full blown copy of Nextcloud and report back.

I'm seeing something similar that someone may be experiencing the same thing here https://github.com/nextcloud/docker/pull/1610 in their statement of "Currently lacks support for some Nextcloud functionality, like pretty URLs".

lcrilly commented 1 year ago

Looks like Pretty URLs require some URI rewrite "magic" to work properly.

The magic itself is wrapped up in a Nextcloud occ script which (for Apache) creates some .htaccess rewrite rules, as described in this help article.

Until 1.30.0, Unit did not have the ability to rewrite URIs. So we can probably get this working now :)

tippexs commented 1 year ago

Hi Folks - Thanks @lcrilly for sharing the links! This is helpful! This needs some further investigation. I will find some time over the weekend to play around with Nextcloud on Unit and will update this issue. Anybody else steping over this issue can play around with it as well.

tippexs commented 11 months ago

I was able to create a working configuration BUT the issue is not with the rewrites at all. I am able to inject an index.php$request_uri in the request BUT the base URI of nextcloud is set to index.php. This has nothing to do with client-side rewrites as the links that are comming back from my requst ALWAYS contain index.php/x/y/z.

So my question is basically how to remove index.php from the links returned by nextcloud?

tippexs commented 11 months ago

Nevermind! Goi it working. Will post by Configuration after some cleanup.

tippexs commented 11 months ago

The rewrite to add the index.php in each request looks like a legacy thing THAT'S not needed with the newer version of NC I have used for testing.

My setup uses version 27.1.2.

My configuration:

{
  "listeners": {
    "*:80": {
      "pass": "routes"
    }
  },
  "routes": [
    {
      "match": {
        "uri": [
          "/.well-known/carddav",
          "/.well-known/caldav"
        ]
      },
      "action": {
        "return": 301,
        "location": "/remote.php/dav"
      }
    },
    {
      "match": {
        "uri": [
          "/.well-known/*"
        ]
      },
      "action": {
        "pass": "applications/nextcloud/index"
      }
    },
    {
      "match": {
        "uri": [
          "/build/*",
          "/tests/*",
          "/config/*",
          "/lib/*",
          "/3rdparty/*",
          "/templates/*",
          "/data/*",
          "/.*",
          "/autotest*",
          "/occ*",
          "/issue*",
          "/indie*",
          "/db_*",
          "/console*"
        ]
      },
      "action": {
        "return": 404
      }
    },
    {
      "match": {
        "uri": [
          "/core/ajax/update.php*",
          "/cron.php*",
          "/ocs-provider*.php*",
          "/ocs/v1.php*",
          "/ocs/v2.php*",
          "/public.php*",
          "/remote.php*",
          "/status.php*",
          "/updater*.php*"
        ]
      },
      "action": {
        "pass": "applications/nextcloud/direct"
      }
    },
    {
      "match": {
        "uri": "/ocs-provider*"
      },
      "action": {
        "pass": "applications/nextcloud/ocs"
      }
    },
    {
      "match": {
        "uri": [
          "/index.php",
          "index.php/*"
        ]
      },
      "action": {
        "pass": "applications/nextcloud/index"
      }
    },
    {
      "action": {
        "share": "/var/www/html$uri",
        "fallback": {
          "pass": "applications/nextcloud/index"
        }
      }
    }
  ],
  "applications": {
    "nextcloud": {
      "type": "php",
      "user": "www-data",
      "processes": {},
      "targets": {
        "direct": {
          "root": "/var/www/html/"
        },
        "index": {
          "root": "/var/www/html/",
          "script": "index.php"
        },
        "ocs": {
          "root": "/var/www/html/ocs-provider/",
          "script": "index.php"
        }
      }
    }
  },
  "settings": {
    "http": {
      "log_route": true
    }
  }
}

  1. The reference implementation in our docs is wrong. The ocm-provider is no longer a thing. I have to do some more checks about it but at least it is not part of the official nextlcoud docker image.

  2. Just with this configuration it was not possible to get the index.php off the request URI as nextcloud put it always back in. I have added the following to the config/config.php.

  'overwrite.cli.url' => 'http://localhost:8888',
  'htaccess.IgnoreFrontController' => true,

The overwrite.cli.url should already be added by the setup script. Make sure to change it IF your nextlcoud is behind a loadbalancer.

The htaccess.IgnoreFrontController looks like a quick and dirty fix to me as per this discussion. https://help.nextcloud.com/t/removing-index-php-from-the-nextcloud-uri/13055/9

Feel free to dig a little bit deeper what the root cause is for this issue.

  1. The log_route was added to inspect the router and validate what route was taken for a request. Something to notice: As nextcloud includes an index.html file in its document root Unit will pick this index.html with this route:
    {
      "action": {
        "share": "/var/www/html$uri",
        "fallback": {
          "pass": "applications/nextcloud/index"
        }
      }
    }

It will pick the route 6 from the routes array as you can see in the log:

2023/10/18 07:36:39 [notice] 104#105 *261 http request line "GET / HTTP/1.1"
2023/10/18 07:36:39 [info] 104#105 *261 "routes/0" discarded
2023/10/18 07:36:39 [info] 104#105 *261 "routes/1" discarded
2023/10/18 07:36:39 [info] 104#105 *261 "routes/2" discarded
2023/10/18 07:36:39 [info] 104#105 *261 "routes/3" discarded
2023/10/18 07:36:39 [info] 104#105 *261 "routes/4" discarded
2023/10/18 07:36:39 [info] 104#105 *261 "routes/5" discarded
2023/10/18 07:36:39 [notice] 104#105 *261 "routes/6" selected

2023/10/18 07:36:39 [notice] 104#105 *209 http request line "GET /index.php HTTP/1.1"
2023/10/18 07:36:39 [info] 104#105 *209 "routes/0" discarded
2023/10/18 07:36:39 [info] 104#105 *209 "routes/1" discarded
2023/10/18 07:36:39 [info] 104#105 *209 "routes/2" discarded
2023/10/18 07:36:39 [info] 104#105 *209 "routes/3" discarded
2023/10/18 07:36:39 [info] 104#105 *209 "routes/4" discarded
2023/10/18 07:36:39 [notice] 104#105 *209 "routes/5" selected
2023/10/18 07:36:39 [info] 3178#3178 "nextcloud" application started

The index.html document redirects the application to index.php. So the next route will be the index.php route I have created. For legacy support - this is the place we can add the rewrite option 

{
      "match": {
        "uri": [
          "/index.php",
          "index.php/*"
        ]
      },
      "action": {
        "rewrite": "index.php$request_uri",
        "pass": "applications/nextcloud/index"
      }
}

BUT THIS IS NOT NEEDED with nextcloud version 27.

  1. This Demo DOES NOT SET the security headers as they are listed in the .htaccess file. I will update the configuration asap to refelect this. With Unit 1.31.0 we are able to set http response headers so we should do it! In any case, this looks like another legacy thing.

With 27 the response headers are alreday set. I have -NOT- looked into the code but I asume they will be set by PHP.

*   Trying 127.0.0.1:8888...
* Connected to localhost (127.0.0.1) port 8888 (#0)
> GET /apps/ HTTP/1.1
> Host: localhost:8888
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< X-Powered-By: PHP/8.2.11
***
< Referrer-Policy: no-referrer
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-Permitted-Cross-Domain-Policies: none
< X-XSS-Protection: 1; mode=block
***
< X-Request-Id: acSphoYUCFRwkGaQWRYD
< Content-Security-Policy: default-src 'none';base-uri 'none';manifest-src 'self';script-src 'self';style-src 'self' 'unsafe-inline';img-src 'self' data: blob:;font-src 'self' data:;connect-src 'self';media-src 'self';frame-src 'self';frame-ancestors 'self';form-action 'self'
< Feature-Policy: autoplay 'self';camera 'none';fullscreen 'self';geolocation 'none';microphone 'none';payment 'none'
< X-Robots-Tag: noindex, nofollow
***
< Server: Unit/1.31.0
< Date: Wed, 18 Oct 2023 08:03:22 GMT
<

The Dockerfile for my test was genrated by the update.sh script with some light modifications from here: https://github.com/nextcloud/docker/pull/1610

Thanks for that! My changes:

diff --git a/update.sh b/update.sh
index 14981ea..3a0cf25 100755
--- a/update.sh
+++ b/update.sh
@@ -11,7 +11,7 @@ declare -A cmd=(
        [apache]='apache2-foreground'
        [fpm]='php-fpm'
        [fpm-alpine]='php-fpm'
-       [unit]='unitd --no-daemon --control unix:/var/run/control.unix.sock'
+       [unit]='unitd --no-daemon'
 )

 declare -A base=(
@@ -123,8 +123,8 @@ function create_variant() {
                s/%%VARIANT%%/'"$variant"'/g;
                s/%%VERSION%%/'"$fullversion"'/g;
                s/%%BASE_DOWNLOAD_URL%%/'"$2"'/g;
-               s/%%CMD%%/'"${cmd[$variant]}"'/g;
                s|%%VARIANT_EXTRAS%%|'"${extras[$variant]}"'|g;
+               s/%%CMD%%/'"${cmd[$variant]}"'/g;
                s/%%APCU_VERSION%%/'"${pecl_versions[APCu]}"'/g;
                s/%%MEMCACHED_VERSION%%/'"${pecl_versions[memcached]}"'/g;
                s/%%REDIS_VERSION%%/'"${pecl_versions[redis]}"'/g;

Changes to the template

diff --git a/Dockerfile-unit.template b/Dockerfile-unit.template
index c64e6d1..b1b9806 100644
--- a/Dockerfile-unit.template
+++ b/Dockerfile-unit.template
@@ -1,4 +1,4 @@
-FROM nginx/unit:%%UNIT_VERSION%%-php%%PHP_VERSION%%
+FROM unit:php

 # entrypoint.sh and cron.sh dependencies
 RUN set -ex; \

The official images are not available in PHP 8.1. See https://hub.docker.com/_/unit If you want to stick with a custom PHP Version I suggest a docker multi-stage build and build the base image with unit:minimal first and add the PHP stack. I am working on a tutorial for this. Will link it in this issue.

Wit the changes above you should be able to run update.sh and a docker build in 27/unit/.

I am really soory for the long delay and hope this is working for you. What version do you use? I am more than happy to give it a try with another version.

Would it be helpful to push my updated version of the update.sh script to a git repository? Would it be helfpul to share this in this issue: https://github.com/nextcloud/docker/issues/1063

I think Unit is a strong alternative to the PHP-FPM! Happy to use this thread for further discussions.

Let me know if this work for you. Cheers Timo

tiredofit commented 11 months ago

Wow - What a response! I'm having troubles loading any page with this configuration now. I too am using 27.1.2 as my installed version. I'll do some debugging and report back shortly.

ananace commented 9 months ago

As a note, it seems that (with NC 27 at least) all that's necessary to get pretty URLs working is to set front_controller_active - started properly dogfooding nextcloud/docker#1610 seeing as the regular nextcloud images and nginx-unit are both on the same PHP version now anyway.