Find a way to reproduce sub page listing without relying on static site generator

[renoirb]

We have about 6 000 documents in our main content pages. Regenerating static HTML version, send to a Lucene index every updated document at every merge to master would be intensive for no real benefit.

How about we remove the need for the static site generator to require a full generation run at every change. One reason most static site generator has to require a full run are cases when you want to list folders with sub pages.

Why not leverage an already baked-in feature from the web server and let it generate sub pages for us.

That way our static-site generator won’t need to have this task to be done.

Expected tasks

• [x] Have a proof of concept that fills requirements
• [ ] Create web server package with "Directory listing" that we can theme
• [ ] Create files in static site generator content source "directory listing templates" to be used by web server
• [ ] Commit NGINX configuration to make use of the templates into https://github.com/webplatform/docs along with redirect maps config of issue #6

[renoirb]

Have a proof of concept that fills requirements :+1:

[renoirb]

How to configure NGINX (very rough) notes.

Steps

1. Get source and build dependencies

   wget http://nginx.org/download/nginx-1.9.2.tar.gz
   tar xfz nginx-1.9.2.tar.gz
   apt-get -y install build-essential zlib1g-dev libpcre3 libpcre3-dev libbz2-dev libssl-dev tar unzip

2. Get modules we want to use

   git clone https://github.com/aperezdc/ngx-fancyindex.git nginx-fancyindex
   git clone https://github.com/giom/nginx_accept_language_module.git
   git clone https://github.com/simpl/ngx_auto_lib.git
   git clone https://github.com/anomalizer/ngx_aws_auth.git
   git clone https://github.com/openresty/headers-more-nginx-module.git

3. Configure and build

   cd nginx-1.9.2/
   ./configure --prefix=/opt/nginx \
          --add-module=../nginx-fancyindex \
          --add-module=../nginx_accept_language_module \
          --add-module=../ngx_auto_lib \
          --add-module=../ngx_aws_auth \
          --add-module=../headers-more-nginx-module
   make

   As root

    sudo -s
    make install

4. Create Upstart config file

   vi /etc/init/nginx.conf

   Add contents

     // file: /etc/init/nginx.conf
     description "nginx - small, powerful, scalable web/proxy server"
   
     start on filesystem and static-network-up
     stop on runlevel [016]
   
     expect fork
     respawn
   
     pre-start script
         [ -x /opt/nginx/sbin/nginx ] || { stop; exit 0; }
         /opt/nginx/sbin/nginx -q -t -g 'daemon on; master_process on;' || { stop; exit 0; }
     end script
   
     exec /opt/nginx/sbin/nginx -g 'daemon on; master_process on;'
   
     pre-stop exec /opt/nginx/sbin/nginx -s quit

5. To test FancyIndex, add to a location block

   location / {
      // ...
      fancyindex on;
      // where /foo/directorylisting is available from NGINX itself.
      fancyindex_css_href "/foo/directorylisting/fi.css";
      fancyindex_footer   "/foo/directorylisting/fi_footer.html";
      fancyindex_header   "/foo/directorylisting/fi_header.html";
   }

Reference

• NGINX Source
• Build NGINX on Debian Squeeze HOWTO
• NGINX 3rd party modules