search

sleede / fab-manager

Fab-Manager is the FabLab management solution. It provides a comprehensive, web-based, open-source tool to simplify your administrative tasks and your marker's projects.
http://www.fab-manager.com
Other
227 stars 113 forks source link

Adding projects to new instance results in 413 and 422 errors #436

Closed albancrommer closed 1 year ago

albancrommer commented 1 year ago

Describe the bug

Nginx / rails are behaving strangely. Can't add new projects.

I know this is supposed to be an nginx bug (413) but the mix with 422 is confusing.

Maybe getting more logs from rails would help ?

To Reproduce

Steps to reproduce the behavior:

  1. Run fab-manager with docker-compose
  2. Add a new project
  3. Add a title and a description. cf. screenshot 1
  4. It fails with error 422 "Unprocessable Entity" cf. screenshot 2
  5. Add more informations, like author, material, tags, ...
  6. Same 422 error.
  7. Add a valid SVG CAO file.
  8. It fails with error 413 "Request Entity Too Large" cf. screenshot 3. Note: you MUST open the browser console to observe.

Expected behavior

It should succeed, or at least print errors in a meaningful way.

Screenshots

Screenshot 1

Screenshot 2

Screenshot 3

Server (please complete the following information):

Browser (please complete the following information):

Additional context

This is a standard, sh script deploy / docker-compose based install

I pulled the latest version, it didn't fix it. Before that I had the 2023-01-09T14:26:00.781041181Z / 28ee9e18f2dc not on the hub anymore.

Nginx client_max_body_size

I have checked the nginx configuration for the parameter related to 413 and even injected it in the nginx.conf.

This is the conf running nginx -Tlive in the nginx docker :

# configuration file /etc/nginx/nginx.conf:

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;
    client_max_body_size 0;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

# configuration file /etc/nginx/mime.types:

types {
    text/html                                        html htm shtml;
    text/css                                         css;
    text/xml                                         xml;
    image/gif                                        gif;
    image/jpeg                                       jpeg jpg;
    application/javascript                           js;
    application/atom+xml                             atom;
    application/rss+xml                              rss;

    text/mathml                                      mml;
    text/plain                                       txt;
    text/vnd.sun.j2me.app-descriptor                 jad;
    text/vnd.wap.wml                                 wml;
    text/x-component                                 htc;

    image/avif                                       avif;
    image/png                                        png;
    image/svg+xml                                    svg svgz;
    image/tiff                                       tif tiff;
    image/vnd.wap.wbmp                               wbmp;
    image/webp                                       webp;
    image/x-icon                                     ico;
    image/x-jng                                      jng;
    image/x-ms-bmp                                   bmp;

    font/woff                                        woff;
    font/woff2                                       woff2;

    application/java-archive                         jar war ear;
    application/json                                 json;
    application/mac-binhex40                         hqx;
    application/msword                               doc;
    application/pdf                                  pdf;
    application/postscript                           ps eps ai;
    application/rtf                                  rtf;
    application/vnd.apple.mpegurl                    m3u8;
    application/vnd.google-earth.kml+xml             kml;
    application/vnd.google-earth.kmz                 kmz;
    application/vnd.ms-excel                         xls;
    application/vnd.ms-fontobject                    eot;
    application/vnd.ms-powerpoint                    ppt;
    application/vnd.oasis.opendocument.graphics      odg;
    application/vnd.oasis.opendocument.presentation  odp;
    application/vnd.oasis.opendocument.spreadsheet   ods;
    application/vnd.oasis.opendocument.text          odt;
    application/vnd.openxmlformats-officedocument.presentationml.presentation
                                                     pptx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
                                                     xlsx;
    application/vnd.openxmlformats-officedocument.wordprocessingml.document
                                                     docx;
    application/vnd.wap.wmlc                         wmlc;
    application/wasm                                 wasm;
    application/x-7z-compressed                      7z;
    application/x-cocoa                              cco;
    application/x-java-archive-diff                  jardiff;
    application/x-java-jnlp-file                     jnlp;
    application/x-makeself                           run;
    application/x-perl                               pl pm;
    application/x-pilot                              prc pdb;
    application/x-rar-compressed                     rar;
    application/x-redhat-package-manager             rpm;
    application/x-sea                                sea;
    application/x-shockwave-flash                    swf;
    application/x-stuffit                            sit;
    application/x-tcl                                tcl tk;
    application/x-x509-ca-cert                       der pem crt;
    application/x-xpinstall                          xpi;
    application/xhtml+xml                            xhtml;
    application/xspf+xml                             xspf;
    application/zip                                  zip;

    application/octet-stream                         bin exe dll;
    application/octet-stream                         deb;
    application/octet-stream                         dmg;
    application/octet-stream                         iso img;
    application/octet-stream                         msi msp msm;

    audio/midi                                       mid midi kar;
    audio/mpeg                                       mp3;
    audio/ogg                                        ogg;
    audio/x-m4a                                      m4a;
    audio/x-realaudio                                ra;

    video/3gpp                                       3gpp 3gp;
    video/mp2t                                       ts;
    video/mp4                                        mp4;
    video/mpeg                                       mpeg mpg;
    video/quicktime                                  mov;
    video/webm                                       webm;
    video/x-flv                                      flv;
    video/x-m4v                                      m4v;
    video/x-mng                                      mng;
    video/x-ms-asf                                   asx asf;
    video/x-ms-wmv                                   wmv;
    video/x-msvideo                                  avi;
}

# configuration file /etc/nginx/conf.d/fabmanager.conf:
upstream puma {
  server fabmanager:3000;
}

server {
  listen 80;
  server_name fabmanager.local;
  root /usr/src/app/public;

  location ^~ /packs/ {
    gzip_static on;
    expires max;
    add_header Cache-Control public;
  }

  try_files $uri/index.html $uri @puma;
  location @puma {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_redirect off;
    proxy_pass http://puma;
  }

  client_max_body_size 4G;
  keepalive_timeout 10;

  error_page 500 502 504 /500.html;
  error_page 503 @503;

  # Return a 503 error if the maintenance page exists.
  if (-f /usr/src/app/public/maintenance.html) {
    return 503;
  }

  location @503 {
    # Serve static assets if found.
    if (-f $request_filename) {
      break;
    }

    # Set root to the shared directory.
    root /usr/src/app/public/;
    rewrite ^(.*)$ /maintenance.html break;
  }

  location /.well-known/acme-challenge {
    root /etc/letsencrypt/webrootauth;
    default_type "text/plain";
  }

  # no spam bot
  if ($http_referer ~* (guardlink.org|free-share-buttons|social-buttons|buy-cheap-online.info|social-buttons.com|free-share-buttons.com|darodar.com|blackhatworth.com|hulfingtonpost.com|priceg.com|semalt.com|imaspammer.com|iedit.ilovevitaly.com|7makemoneyonline.com|iedit.ilovevitaly.com|7makemoneyonline.com|gamersyde.com|iloveitaly.com|econom.co|semalt.com|forum.topic44637676.darodar.com|darodar.com|iskalko.ru|ilovevitaly.ru|ilovevitaly.com|ilovevitaly.co|o-o-8-o-o.ru|o-o-6-o-o.ru|buttons-for-website.com|semalt.semalt.com|cenoval.ru|priceg.com|darodar.com|cenokos.ru|seoexperimenty.ru|gobongo.info|vodkoved.ru|adcash.com|websocial.me|cityadspix.com|luxup.ru|ykecwqlixx.ru|superiends.org|slftsdybbg.ru|edakgfvwql.ru|socialseet.ru|screentoolkit.com|econom.co|semalt.com|savetubevideo.com|shopping.ilovevitaly.com|iedit.ilovevitaly.com|forum.topic52548358.darodar.com|forum.topic53813291.darodar.com|share-buttons.com|event-tracking.com|success-seo.com|free-floating-buttons.com|get-free-social-traffic.com|chinese-amezon.com|get-free-traffic-now.com|free-social-buttons.com|videos-for-your-business.com)) { return 403; }

}
sylvainbx commented 1 year ago

I guess the problem here is that the project image was missing during your first tries, so it result in 422 unprocessable entity. In your last try you added an image, so it was ok for the requirement but it exceed the maximum allowed size, so it resulted in 413 request entity too large. Can you check the provided image size doesn't exceed the maximum set in the MAX_IMAGE_SIZE variable (see /apps/fabmanager/config/env file)?

Anyway, I agree that the resulting error message doesn't help. We already have plans to improve that in the near future, as part of the react migration.

albancrommer commented 1 year ago

Content of the config env file :

root@fabmanager:/apps/fabmanager# grep MAX config/env
MAX_IMPORT_SIZE='5242880'
MAX_IMAGE_SIZE='10485760'
MAX_CAO_SIZE='20971520'
MAX_SUPPORTING_DOCUMENT_FILE_SIZE='5242880'

New try with the project image

The image weights 600Ko, is a standard JPEG. Still got the 422 unprocessable entity.

Hints / Questions

422 : Is is possible to get more logs from rails ? Like switch to "DEBUG" the logger level ? 413 : My guts tell me this is strictly an nginx problem, because all searches on that code result with nginx and client_max_body_size.

sylvainbx commented 1 year ago

I saw that your nginx configuration file contains client_max_body_size 0; in the http {} section. Maybe it can be responsible for this? Otherwise, we set the limit to 4 Gb in the nginx fabmanager's configuration file so I hope the problem was not there... Which nginx version are you running, the default one?

You can adjust the log level by setting LOG_LEVEL=debug in config/env and restarting the app (docker-compose down && docker-compose up -d). Moreover, you may get more details in the browser's network inspector, by clicking on the request which terminates on a 422 error.

albancrommer commented 1 year ago
  1. The client_max_body_size 0 is defined in the official nginx documentation as the way to disable it

Setting size to 0 disables checking of client request body size.

  1. The log level was already on DEBUG, but I think I should see more informations. This is weird.

There was not so much more information in the 422 error requests in network inspector.

sylvainbx commented 1 year ago

I just made some tests ... I cannot reproduce the issue when connected as a member or as an admin, when saving the project as draft or publishing it, with an image attached or with no image.

Very strange that 422 errors do not show any details in the resulting JSON because that's what the code does if saving the project fails. You're probably right about nginx handling the request and causing the issue but as long as I can't reproduce it I have no idea of what kind of configuration is causing the trouble... You can try bypassing nginx by adding the following directive in the docker-compose.yml file:

services:
  fabmanager:
    ports:
      - "3000:3000"

Then restart the service and try to access your instance using myhostname.com:3000 and see if the error still occurs. Note that you may have to set ALLOW_INSECURE_HTTP=true in config/env to be able to log in.

albancrommer commented 1 year ago

Sorry, issue #438 blocking my path for now, will try next.

albancrommer commented 1 year ago

Alright, some progress.

413 Errors: Solved

I forgot I had another nginx reverse proxy at the gateway of our infrastructure. Adding the client_max_body_size directive to that nginx got rid of the problem.

422 Errors

The log/production.log has some information about the error.

I, [2023-03-01T12:58:38.022390 #7]  INFO -- : Started POST "/api/projects/" for 172.22.0.6 at 2023-03-01 12:58:38 +0000
I, [2023-03-01T12:58:38.023864 #7]  INFO -- : Processing by API::ProjectsController#create as JSON
I, [2023-03-01T12:58:38.024030 #7]  INFO -- :   Parameters: {"project"=>{"name"=>"Story box", "project_image_attributes"=>{"attachment"=>#<ActionDispatch::Http::UploadedFile:0x00007f3ffd3d01a0 @tempfile=#<Tempfile:/tmp/RackMultipart20230301-7-blocfv.png>, @original_filename="1*YI3QuyzhfVCzKQeGLcY4Mw.png", @content_type="image/png", @headers="Content-Disposition: form-data; name=\"project[project_image_attributes][attachment]\"; filename=\"1*YI3QuyzhfVCzKQeGLcY4Mw.png\"\r\nContent-Type: image/png\r\n">}, "description"=>"<p>This is a projet</p>", "status_id"=>"", "component_ids"=>[""], "machine_ids"=>[""], "user_ids"=>[""], "licence_id"=>"", "theme_ids"=>[""], "tags"=>"", "state"=>"published"}, "authenticity_token"=>"7pE75FpLaIgjgVeBPiILMF4ShaSXIq/BSTUFhm1hqJMIFtFUY5Wlnh5duUtv4G2+tu8mNVQKShWKAc7lJ2j3OQ=="}
W, [2023-03-01T12:58:38.024433 #7]  WARN -- : HTTP Origin header (https://<DOMAIN.NAME>) didn't match request.base_url (http://DOMAIN.NAME>)
I, [2023-03-01T12:58:38.024892 #7]  INFO -- : Completed 422 Unprocessable Entity in 1ms (ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
F, [2023-03-01T12:58:38.026506 #7] FATAL -- :   
F, [2023-03-01T12:58:38.026572 #7] FATAL -- : ActionController::InvalidAuthenticityToken (ActionController::InvalidAuthenticityToken):
F, [2023-03-01T12:58:38.026631 #7] FATAL -- :

The error seems to come from a HTTP/HTTPS error in the form handler "forgery protection" filter.

Other forms work on the app, like machine creation. So far only the projects category does the 422.

My proxy conf goes like this :

The docker-compose env config file has the following settings:

DEFAULT_HOST=fabmanager.crommer.net
DEFAULT_PROTOCOL=https
albancrommer commented 1 year ago

Hi @sylvainbx sorry got delayed do you have any idea about what would be the reason / a way to test or fix it ?

sylvainbx commented 1 year ago

Hi @albancrommer, I'm sorry I don't see anything I can do from FM's point of view as long as this issue is not reproductible on any of my setups. I guess this is related to your environment. Maybe your other nginx is lacking a directive to set the HTTP Origin header? For example, in our nginx config, we set proxy_set_header Host $http_host;, maybe you have to do something similar on your other nginx...

albancrommer commented 1 year ago

Hey @sylvainbx, thanks a lot for your feedback!

I managed to solve the problem by adding nginx headers, as you recommanded :)

I let you close the issue.

Solution

Adding any of these configs to the nginx vhost worked.

More on this

The Host header was already there on the first proxy, tried to add X-Forwarded-For, X-Forwarded-Proto, X-Forwarded-Server but didn't work. Tried Origin, it worked, and then the 3 issues comment fields, and they worked too.

The conf was added on the SSL/TLS termination proxy, but it should work on others.