[Bug]: A-force -> Not being able to upload file using binary format and multi part form data

[sneha122]

Is there an existing issue for this?

• [X] I have searched the existing issues

Description

https://theappsmith.slack.com/archives/C0341RERY4R/p1717677643247429 Error Encountered By the user -

[PE-RST-5000]
Your API failed to execute
[DownstreamError]
org.springframework.web.reactive.function.client.WebClientRequestException: Unable to parse content. Expected to receive an array or object of multipart data

API fails to upload large files and throws an error. Some users have reported that it works if the file size is <20MBs.

Steps To Reproduce

https://theappsmith.slack.com/archives/C0341RERY4R/p1717677643247429

Public Sample App

No response

Environment

Production

Severity

High (Blocker to building or releasing)

Issue video log

No response

Version

Self hosted - 1.26.0

[Peterkrol12]

Do we have an update on this issue? Been a while since last activity

[sondermanish]

Hi! @Peterkrol12 In order to allow upload of larger files we would need to change a few settings to allow the web-server to send larger files. Could you elaborate on your deployment setup?

[Peterkrol12]

Hi @sondermanish - We have a few appsmith instances running using the EC2 AMI

[sondermanish]

Thanks! @Peterkrol12 Since the file size upload limit defined by default in the web-server configuration (Nginx) is less that what your current usages are, the web-server is truncating the files, hence the parsing doesn't happen.

We could try increasing the file upload limit using this variable in nginx configuration: client_max_body_size . Adding a stack overflow issue for further understanding. Let us know if this works. Or if the web-server is caddy then we would have to look for separate variables. Let us know if you require help with setting up the configuration we can get on a call if required. Thanks!

[Peterkrol12]

Hi @sondermanish - Thanks for your response!

I must say, I am not entirely following your suggestion. As provided previously, we are using the Appsmith AWS AMI on multiple instances. To access these instances, we only defined some DNS A records to point towards the public IP addresses of the instances. The only web server configuration we have, is in the AWS AMI of Appsmith itself. We looked through the instance and docker container and found a configuration file for Caddy called caddy-reconfigure.mjs. In this file, some properties including the Caddy client_max_body_size equivalent are defined. This property is set as follows:

  request_body {
    max_size ${process.env.APPSMITH_CODEC_SIZE || 150}MB
  }

So if done nothing, it should default to 150MB, but to verify, we looked at our docker.env in our EC2 instance. The APPSMITH_CODEC_SIZE property was defined in the docker.env, but it had leading # and was therefore commented out.

Am I looking at the wrong thing, or am I misunderstanding something?

[sondermanish]

Thanks for the response @Peterkrol12 The APPSMITH_CODEC_SIZE is also used to control this spring variable spring.webflux.multipart.max-in-memory-size which in essence is controlling the size of multipart request, by default it's set at 150 MB, we can try increasing that. Since this is also controlling the caddy file size, it would be easy for us to tweak this. Could you try increasing the value from 150 to a desired file size value?

[Peterkrol12]

Thanks for the continues support @sondermanish!

I changed the APPSMITH_CODEC_SIZE environment variable in the docker.env to 500 and restarted Appsmith. Afterwards I tried the file upload part of our application and received the same exception.

[sondermanish]

@Peterkrol12 Apologies it's not working for you, let me reproduce this and find an RCA for you.