USACE / instrumentation

Instrumentation project issue tracking and project planning

MIT License

4 stars 1 forks source link

[BUG REPORT]: Uploader #169

Closed ScottMassa closed 1 month ago

ScottMassa commented 2 months ago

Describe the bug

The uploader fails with "400 bad request" when data file has several thousand records

Steps To Reproduce

Go to uploader [Uploading RPool.csv…]()
Click on choose file, select RPool.csv
Import as TS measurement
See error

Expected behavior

Returns "201 Created" and uploads successfully.

Actual behavior

Returns 400 bad request but will upload when you reduce the file from 7500 rows to 1000 rows.

Screenshots

dennisgsmith commented 2 months ago

@ScottMassa, thanks for reporting this.

Is this a new issue (i.e. were you previously able to upload documents over 1000 rows using this tool)?
When was the first time you encountered the issue?

ScottMassa commented 2 months ago

We could previously upload larger data sets. I can’t remember exactly but probably 50k to 100k records . One of our users in Pittsburg asked me about the uploader being broken this morning. I gave it a try and got the 400 error, then tried it with 1000 records and it worked. I also checked to make sure there were no bad characters in the values.

dennisgsmith commented 2 months ago

Thanks Scott. I'm looking into this now. If you could provide the response body sent back with the failed request (you can get this from the "Response" tab in the network section of the Devtools) it would be helpful for us to diagnose and resolve this issue. If you could also send me an email with the offending file attached, that would also be very helpful.

ScottMassa commented 2 months ago

Info is below and attached.,

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

400 Bad Request

Bad Request

Your browser sent a request that this server could not understand.

Request URL: https://midas.sec.usace.army.mil/api/projects/5b6f4f37-7755-4cf9-bd02-94f1e9bc5984/timeseries_measurements?dry_run=true Request Method: POST Status Code: 400 Bad Request Remote Address:

Referrer Policy: strict-origin-when-cross-origin

dennisgsmith commented 2 months ago

@ScottMassa We believe this issue lies in the configuration of the reverse proxy. I have submitted a ticket to CWBI for further investigation: CWBI-21280. Please see this ticket for further information.

dennisgsmith commented 1 month ago

This this issue has been resolved in TEST and PROD by updating the WAF to allow larger payloads, see the liked ticket above for more details.