Open cristinamullin opened 1 year ago
dataRetrieval uses GET unless the URL gets too long, then it switches to a POST. However, just recently it seems like the limits for how big a POST can be has been restricted compared to what it use to be. I'm still looking into if it is in fact a new restriction or if I just hadn't noticed it until recently. Unfortunately at the moment the POSTs don't seem to give the flexibility over the GET that I was assuming we'd see.
Describe the bug
Users are interested in querying the WQP by a long list of parameters (sites or characteristics), but their queries will not work if they input too many parameters. There appears to be a max URL length for web service calls. Ideally, we will be able to add some logic to the TADA data retrieval functions to handle very long parameter lists (chunk queries) automatically.
Currently, the max within any given filter appears to be 30-45, which would align/make sense with the 2,083 character max for Chrome.
I think dataRetrieval uses Curl for the queries. Regardless, it's probably a good idea to restrict URL lengths. Not sure if there is an option to specify to use POST for the queries using dataRetrieval? https://www.waterqualitydata.us/webservices_documentation/#queries-using-post
Reminders for TADA contributors addressing this issue
New features should include all of the following work:
[ ] Create the function/code.
[ ] Document all code using comments to describe what is does.
[ ] Create tests in tests folder.
[ ] Create help file using roxygen2 above code.
[ ] Create working examples in help file (via roxygen2).
[ ] Add to appropriate vignette (or create new one).