REST API returns a 500 error when URL query is correctly encoded

n8green commented 1 year ago

Bug report

Required System information

Node.js version: 18.2.0
NPM version: 8.9.0
Strapi version: 4.10.5
Operating system: Windows 10

Describe the bug

The Strapi API cannot accept URL encoded square brackets in query parameters. As they are reserved characters (per https://datatracker.ietf.org/doc/html/rfc3986#section-2.3), I suspect this is a bug.

When encoded square brackets are included in a request to the Strapi API, a 500 error is returned.

Steps to reproduce the behavior

I set up a new instance of Strapi, and a new collection named pages. I'd like to access this data from my PHP application, querying the API for the relevant data for a page using the filtering features.

Per the strapi documentation, the way to filter a collection is to use a square bracket syntax in the request query, eg

http://hostname:1337/api/pages?filters[slug][$eq]=home

When I manually make the above request in my browser, it returns the response I would expect, bringing back the page which matches my filter query.

When I use PHP to make the request programmatically however, the URL is (correctly) encoded as follows

http://hostname:1337/api/pages?filters=%5Bslug%5D%5B%24eq%5D%3Dhome

This causes the Strapi API to return a 500 status and InternalServerError.

Logs report the following

Error: The filters parameter must be an object or an array
    at convertFiltersQueryParams (C:\xxx\pi-site-strapi\node_modules\@strapi\utils\lib\convert-query-params.js:394:11)
    at transformParamsToQuery (C:\xxx\pi-site-strapi\node_modules\@strapi\utils\lib\convert-query-params.js:514:19)
    at Object.findMany (C:\xxx\pi-site-strapi\node_modules\@strapi\strapi\lib\services\entity-service\index.js:83:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.<anonymous> (C:\xxx\pi-site-strapi\node_modules\@strapi\strapi\lib\services\entity-service\index.js:327:20)
    at async Object.find (C:\xxx\pi-site-strapi\node_modules\@strapi\strapi\lib\core-api\service\collection-type.js:34:23)
    at async Object.find (C:\xxx\pi-site-strapi\node_modules\@strapi\strapi\lib\core-api\controller\collection-type.js:23:39)
    at async returnBodyMiddleware (C:\xxx\pi-site-strapi\node_modules\@strapi\strapi\lib\services\server\compose-endpoint.js:52:18)
    at async policiesMiddleware (C:\xxx\pi-site-strapi\node_modules\@strapi\strapi\lib\services\server\policy.js:24:5)
    at async serve (C:\xxx\pi-site-strapi\node_modules\koa-static\index.js:59:5)

Both PHPs Guzzle library and http_build_query function encode square brackets by default, which is correct behaviour per http spec.

Expected behavior

The Strapi API should be able to use URL encoded square brackets without triggering a 500.

rp3r commented 1 year ago

We have the same issue with Strapi v4.9.2

@n8green have you found any workaround for this issue? We currently can't change our default encoding behaviour.

alexandrebodin commented 1 year ago

Hi looking at your example the "conventional" encoding you are using is not compatible with the one we use from here https://www.npmjs.com/package/qs. I don't know much about the PHP ecosystem and if you could fine a compatible library sadly.

fekoch commented 1 year ago

@n8green I had the same error message, but using node and the recommended library. At first I also thought the encoding is the issue, because qs also (correctly) encodes the square brackets when calling .stringify().

But after testing via the option encodeValuesOnly: true I determined that Strapi can parse both encoded and unencoded square brackets.

My problem was that I tried to stringify my filter-object and pass the string as the Query-Parameter to filter. (Like .../pages?filters=<string filter>)

To use it correctly, I had to include the filter-key in the json, like so:

const queryParameters = {
   filters: {
      slug: {
         $eq: "home"
      }
   }
}

const query = qs.stringify(queryParameters); // 'filters%5Bslug%5D%5B%24eq%5D=home'

// or: qs.stringify(queryParameters, {encodeValuesOnly: true}) // 'filters[slug][$eq]=home'

const response = await fetch(`http://localhost:1337/api/pages?${query}`)

danielbeardsley commented 1 week ago

We (at iFixit) ran into this same issue and it took us a while to figure out.

This is definitely a bug, and it's not the fault of the qs module which correctly handles decoding square brackets in the url.

All the docs I've seen about query string encoding say you should be able to safely encode any url chars and that you must encode square brackets: https://docs.mapp.com/docs/url-encoding-and-what-characters-are-valid-in-a-uri

strapi / strapi