prepare for 16,000 character limit on the URL length of requests made to list table records

[walkingpendulum]

Recently I received such email from Airtable team:

In the coming months, we’ll be implementing a 16,000 character limit on the URL length of requests made to list table records using the public REST API. Once the limit is enforced, requests to this endpoint that exceed 16k characters will begin to fail.

Who is impacted?

According to our logs, in the past year there has been at least one instance of a request that met this criteria for a Base associated with your Workspace. In order to prevent potential interruption any workflows, you may need to migrate to one of the following alternatives.

What do I need to do?

If your workflow involves making these requests using Airtable.js, we have already made the change to address this with the latest version. In order to take advantage of this, you'll just need to update the version you are using to 1.15.0.

Using POST instead of GET

We’ve created a new POST version of the existing list table records endpoint. This allows you to pass the same options as JSON in the request body that you normally would pass using query parameters. This will greatly reduce the overall URL length since these parameters will no longer need to be included in the URL itself. For example, the following request:

curl -X GET "https://api.airtable.com/v0/{baseId}/{tableIdOrName}?filterByFormula=SEARCH%28%22value%22%2C%7BField%20Name%7D%29&maxRecords=20" \
-H "Authorization: Bearer YOUR_API_KEY"

Would become:

curl -X POST "https://api.airtable.com/v0/{baseId}/{tableIdOrName}/listRecords" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
--data '{"filterByFormula": "SEARCH(\"value\",{Field Name})", "maxRecords": 20}'

Again, if you use the Airtable.js library to access your base data, you’ll just need to update to v0.11.5 which includes changes to automatically use the POST endpoint for these requests when the URL length would otherwise exceed this limit.

We use PyAirtable to work with Airtable API. Do you plan to replace GET requests for listRecords with POST requests?

[gtalarico]

Thanks for sharing. Yes we should def. do this

[FredZhao-at]

Hi everyone! I'm Fred, an engineer from the Airtable API team (and a teammate of @marks!). It looks like #210 is implementing this, so I wanted to chime in to lend support. :)

While we originally planned to start this enforcement in January 2023, last week we needed to fully deploy this change ahead of schedule, as part of some required infrastructure improvements to maintain the stability of our public API.

This accelerated our originally communicated timeline by a month, so I want to apologize for the rush. Please be assured that this decision was not made in haste, and only from required infrastructural needs.

While most users will never hit the 16,000 character limit, we know that query parameters like filterByFormula can cause some cases to exceed that limit. So using POST for all List Records requests should definitely be the right fix here going forward.

[NicoHood]

@FredZhao-at Do you know when the filterByFormula parameter will support field ids, not only names? This will safe us tons of work and bandwidth.

[walkingpendulum]

hello, guys! just wanted to gently poke this ticket. hope this PR will be merged in time for us consumers to accomodate Airtable API changes

[gtalarico]

@FredZhao-at @walkingpendulum @marks do you recommend only making POST if len is over the limit, Or would it be ok to always make a POST?

[walkingpendulum]

@gtalarico I have no opinion here. my only concern is pyairtable keeps working.

[marks]

Hi @gtalarico - thanks for keeping this great package up to date. No official guidance from Airtable. If you want to always use POST, that's perfectly valid.

(Airtable.js does still do a GET unless the URL is over 15000 characters (or the developer specifies POST specifically.)

[gtalarico]

Great will move forward this then and we can adjust if needed.

[walkingpendulum]

hello, guys! any news here?