Authentication problem

[kbsali]

Hi,

I'm trying to export tables from a db of mine, but I am always getting 406 errors, although the very same call with the same api key through curl works just fine. Any idea what could cause this? (I'm running Linux Manjaro if this can help troubleshooting)

$ airtable-export --version
airtable-export, version 0.6

$ airtable-export export --key=keyXXX export appXXX prizes -v
Error: 406 Client Error: Not Acceptable for url: https://api.airtable.com/v0/export/appXXX
For more information check: https://httpstatuses.com/406

$ curl https://api.airtable.com/v0/appXXXprizes\?api_key\=keyXXX
{"records":[{"id":"rec5EuisiankLtNJg","fields":...

[alexclst]

I started getting the same error on every try I make with this tool a week ago. Exactly the same result. curl is just happy, but airtable-export returns the 406 errors all the time. It is almost like the Airtable API changed, and is now not letting this tool do its thing. Except that since curl is happy, that doesn't seem the issue. Tried version 0.6 and the same issue exists.

[rivol]

Airtable seems to be checking User-Agent header for some reason, specifically whether it contains httpx :-(

passing -H "User-Agent: httpx/1.2.3" or similar to curl also results in 406. Similarily, setting user-agent to something else in my Python code results in proper response.

[alexclst]

Thanks, @rivol, your comment led me to find a workaround:

Version 0.6 of this tool added a --user-agent flag. If you include that to add a custom user agent the tool works just fine. As long as you aren't using httpx in the chosen user agent you should be just fine. Add something like --user-agent="custom/0.6" to anywhere you use this tool. This may essentially repeat what you said above, but I wasn't entirely sure this is what you were referring to, so wanted to leave an explicit example of the syntax to use.

[kbsali]

@alexclst I have tried adding a custom user agent, but then I'm getting a 404! :/

$ airtable-export export --key=keyXXX export appXXX prizes --user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"
Error: 404 Client Error: Not Found for url: https://api.airtable.com/v0/export/appXXX
For more information check: https://httpstatuses.com/404

$ airtable-export export --key=keyXXX export appXXX prizes 
Error: 406 Client Error: Not Acceptable for url: https://api.airtable.com/v0/export/appXXX
For more information check: https://httpstatuses.com/406

What am I doing wrong this time? Could you show me of you got it to work? Thanks

[alexclst]

@kbsali, the example I gave is essentially what I used ("custom" was a different word for me, but the same simple formulation). I tried a single word and got a 404 too, which is why I added the "/0.6". I guess that the more complex user agent you tried is also a problem... Try something along the lines of my example.

[kbsali]

custom/0.6 didn't work either 😞

$ airtable-export export --user-agent="custom/0.6" --key=keyXXX export appXXX prizes
Error: 404 Client Error: Not Found for url: https://api.airtable.com/v0/export/appXXX
For more information check: https://httpstatuses.com/404

although

$ curl https://api.airtable.com/v0/appXXX/prizes\?api_key\=keyXXX -H "User-Agent: custom/0.6" --head 
HTTP/1.1 200 OK
access-control-allow-headers: authorization,content-length,content-type,user-agent,x-airtable-application-id,x-airtable-user-agent,x-api-version,x-requested-with
access-control-allow-methods: DELETE,GET,OPTIONS,PATCH,POST,PUT
access-control-allow-origin: *
airtable-uncompressed-content-length: 14075
...

worked fine.

-H "User-Agent: httpx/1.2.3" as @rivol mentioned also fails with 406 :

$ curl https://api.airtable.com/v0/appXXX/prizes\?api_key\=keyXXX -H "User-Agent: httpx/1.2.3" --head
HTTP/1.1 406 Not Acceptable
Content-Length: 1265
Content-Type: text/html
Date: Thu, 08 Apr 2021 15:20:39 GMT
ETag: "606d0582-4f1"
Server: Tengine
Connection: keep-alive

@alexclst would you mind sharing the exact command you are running? :pray: thanks!

[alexclst]

@kbsali, it is a bit more complicated than a single line, since I have it running as a bash script every night via the macOS launchd process. Here is the basics of what I've done, and worked as intended last night:

backups=/Users/server/Backups/Airtable

function my_sync {
    airtable-export export $my_id $my_tables --key=$my_key --sqlite "$backups/$my_base.db" --user-agent="tenseg/0.6"
}

my_base=Hosting_Status
my_id=app...
my_tables="Dora"
my_key=$(/usr/bin/security find-generic-password -s "alex-at" -w)
my_sync

The first line is defined at the start of the script, I then have a function that among other things runs the export, and at the bottom a number of code blocks that back up a number of individual bases.

I can't imagine that tenseg (the name of the consulting company I run that does web development) would act any different from custom, but maybe it is?

[alexclst]

@kbsali, another thought: Have you ensured that airtable-export is up to date? Only the latest version, 0.6, even has the --user-agent flag.

[kbsali]

Thanks @alexclst ! Thanks for your script, I checked more carefully my call and realised I had put... the export keyword twice ! I do not have an export table, damned! :facepalm: So it works now, thanks for taking the time to help out! :+1: