looker-open-source / looker_deployer

A tool to help deploy objects from one Looker instance to another
Apache License 2.0
56 stars 25 forks source link

Error: Getting "Requires authentication" error while exporting content. #146

Open vaibhavyadnik opened 8 months ago

vaibhavyadnik commented 8 months ago

Hello Team, I am getting “Requires authentication” error while exporting content. I have checked looker.ini file in that all credentials and base_url’s are correct and working. Earlier it was working fine. I am getting this error since last week. Are there any changes/modifications done in the code or do I need to add anything in my export command ? Could you please help me with this it is a bit urgent. Also I have attached logs for your reference:

File "/usr/local/lib/python3.7/site-packages/looker_sdk/rtl/auth_session.py",line87,in_get_token self._login(transport_options) File "/usr/local/lib/python3.7/site-packages/looker_sdk/rtl/auth_session.py", line 153, in _login transport_options=transport_options, File "/us/local/lib/python3.7/site-packages/looker_sdk/rtl/auth_session.py",line248,in _ok raise error.SDKError(response.value) looker_sdk.error.SDKError: b'{"message": "Requires authentication.", "documentation_url":"https://cloud.google.com/looker/docs/"}'

vaibhavyadnik commented 8 months ago

Any update on this ??

nuclearducky commented 8 months ago

I'm also receiving the same error, with the latest release of ldeploy. Last time I was successfully able to use the feature was probably a couple months ago. @vaibhavyadnik were you able to find a solution?

thatguyfig commented 2 months ago

Basically this is because the access_token obtained at the start ends up expiring before the process if finished and hence gets 401 unauthorised.

A smart thing the project could do is be wary of the expiration time, and if that time is passed obtain a new token so operations can continue before proceeding.

I had this issue exporting content from a giant looker instance and instead you need to export folder-by-folder to ensure the export completes before the access_token expires. From my testing it appears to expire after 1 hour.

thatguyfig commented 2 months ago

Scratch that - it appears instead it's a problem with how gazer is handling this and does not attempt to refresh expired tokens in-process.

thatguyfig commented 2 months ago

As a workaround, you can use a bash script to export smaller subsets in a repeated fashion to avoid issues with access_token expiring - i.e.

SOURCE="dev"
declare -a FOLDERS=("1479" "1090" "276")

# iterate over the folder IDs, as gazer ends up expiring the access token for massive exports
for i in "${FOLDERS[@]}"
do
    # export all content - dashboards, folders etc
    ldeploy content export --env $SOURCE --folders $i --local-target "./${SOURCE}"

done

Fill in the list of folder IDs you want exporting and have at it. It might take a good 6 hours or so, but it will eventually get it done.

You can also take it one step further and instead query the IDs of a folder via the API (to populate the list of folder ids), using something simple like curl & jq