check-links fails when hitting binder.ploomber.io

[edublancas]

we need to investigate, unsure if this is ap problem with binder.ploomber.io or with our implementation

see: https://github.com/ploomber/jupysql/pull/252

note that this might be an intermittent problem

[mehtamohit013]

AC Criteria:

• Investigate why binder is failing and fix it

[edublancas]

adding some info that might be useful:

I realized that one of my PRs was failing because twitter was returning 403 for valid URLs (they don't throw any errors when opened from the browser). Looks like some domains don't like programmatic access so I tried modifying the headers as suggested here, but it didn't work.

So I ended up creating a list of domains for which a 403 code shouldn't be considered an error: https://github.com/ploomber/pkgmt/commit/40ef6533903671ecef925704c802c965836e89b4

With binder is different since have control over the server configuration, but I don't know how easy would be to patch this. Maybe setting the headers will work.

[mehtamohit013]

@edublancas @idomic

For me, it is working fine locally. Code used:

# Create a GET request to the server
import requests

url = 'https://binder.ploomber.io/'
result = requests.get(url)
print(result.content.decode())

[edublancas]

to debug https://github.com/ploomber/jupysql/pull/510 let's add a --verbose option to pkgmt check-links. adding verbose should display the raw response returned by the requests that failed (hence, it'll display the raw response from binder. then, we can update https://github.com/ploomber/jupysql/pull/510 to install pkgmt from with with the added option, re-run the CI and see what response we're getting.

once we see that, we can decide next steps

[edublancas]

so this is an intermittent problem, see: https://github.com/ploomber/jupysql/pull/510#issuecomment-1563196902

let's close this as it's not a big problem and it's unclear how to fix it. thanks @mehtamohit013 for investigating this!