search

jgarber623 / link-header-parser-ruby

A Ruby gem for parsing HTTP Link headers.
https://rubygems.org/gems/link-header-parser
MIT License
8 stars 1 forks source link

Data Collection Request: sample HTTP Link headers #3

Open jgarber623 opened 5 years ago

jgarber623 commented 5 years ago

If your website includes HTTP Link headers in its response headers to HEAD requests, could you run the following command (or similar depending on your OS), replacing $url with your website's address, and paste the results in a comment on this Issue?

curl --head --silent $url 2>&1 | grep -i Link: | pbcopy
jgarber623 commented 5 years ago

https://sixtwothree.org:

Link: <https://assets.sixtwothree.org/>; rel="preconnect"
Link: <https://fonts.googleapis.com/>; rel="preconnect"
Link: <https://fonts.gstatic.com/>; rel="preconnect"
Link: <https://sixtwothree.org/webmentions>; rel="webmention"
cweiske commented 5 years ago 
$ curl --head --silent cweiske.de 2>&1 | grep Link:
Link: <http://cweiske.de/stapibas/xmlrpc.php>; rel="webmention", <http://phubb.cweiske.de/hub.php>; rel="hub"
aaronpk commented 5 years ago

my website and my curl client speak HTTP2 so the header name was lowercase:

$ curl --head --silent https://aaronparecki.com 2>&1 | grep -i Link:
link: <https://switchboard.p3k.io/>; rel="hub"
link: <https://aaronparecki.com/auth>; rel="authorization_endpoint"
link: <https://aaronparecki.com/micropub>; rel="micropub"
link: <https://aperture.p3k.io/microsub/1>; rel="microsub"
link: <https://aaronparecki.com/auth/token>; rel="token_endpoint"
link: <https://aaronparecki.com/>; rel="self"
aaronpk commented 5 years ago

see also, the tests in: https://github.com/indieweb/link-rel-parser-php/blob/master/tests/BasicTest.php

cweiske commented 5 years ago

also see the parseLinks-Tests in https://github.com/pear/HTTP2/tree/master/tests

Zegnat commented 5 years ago 
$ curl --head --silent https://sink.zegnat.net/ 2>&1 | grep -i Link:
link: <https://sink.zegnat.net/micropub>; rel="micropub"
link: <https://sink.zegnat.net/micropub>; rel="authorization_endpoint"
link: <https://sink.zegnat.net/micropub>; rel="token_endpoint"

I used to have this all as a single link header, but had to split it up because consumer support was limited… In the early days the command would have returned:

link: <https://sink.zegnat.net/micropub>; rel="micropub authorization_endpoint token_endpoint"
jordangov commented 5 years ago

Why only for HEAD requests? What if an API doesn't do very well and only implements the Link header on other methods?

Anyway... the CanvasLMS API that I'm dealing with right now uses them for paging:

Link: <https://[DOMAIN]/api/v1/courses/1/assignments?page=1&per_page=10>; rel="current",<https://[DOMAIN]/api/v1/courses/1/assignments?page=2&per_page=10>; rel="next",<https://[DOMAIN]/api/v1/courses/1/assignments?page=1&per_page=10>; rel="first",<https://[DOMAIN]/api/v1/courses/1/assignments?page=6&per_page=10>; rel="last"
jgarber623 commented 5 years ago

Why only for HEAD requests? What if an API doesn't do very well and only implements the Link header on other methods?

@jordangov This gem is specifically for parsing HTTP Link response headers (and not for parsing <link> elements in the response body). 😀

jgarber623 commented 5 years ago

@aaronpk @cweiske Thanks for the links to those test suites!

jgarber623 commented 5 years ago

@jordangov Oh, now I see the rather fine point you've made. Updated the Issue's description!

snarfed commented 5 years ago

try https://indiemap.org/ ! gives you 2300 sites all in one place, easily query-able, and self serve. 😁

eg here's an example graph from https://indiemap.org/docs.html#stats :

instructions on how to write your own queries here: https://indiemap.org/docs.html#data-mining