as a patron I want the pdf endpoint to extract all urls from Global Connectivity Report so I can check their status

[mojomonger]

IARE:

https://internetarchive.github.io/iare/?url=https://www.itu.int/dms_pub/itu-d/opb/ind/d-ind-global.01-2022-pdf-e.pdf

produces only 1 URL link.

There are hundreds in the document, as you can see by looking at the document directly:

https://www.itu.int/dms_pub/itu-d/opb/ind/d-ind-global.01-2022-pdf-e.pdf

[dpriskorn]

Would it be useful to have a debug=true parameter that dumps all the text and annotations?

[mojomonger]

if that is the best way to dump the text, then yes!

[dpriskorn]

investigation: unknown document producer cause of not finding any links:

• no link-annotations
• all the urls have spaces in them so the regex does not find them

You cannot see this when rendered, but it is a broken/non-standard pdf. an edge case.

see https://archive.org/services/context/iari/v2/statistics/pdf?url=https://www.itu.int/dms_pub/itu-d/opb/ind/d-ind-global.01-2022-pdf-e.pdf&debug=true&refresh=true

this works (no spaces): the spaces here cause the regex to not find the links:

[dpriskorn]

possible solution https://github.com/internetarchive/iari/issues/852

[dpriskorn]

in this edge case it would work to not remove the linebreaks and instead remove all spaces