search

jina-ai / reader

Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/
https://jina.ai/reader
Apache License 2.0
7.02k stars 554 forks source link

Twitter/X parse error #145

Closed mazzzystar closed 1 month ago

mazzzystar commented 1 month ago 
https://r.jina.ai/https://twitter.com/USTechWorkers/status/1843744799607898260

I got:

Title: X

URL Source: https://twitter.com/USTechWorkers/status/1843744799607898260

Markdown Content:
Don’t miss what’s happening

People on X are the first to know.
nomagick commented 1 month ago

This should be X changing its implementation a little bit and providing some preview content. By default, Reader eagerly tries to return ASAP, this has resulted in Reader returning the preview content instead of the full content, which is made available after.

To walk around this, you may pass X-Timeout: 30 header to explicitly set a deadline. This changes the return policy and makes Reader wait more patiently until deadline or networkidle0

mazzzystar commented 1 month ago

Thx : )