search

adbar / trafilatura

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
https://trafilatura.readthedocs.io
Apache License 2.0
3.66k stars 262 forks source link

Extracting full text from an URL returns None #739

Open vrnch opened 2 weeks ago

vrnch commented 2 weeks ago 
import trafilatura 

url = 'https://venturebeat.com/ai/moondream-raises-4-5m-to-prove-that-smaller-ai-models-can-still-pack-a-punch/'
download = trafilatura.fetch_url(url=url)
article = trafilatura.extract(download)

print(article)
None
vrnch commented 2 weeks ago

https://www.hoteliermagazine.com/robots-are-the-future-of-hospitality-in-canada/ same

adbar commented 2 weeks ago

These pages appear to be protected against straight downloads. Please check the troubleshooting section of the docs for more ideas.