yoheinakajima / instagraph

Converts text input or URL into knowledge graph and displays
MIT License
3.27k stars 281 forks source link

[Question] Scrape text from website function never used? #54

Closed Ashes47 closed 11 months ago

Ashes47 commented 11 months ago

Following function is never used:

def scrape_text_from_url(url):
    response = requests.get(url)
    if response.status_code != 200:
        return "Error: Could not retrieve content from URL."
    soup = BeautifulSoup(response.text, "html.parser")
    paragraphs = soup.find_all("p")
    text = " ".join([p.get_text() for p in paragraphs])
    print("web scrape done")
    return text`