Implement Feature to Fetch and Store News Data in SQLite Database

[espin086]

Summary

Here is the RapidAPI we will use to get news data: https://rapidapi.com/rphrp1985/api/newsnow/

We need to develop a feature that automatically fetches news articles on ANY Topic Speficied with a specific focus on interpretability in machine learning, and stores them in our SQLite database. This feature should capture comprehensive details about the articles, including titles, images, video links, publication dates, short descriptions, full text, and source URLs.

Requirements:

• Data to Fetch and Store: For each news article related to AI safety and interpretability in machine learning, the following information should be captured:

  • Title: The title of the article.

  • Top Image: URL to the main image of the article.

  • Videos: Any related video content URLs.

  • URL: The direct link to the news article.

  • Date: The publication date of the article.

  • Short Description: A brief summary of the article.

  • Text: The full text of the article.

  • Source: The name of the publication or source website.

• Database Schema: The SQLite database should have a table named ai_safety_news specifically designed to store the fetched news data. The schema should include columns for each of the data points listed above.
• Update Frequency: The feature should be capable of fetching new articles hourly.
• Error Handling: Implement robust error handling to manage issues such as API rate limits, data parsing errors, and database insertion failures.
• Documentation: Provide detailed documentation on how to configure any necessary API keys or credentials and instructions on how the news fetching feature operates.

Acceptance Criteria

• The system successfully fetches and stores news articles based on user input and interpretability in machine learning in the news table in the SQLite database.
• The feature updates the database with new articles as they are published.
• Radial Button for News Analysis: One can search for news, the news are displayed as a table
• Error handling is in place to ensure the system's stability and reliability.
• Comprehensive documentation is provided for configuring and using the feature.

[ZaibyS]

@espin086 How much historical data we should fetch from the news API, same duration as we have for stock data?