search

mnwato / tradingview-scraper

This is a tool to scrape everything on https://tradingview.com <Thank to contributers>
MIT License
65 stars 28 forks source link
scraper tradingview

readme

TradingView Scraper

Python 3.8 Code style: black MIT License

This is a Python library for scraping ideas and indicators from TradingView.com. The goal is to develop this package to scrape anything on TradingView.com with real-time responses.
Thanks to contributors!

To-Do List

Completed Tasks

Pending Tasks

To be aware of the latest changes, go to the end of this page.

Features

Here’s a revised version of the Installation section that enhances clarity and encourages user engagement:

Installation

To get started with the TradingView Scraper library, follow these simple steps:

  1. Open your terminal: Launch your preferred command line interface.

  2. Install the package: Run the following command to install the TradingView Scraper:

    pip install tradingview-scraper

  3. Upgrade if necessary: If you already have the library installed and want to upgrade to the latest version, use:

    pip install --upgrade tradingview-scraper

Here’s a revised version of the Examples section, focusing on clarity, ease of understanding, and providing essential information about default values:

Examples

1. Fast Running (Default Parameters)

To quickly scrape ideas using default settings, use the following code:

from symbols.ideas import Ideas

# Initialize the Ideas scraper with default parameters
ideas_scraper = Ideas()  # Default: export_result=False, export_type='json'
ideas = ideas_scraper.scrape()  # Default symbol: 'BTCUSD'
print("Ideas:", ideas)

Default Parameters:

2. Getting Ideas for a Specific Symbol, Export Type, and Pages

To scrape ideas for a specific symbol and export them as a CSV file, you can specify the parameters:

from symbols.ideas import Ideas

# Initialize the Ideas scraper with custom parameters
ideas_scraper = Ideas(
  export_result=True,  # Set to True to save the results
  export_type='csv'    # Specify the export type (json or csv)
)

# Scrape ideas for the ETHUSD symbol, from page 1 to page 2
ideas = ideas_scraper.scrape(
  symbol="ETHUSD",
  startPage=1,
  endPage=2,
  sort="popular"  #  Could be 'popupar' or 'recent'
)
print("Ideas:", ideas)

Output Format: The output will always be a list of Python dictionaries, structured as follows:

[
  {
      "title": "Bitcoin -65% crash, ETH -83%, DOGE -89%, SHIBA -90%",
      "paragraph": "Name your altcoin in the comment section, and I will do a technical analysis for you!\n\nThe crypto market looks ...",
      "preview_image": "https://s3.tradingview.com/6/6VQphWH6_mid.png",
      "author": "Xanrox",
      "comments_count": "295",
      "boosts_count": "678",
      "publication_datetime": "2024-08-18T05:55:19.000Z",
      "is_updated": "True",
      "idea_strategy": "Short"
  }
]

3. Getting Indicators Status

To scrape the status of specific indicators, use the following code:

from symbols.indicators import Indicators

# Initialize the Indicators scraper with export options
indicators_scraper = Indicators(export_result=True, export_type='json')

# Scrape indicators for the BTCUSD symbol from the BINANCE exchange
indicators = indicators_scraper.scrape(
    exchange="BINANCE",
    symbol="BTCUSD",
    indicators=["RSI", "Stoch.K"]
)
print("Indicators:", indicators)

Output:

{"RSI": "46.34926112", "Stoch.K": "40.40173723"}

4. Getting All Indicators

If you want to retrieve all available indicators for a symbol, set allIndicators=True:

# Scrape all indicators for the BTCUSD symbol
indicators_scraper = Indicators(export_result=True, export_type='json')
indicators = indicators_scraper.scrape(
    symbol="BTCUSD",
    allIndicators=True
)
print("All Indicators:", indicators)

5. Getting News Headlines/Content

# Create an instance of the NewsScraper with export options
news_scraper = NewsScraper(export_result=True, export_type='json')

# Retrieve news headlines from a specific provider
news_headlines = news_scraper.scrape_headlines(
    provider='newsbtc',  # Specify the news provider
    # symbol='BTCUSD',      # Uncomment and specify if needed
    # exchange='BINANCE', # Uncomment and specify if needed
    sort='latest'
)

# Retrieve detailed news content for a specific story
news_content = news_scraper.scrape_news_content(
    story_path=news_headlines[0]['storyPath']  # Specify the story path from scraped headlines
)

Output (news headline):

[
  {
    "breadcrumbs": "News > U.Today > Bitcoin ETFs Record Enormous Outflows",
    "title": "Bitcoin ETFs Record Enormous Outflows",
    "published_datetime": "Wed, 04 Sep 2024 07:55:38 GMT",
    "related_symbols": [
      {
        "name": "BTCUSDT",
        "logo": "https://s3-symbol-logo.tradingview.com/crypto/XTVCUSDT.svg"
      }
    ],
    "body": ["""<List of text page content>"""],
    "tags": ["Crypto", "U.Today"]}
]

Output (news content):

[
  {
    "id": "tag:reuters.com,2024:newsml_L1N3KM09S:0",
    "title": "Goldman Sachs sees biggest boost to US economy from Harris win",
    "provider": "reuters",
    "sourceLogoId": "reuters",
    "published": 1725443676,
    "source": "Reuters",
    "urgency": 2,
    "permission": "preview",
    "relatedSymbols": [
      {
        "symbol": "BITMEX:XBTETH.P",
        "currency-logoid": "country/US",
        "base-currency-logoid": "crypto/XTVCBTC"
      },
      {
        "symbol": "ICEUS:DXY",
        "logoid": "indices/u-s-dollar-index"
      }
    ],
    "storyPath": "/news/reuters.com,2024:newsml_L1N3KM09S:0-goldman-sachs-sees-biggest-boost-to-us-economy-from-harris-win/"
  }
]

Changes:

License:

[MIT]