business-science / tidyquant

Bringing financial analysis to the tidyverse
https://business-science.github.io/tidyquant/
Other
850 stars 176 forks source link

tq_exchange("NASDAQ") not working #226

Closed mdancho84 closed 2 weeks ago

mdancho84 commented 1 year ago

I'm getting the following error. It looks like 403 meaning access is being prevented. Any alternatives for getting exchange data for NASDAQ, AMEX, and NYSE?

tq_exchange("NASDAQ")
Getting data...

Error in open.connection(con, "rb") : 
  cannot open the connection to 'https://api.nasdaq.com/api/screener/stocks?tableonly=true&exchange=nasdaq&download=true'
In addition: Warning message:
In open.connection(con, "rb") :
  cannot open URL 'https://api.nasdaq.com/api/screener/stocks?tableonly=true&exchange=nasdaq&download=true': HTTP status was '403 Forbidden'
jamesfulford commented 1 year ago

Seems like anti-scraper filter? I can reach URL from browser but not from curl.

gabrielodom commented 1 year ago

Following for updates (and I will see if I can get anything to work).

munishkgitbub commented 1 year ago

Has this issue been sorted out? Its not giving for AMEX and NYSE exchange data also.

marinov-alexander commented 1 year ago

Has this issue been resolved or looked at?

arunamalla commented 1 year ago

yes, looks like an anti-spam filter is there.

api.nasdaq.com - sever is allowing only few values as 'User Agent' in the http request header. below are the allowed values as user agent { "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" }

modify package in local machine to make HTTP call with above mentioned user-agent

mdancho84 commented 1 year ago

I will test this out. Thanks a lot. 😀

ptoche commented 10 months ago

I'm getting issues too, that I wasn't getting back in Jan 2022:

nyse <- tq_exchange("NYSE")
Getting data...

Error in open.connection(con, "rb") : 
  cannot open the connection to 'https://api.nasdaq.com/api/screener/stocks?tableonly=true&exchange=nyse&download=true'
In addition: Warning message:
In open.connection(con, "rb") :
  URL 'https://api.nasdaq.com/api/screener/stocks?tableonly=true&exchange=nyse&download=true': status was 'Failure when receiving data from the peer'
mdancho84 commented 2 weeks ago

This is working now. Make sure to use tidyquant >= 1.0.7.9001

Install using:

remotes::install_github("business-science/tidyquant")
> tq_exchange("NASDAQ")
Getting data...

# A tibble: 3,970 × 7
   symbol company                                          last.sale.price    market.cap country       ipo.year industry                         
   <chr>  <chr>                                                      <dbl>         <dbl> <chr>            <int> <chr>                            
 1 AACG   ATA Creativity Global American Depositary Shares           0.6        19200078 China             2008 Other Consumer Services          
 2 AADI   Aadi Bioscience Inc. Common Stock                          1.47       36183806 United States       NA Biotechnology: Pharmaceutical Pr…
 3 AAGR   African Agriculture Holdings Inc. Common Stock             0.123       7123407 United States     2021 Farming/Seeds/Milling            
 4 AAGRW  African Agriculture Holdings Inc. Warrant                  0.009        520801 United States     2021 Farming/Seeds/Milling            
 5 AAL    American Airlines Group Inc. Common Stock                 10.0      6599874326 United States       NA Air Freight/Delivery Services    
 6 AAME   Atlantic American Corporation Common Stock                 1.55       31619625 United States       NA Life Insurance                   
 7 AAOI   Applied Optoelectronics Inc. Common Stock                  8.28      338616520 United States     2013 Semiconductors                   
 8 AAON   AAON Inc. Common Stock                                    88.7      7185056096 United States       NA Industrial Machinery/Components  
 9 AAPL   Apple Inc. Common Stock                                  226.    3436895168850 United States     1980 Computer Manufacturing           
10 ABAT   American Battery Technology Company Common Stock           1.03       60356077 United States       NA Metal Mining                     
# ℹ 3,960 more rows
# ℹ Use `print(n = ...)` to see more rows