wahidsaleemi / finance

Stock scripts
MIT License
3 stars 0 forks source link

Discussion of API offering Earnings Data #3

Open SCCMAdmin opened 5 years ago

SCCMAdmin commented 5 years ago

@wahidsaleemi in Issue #1 you mentioned... "It would be better to find a web API so we don't need to parse HTML. There are several non-free ones but I haven't found one that's free: https://stackoverflow.com/questions/4945626/free-api-or-similar-to-get-next-earnings-date-for-stocks"

Regarding a web API or TWS API:

  1. How do you know https://stocksearning.com has an API we are accessing? I couldn't find it advertised. For me to learn that's all. Also I found these are estimated earnings dates in the example of DAL it is 10 Jan.

  2. How do we know if https://www.earningswhispers.com has an API we can access? Hard to find anything advertised openly. Sample symbol DAL which was removed from output CSV has BMO 17 Jan earnings on this site. Different to the estimate at https://stocksearning.com. I think using https://www.earningswhispers.com in future if possible, could improve the data we are relying on.

  3. I know TWS has an API. Do you know how to get earnings to appear in TWS (even at a subscription cost)? (you made a note about Stan in the past perhaps we should investigate. Maybe it's called Wall Street Horizon. I'll ask him on FB if you don't know)

I use a piece of software called XLQ (Excel add-in) which makes use of various free and paid data sources by moving downloaded data into Excel. See https://qmatix.com/ and scroll down on main page for data sources list. I know Tiingo was added as a free source after you register and grab a token. However after a look around the API they have is $10/month, but most importantly, I can't see they offer earnings data. I could ask them if need be. So yeah, need to find some earnings data sources as a starting point.

You said you know several at cost earnings data sources. What are they if you care to share?

wahidsaleemi commented 5 years ago
  1. The https://stocksearning.com does not have an API. We are just using Invoke-WebRequest and the Internet Explorer rendering engine to browse the site via PowerShell. This is why this script requires Windows (Internet Explorer) and why there could be errors. An API would provide very predictable return information. Instead of "browsing" a web page and getting back HTML, you would get back json-formatted or xml-formatted fields. You would know exactly which field has the stock symbol, earnings date, and other information. For example, try this: Invoke-RestMethod -Uri https://api.earningscalendar.net/?date=20181205

  2. For www.earningswhispers.com, it doesn't look like there is an API we can access so it would still be parsing HTML. I'm not sure if it would be any easier. You could add a -Source parameter so users of the script can choose between the two sites. If you look at the page source, you'll see some javascript that is being used to retrieve the earnings date, you would need to replicate that functionality in PowerShell.

  3. I didn't look into TWS API for earnings data because of the cost and don't know if it would even be possible to access that information from TWS' API.

  4. For qmatrix or any of the paid data sources, I can help you write a script against it if you've subscribed. Most API's actually do require registration and getting a token, so that's very common. A couple of the paid ones: -- http://www.zacksdata.com/data/calendar-events-revisions/ -- quandl's API is free but the earnings data comes from Zacks: https://docs.quandl.com/v1.0/docs/data-organization -- earningscalendar.net tells you which companies announced earnings on a specific date but not when upcoming earnings are, for that - it's $50/month. -- intrinio is $100/month: https://intrinio.com/data-tag/next_earnings_date?

You can search for "earnings data API" and sift through many more. If you find one that's relatively inexpensive, that might be worthwhile to code against.

wahidsaleemi commented 5 years ago

For #2 above, you can look at this sample which gets the date without scraping the website: https://pastebin.com/y7Vrrneq

EDIT: I've updated the code to use earningswhisper.com; it should run a bit faster now too.

SCCMAdmin commented 5 years ago

Thanks @wahidsaleemi .

Re:

  1. Understood. Parsing text copy of downloaded HTML page.

  2. Noted. Also will test out new script version using earningswhispers.com (please note whispers is plural and in notes I can see you have typed whisper). Hey, the readme files that are here on GitHub, why is their extension .md ? What does that mean?

  3. IB's TWS API is free (non-commercial). Normally you need an IB account and data paid for, for the data you want to extract. Full documentation here if interested http://interactivebrokers.github.io/tws-api/index.html. FYI, I've asked Stan about the $5/month comment and awaiting a response.

  4. Qmatix XLQ being an Excel add-in, uses other data sources to get data into Excel. I use the IB data source and I can pull stock and option prices (and other related metadata) into Excel for the data I have paid for in IB, e.g. US stocks and options pack. So technically no additional costs as there is no additional data source (like another web site). So if I can get earnings data in IB TWS, then I'm sure I can it into Excel too. But I digress (however that could be useful in itself for other things I do). I will let you know what future travels in this area discover.

SCCMAdmin commented 5 years ago

Oh... regarding "For #2 above, you can look at this sample which gets the date without scraping the website: https://pastebin.com/y7Vrrneq" I did see this. Yes I understand that one too.

I see my output is (correct!): Wednesday, 30 January 2019 8:30:00 AM ...matching Tuesday, January 29 2019 4:30:00 PM (ET. on the website and I guess your prompt shows this). I note this one too for AAPL is showing confirmed at earningswhispers.com and stocksearning.com is still behind showing a 7th Feb estimation.

Will comment on script updates in new issue. ;-]

wahidsaleemi commented 5 years ago

the .md extension is for Markdown: https://en.wikipedia.org/wiki/Markdown

I did look into TWS API, seemed complicated to me but there's tons of github projects that do stuff with it. Most are written in java or python. Personally, I don't have interest in building anything on their API. The Excel add-in seems promising though. I usually just sell options spreads on the indexes (SPX mostly) so haven't really spent too much time on automating for stocks.

edouardswiac commented 4 years ago

Late to the show but check out https://www.earningscalendar.net/ (discloser: I help run it)

SCCMAdmin commented 4 years ago

Thanks @edouardswiac . We shall keep that in mind. As you can see this is a slow project. :-) Happy New Year to you and @wahidsaleemi and all readers!

tmxwshmt commented 9 months ago

@SCCMAdmin Stumbled on this thread, there is now an API feed available in TWS for earnings dates (and other event data too): https://www.interactivebrokers.co.in/sso/Login?action=TA_MARKET_DATA&loginType=1&serviceID1=951&serviceID2=952