vicsharp-shibusa / kyna

Open source stock data collection and analysis.
MIT License
2 stars 1 forks source link

Enhance `kyna-data-import` to be more efficient #29

Closed vicsharp-shibusa closed 5 months ago

vicsharp-shibusa commented 8 months ago

Once we have data migrated to the financials database, we can be more efficient in how we capture data.

Examples:

  1. We do not need to capture fundamental data every day. Instead, we can look at the last piece of fundamental data, project when the next one will arrive (+90 days) and wait until then to try to capture the next chunk.
  2. We do not have to "discover" splits and dividends every day. Once we have captured those, we know which stocks have such things and can limit our queries to those symbols.
  3. We can make use of the eodhd.com calendar endpoints to identify upcoming data capture needs.
  4. We might identify symbols that we want to ignore for a variety of reasons (e.g., not enough data or average closing price is too low) and not attempt to fetch those records during the normal import cycle.

It will be appropriate to sometimes go into "discovery" mode and search all possible tickers for new splits, dividends, and fundamentals.

vicsharp-shibusa commented 8 months ago

This task is bigger than I anticipated; going to slide it to a future milestone. The existing solution works, but will require a bit of refactoring to align with this card.

vicsharp-shibusa commented 7 months ago

May be scrapping this issue if we migrate from eodhd.com to polygon.io. Pulling from milestone for now.

vicsharp-shibusa commented 5 months ago

Not going to do this one - not for a while anyway.