Dividend API timeliness, flickering, coverage

dbregman commented 1 month ago

I've been polling the v3/reference/dividends every hour to see when it updates, and it seems that a lot of dividends for the current day only appear sometime in the middle of the day. To use the data in real-time for trading, the data should be available before the market opens, ideally before 4am or at least before 9:30am. Would be great if the API could be made more timely, if this limitation is not due to when polygon's datasource updates.
When I poll data and see that some dividend no longer appears in the data, I record it as deleted. I've noticed that some dividends "flicker", in other words they get deleted then re-added again a day or two later without any change. This happened so many times it seems like it must be a bug.
I have been comparing dividends with future dates from the API to MarketChameleon (https://marketchameleon.com/Calendar/Dividend) and I notice their calendar often has significantly more coverage, especially of ETFs. I found almost no cases where they were missing something that Polygon had [looking at listed stocks securities, not mutual funds or OTC].

If you guys are considering a different source for your dividend data, maybe you could check out the following: https://www.wallstreethorizon.com/dividends https://marketchameleon.com/DataFeed/Dividends https://orats.com/dividends

AHangstefer commented 1 month ago

Hi! Thanks for reaching out! We recently added an additional source for dividend and splits data and are still integrating the new data into our system so that may be the cause of seeing dividends appear mid day and flickering. Are you still seeing dividend data flicker- removed and re-added? Are you able to share any examples? I'm happy to look into this for you.

dbregman commented 1 month ago

I am still seeing many dividends for the current day (meaning ex-date is today) appearing mid-day, every day. This is the biggest issue. For examples, see the attached csv with ex dates from today: ex-date-2024-05-31.csv

The flicker doesn't seem to be happening much anymore, after looking closer it seems almost all of these happened with a big delete on May 23 and re-add on May 24. There are a few from May 27 though. Mot sure what conclusions can be drawn from this, but here is the data I recorded that shows flickers: dividend_flicker3.csv

AHangstefer commented 4 weeks ago

Hello again! It looks like we update dividends 4x a day- 7am, 9am, 3pm, and 8pm ET. The backend recently updated some things for the dividend endpoint which should help with the flickering you're seeing. Are you still seeing that happen?

dbregman commented 4 weeks ago

I don't have a quick way of checking the flickering at the moment. Regarding the update times, I don't think those update times are what your system currently implements. Here are the times (ET) when I detected updates (I am polling once per hour, 1 hour after the last time so the times drift)

2024-05-22 11:39:45 2024-05-22 16:45:07 2024-05-23 11:46:00 2024-05-23 16:46:07 2024-05-24 11:46:50 2024-05-24 16:47:00 2024-05-27 11:48:23 2024-05-27 16:48:27 2024-05-28 11:49:19 2024-05-28 16:49:39 2024-05-29 11:50:53 2024-05-29 16:56:43 2024-05-30 11:57:58 2024-05-30 16:58:20 2024-05-31 11:59:12 2024-05-31 13:59:15 2024-05-31 16:59:20 2024-06-01 00:59:32 2024-06-03 12:01:52 2024-06-03 17:05:32 2024-06-04 00:05:47 2024-06-04 12:06:03 2024-06-04 17:06:13

It appears the updates are mostly in [11am,12pm), [4pm, 5pm).

AHangstefer commented 3 weeks ago

Hello again! I believe those update times are when the update begins. Are you seeing these updates in the EST timezone or are you times from another timezone?

dbregman commented 3 weeks ago

The timestamps from my last comment are all in EST timezone.

polygon-io / issues

Dividend API timeliness, flickering, coverage #301