Ib 1m hist

[goodboy]

~Super WIP~ Ready for action, ~but~ just a start at doing 1m long term (slow) chart alongside our classic 1s OHLC in the fast chart.

This adjust our history loading in the data feed layer (piker.data.feed) to do multi-time frame data loading concurrently and in a highly reliable manner such that both can be stored in the tsdb as well as explicitly queried, loaded and processed in shared mem arrays.

---

ib related

Since ib is the only currently supported backend with 1s OHLC history, this patch focuses around it but contains necessary adjustments to handle backends (like all the crypto$) which don't have this support (at least not without us writing our own sampler). When a backend doesn't have 1s OHLC history the fast chart simply starts empty and starts filling when the brokerd feed is first booted - during the pikerd parent's lifetime.

Further enhancements in this backend include:

• way better and faster history loading by re-jigging the data feed reset hack task-concurrency (6 years spy loaded to marketstore in like, <= 2mins 🥳 )
  • adjust the frame query timeout to 3s
  • also includes a mutex around the reset hack request (task) to support multi-symbol backfilling from multiple clients
• add back support for the the ad-hoc symbol table system for things like bitcoin futes (brr.cmecrypto) which seems to require inconsistent contract params when selecting in ib_insync
• (28535fa9776cd3f691e87d3ac1ef70641897fb35) add feed reconnect task which reloads whenever a network/feed reset event is detected to avoid hanging for whatever internal timeout-reset ibs api does..
• api adjustments to get the first datetime from Client.get_head_time() with an fqsn input str and use this stamp as the earliest stamp allowed before raising DataUnavailable to the history mgmt layer
• also add a "no data"-for-x-queries threshold where after 6 days worth of empty frame-results we presume the contract has no earlier history and we also raise a DataUnavailable
• a variety of other small backend-internal improvements to the history loading apis and mechanics to support the above.
• (ceca0d9) some tweaks to trades ledger parsing/loading as briefly mentioned in #412 but that need to land with this change set.

The summary of enhancements and bug fixes is more or less in the todo section below:

---

TODO:

• [x] avoid throttle state condition that shows up with too many open data reset hack requests:
  • [x] use global (mutex) state var to support simultaneous contract queries (dabb9e8)
• [x] improving the the 1m loading algo to avoid slow waits on queries after 3s (completed and working well after dabb9e8) also)
  • [x] multi-contract history queries need a mutex around the data reset hackery
  • [x] better task conc around data resets: one task for query, one for gw reset poll loop
• [x] actually a day (or more)'s worth of 1s day (went with 6d if tsdb is up and 1 if not)
  • [x] if the backend suports 1s OHLC write loaded frames to tsdb
  • [x] handle providers who don't support 1s

[goodboy]

Just pushed a bad timeframe wiper hack @guilledk