The first MVP for overlay charts lands in #455 and contained a plethora of precursor PRs including:
full multi-fqsn support in our data feed layer with multiple symbols per brokerd feed (#414)
new samplerd daemon introduced to conduct singleton sampler step event synchronization across multiple flow-actor processors (#440)
mega rework to the pre-graphics ShmArray -> QPainterPath rendering pipeline by adding a formalized IncrementalFormatter layer which allows incremental update (and cache) of source data to pre-graphics formats suitable for minimal conversion to a painter path (#447)
internally adjusted Axis sticky label APIs (moved to lower level PlotItem in #448)
multi-symbol input to the real-time graphics system(s) supporting first draft of multicharts (#449)
epoch-time index alignment of multiple time series and a graphics-layer driven "data viz" sub-sys (#451)
drastically simplified L1 labels to enable tight (horizontal) stacking of y-axis co-domains (#452)
[x] initial multichart latency tuning (#453) through reworks of the display loop's:
maxmin() y-ranging and better support for only draw last uppx pixel column style
only update from last framed (by type) tick
distribution of throttle rate uniformly over number of input fqsns
better Viz API pipeline usage to speed up interaction handlers
pre-graphics 1-d data caching and use in only draw last uppx pixel column style updates
[x] #455 algorithm which allows sanely viewing multiple time series with co-domains normalized via a returns transform:
includes support for incremental yrange update from display loop via inputs to ChartView.interact_graphics_loop()
removal of all Qt.Signal stuff from view box handlers instead opting to call our .interact_graphics_cycle() manually from mouse event callbacks
better leverage of new Viz update methods for faster pipeline updates from incremental quote driven updates in the .ui._display loop.
support for multiple overlay methods as finalized in this commit:
[x] basic initial usage is to start a chart with multiple input symbols:
piker chart btcusdt.binance ethusdt.binance xmrusdt.binance
[x] overlay method control for how to align overlays to one another.
[x] L1 labels can be simplified
[x] we decided to make them overlay on their respective y-axis
[x] how should last datum graphics be offset
[x] currently all last datums are time step aligned.
History view
[x] curves need to be epoch time aligned instead shm index aligned
since obviously it will result in non-time aligned data sets as it
is now using the 'index' basis..
[x] fix pg.LinearRegion item to show all background overlay curves transparently.
Graphics loop
[x] use single Feed.open_multi_steam() for the update task and have it look up appropriate graphics update apis per symbol-quote
[x] refactor a bunch of the charting/graphics apis to mostly to remove
the global-ish ChartPlotWidget instance and API calls, instead
moving all this state into Flow / Flume items and possibly add
a Viz type to replace the Flow and instead make the Flow
a table of associated src data -> procesor pipelines?
more to write up as individual subtasks here
Search UI/UX:
[x] make search results and caching work with multi-symbol selections by the GodWidget
[x] all selected symbols should be shown in fqsn form in the window title bar
Follow up ToDo task list after the last PR (#455) lands above:
These include outstanding todo features which are still in discussion / research.
(Edit-able) list of rt-feeds on chart: the ViewList:
[ ] more or less very similar to the legacy idea of loading and saving
watchlists but instead is a more sophisticated abstraction: a list
of trade-able asset pairs that maps to a display of rt-data-flows
shown overlayed in a single chart view
[ ] obviusly a declarative syntax much like you'd expect for
a legacy watchlist but with support for additional metadata:
[{'btcusdt.binance: {'pin_to': 'qqq.nasdaq.ib'}}, 'qqq.nasdaq.ib']
or something or other.. (we need to figure out the generic
data-structure syntax form(s)
[ ] a way to edit multiple lists within the chart UI using (probably)
vi-like controls similar to those in the ranger file manager
except for moving entries between multiple stacked view lists in the
search pane.
bugs^
[ ] current cachelist is not in same order as that passed on CLI?
common axes for common dst-asset:
[ ] how to support same-dst but cross-broker markets, eg. xbtusdt.kraken and btcusdt.binance
[ ] should we use a single axis?
[ ] how to determine this automatically?
Do a in the same range calc ?
have some kinda of adhoc asset name mapping?
OHLC chart overlays:
[ ] for ohlc main chart overlays how should we do coloring?
currently we pull from a sequence of colors in .ui._display but
we should likely make this configurable, possibly through the UI,
definitely through a config file.
[ ] how to handle overlaid (levered) derivatives where the %returns
are substantially larger due to the lever profile?
[ ] it might make sense in these cases to instead use a "leverage
and delta adjusted" scalar for the opt contract so that you can see
a "somewhat linear" translation of the deriv price vs. the underlying?
the issue is that the call option is clearly causing the bitcoin
move to be mega squashed in the y-range.
[x] L1 labels were simplified (we decided to make them stack on
respective y-axis):
[ ] only show (humanized) size on bid/ask not price?
[ ] in terms of last datum bars, could they be placed in some kind
of no-exactly-aligned (reverse) order of the y-axis from
left-to-right?
[ ] des-synced last bar issue on 1Min chart seemingly due to startup
races where there are N > 2 overlay curves from diff providers?
[ ] is this something we can fix in the .data.feed shm allocation
logic?
[ ]
FSP (subcharts)
[ ] should fsps like vlm which require a subchart be overlayed or only
the currently selected shown by default?
[ ] we can make this configurable and maybe offer a sidepane drop down
showing all combos and/or a toggle set of symbols from the overlay
set?
[ ] can we offer a pair selector to make it possible to show the fsp
subchart for any market (in the viewlist) not currently selected
on the OHLC chart(s)?
this would probably help perf issues when using binance :joy:
=> bugs^
[ ] we still have some strange issue on our (vlm) FSP charts where
when multiple overlays are being backfilled the chart will get all
mucked up with strange lines from offscreen connecting to every step?
(TODO: PUT PIC HERE from @guilledk!)
Widget UIs: panes and controls, search..:
UI for "pair selection"
[ ] dynamic management and selection of a current market pair overlay on a chart as per
the ViewList idea from above:
[ ] (right-click) context menu options to add/remove some chart
from/to a diff viewlist?
[ ] editing a ViewList:
[ ] mouse UX (shift-click-drag-release?) to create a group from a single/flat results tree?
[ ] hot keys for adding a new search result to a group instead of a new widget set (maybe ctl-o or ctl-enter on result row?)
[ ] API allowing for the currently selected market which would
activate functionality for the order mode and possibly (or
independently) the current FSP processing (in subchart).
[ ] current market-feed selection controls:
[ ] highlight the feed's curve vs. the other overlays
[ ] highlight the matching y-axis?
[ ] re-order the axes, selected should be closest to curve?
[ ] show feed meta-data in sidepane label for selected
[ ] allow selection by y-axis?
[ ] double click on axis?
[ ] or, double click on curve?
[ ] or, select by hot key (tab or space global hotkey?)
[ ] activate order mode control for that symbol (see more details in section below)
UI for overlay-curve config:
[ ] support config (meta) data to choose the pin-method (to
.ui.view_mode.overlay_viewlists()) as well as parameters for each
mode if supported (eg. 'loglin_ref_to_curve' can take a target curve
to pin to)
[ ] unify this with the current "feed config" controls: eg. throttle
rate, shm buffer sizing, host discovery, what daemon to use as
source.. etc?
Search UI/UX:
[ ] what viewlist(s) should be loaded by default?
[ ] obvs make it configurable
[ ] how lazily should we load the rt feed for each?
[ ] a new result should be added to existing viewlist?
[ ] obvs make this configurable through toml file.
[ ] how would this play with the idea of an alt-search for
derivatives?
[ ] how to open/start a new view list?
[ ] how to search through existing viewlists
[ ] if you pick a result from one do you load the whole thing?
=> bugs^:
[ ] current results order is not the same as CLI input order..
better distinction of pairs on axes?
[ ] draw little line from each last label to curve?
[ ] some kind of label system to easily know which curve is which feed?
[ ] last label should include fqsn?
[ ] each y-axis should have fqsn?
Order mode
order mode client multi-market API support:
[ ] allow passing a fqsns: list[str] to open_order_mode() and
having this setup the client loop and the ems tasks to use
feed.open_feed() with the same fqsns passed through?
[ ] what should be the handle delivered back? a table of clients
or should we add some kind of client-selectable API already that
strat code can (eventually) use for easy arbing?
selection oriented UX, show each mkt's EMS state by axis:
[ ] pp line arrow-marker should sit on the specific y-axis
[ ] depending on "selected symbol" only show that pp?
[ ] how/when to show all pps?
[ ] show active order summary markers (for those not in view) on each sub-yaxis
[ ] maybe only pinned arrow at ppu level always?
[ ] support submitting orders only for current selection:
[ ] eg. alert submitted should show on only that axis's y-domain
[ ] cc cancelling should only cancel current instrument's orders?
[ ] tie this in with the viewlist / search result pane, show
active orders indicator of some sort?
The UI follow up to #414, it was originally a PR #420 but ended up being a sequence of smaller (factored) PRs which landed finishing up with #455 ๐
Hopefully this is the final core-UX feature you always wanted ๐๐ผ
mult-instrument overlayed real-time and historical data feeds with simultaneous interaction and "current symbol" selectable order mode control
To run and test the UI:
piker chart btcusdt.binance ethusdt.binance xmrusdt.binance
This should (currently) spawn a chart with all real-time quote feeds and history overlayed on both the fast and slow chart and boot using the
method=loglin_ref_to_curve'
overlay method by default.First draft feature set included:
The first MVP for overlay charts lands in #455 and contained a plethora of precursor PRs including:
brokerd
feed (#414)samplerd
daemon introduced to conduct singleton sampler step event synchronization across multiple flow-actor processors (#440)ShmArray
->QPainterPath
rendering pipeline by adding a formalizedIncrementalFormatter
layer which allows incremental update (and cache) of source data to pre-graphics formats suitable for minimal conversion to a painter path (#447)Axis
sticky label APIs (moved to lower levelPlotItem
in #448)L1
labels to enable tight (horizontal) stacking of y-axis co-domains (#452)Viz
API pipeline usage to speed up interaction handlers[x] #455 algorithm which allows sanely viewing multiple time series with co-domains normalized via a returns transform:
yrange
update from display loop via inputs toChartView.interact_graphics_loop()
Qt.Signal
stuff from view box handlers instead opting to call our.interact_graphics_cycle()
manually from mouse event callbacksViz
update methods for faster pipeline updates from incremental quote driven updates in the.ui._display
loop.] = 'loglin_ref_to_curve',
Init basic funtionality / usage:
piker chart btcusdt.binance ethusdt.binance xmrusdt.binance
History view
'index'
basis..pg.LinearRegion
item to show all background overlay curves transparently.Graphics loop
Feed.open_multi_steam()
for the update task and have it look up appropriate graphics update apis per symbol-quoteChartPlotWidget
instance and API calls, instead moving all this state intoFlow
/Flume
items and possibly add aViz
type to replace theFlow
and instead make theFlow
a table of associatedsrc data -> procesor
pipelines?Search UI/UX:
GodWidget
Follow up ToDo task list after the last PR (#455) lands above:
These include outstanding todo features which are still in discussion / research.
Charting graphics layer: flows mgmt, backendy controls, viewlists, schema:
(Edit-able) list of rt-feeds on chart: the
ViewList
:[{'btcusdt.binance: {'pin_to': 'qqq.nasdaq.ib'}}, 'qqq.nasdaq.ib']
or something or other.. (we need to figure out the generic data-structure syntax form(s)ranger
file manager except for moving entries between multiple stacked view lists in the search pane.bugs^
common axes for common dst-asset:
dst
but cross-broker markets, eg.xbtusdt.kraken
andbtcusdt.binance
OHLC chart overlays:
.ui._display
but we should likely make this configurable, possibly through the UI, definitely through a config file..data.feed
shm allocation logic?FSP (subcharts)
=> bugs^
Widget UIs: panes and controls, search..:
UI for "pair selection"
ViewList
idea from above:ViewList
:shift-click-drag-release
?) to create a group from a single/flat results tree?ctl-o
orctl-enter
on result row?)tab
orspace
global hotkey?)UI for overlay-curve config:
.ui.view_mode.overlay_viewlists()
) as well as parameters for each mode if supported (eg.'loglin_ref_to_curve'
can take a target curve to pin to)Search UI/UX:
=> bugs^:
better distinction of pairs on axes?
Order mode
order mode client multi-market API support:
fqsns: list[str]
toopen_order_mode()
and having this setup the client loop and the ems tasks to usefeed.open_feed()
with the samefqsns
passed through?selection oriented UX, show each mkt's EMS state by axis:
cc
cancelling should only cancel current instrument's orders?