This pull request primarily addresses a bug introduced in v3.3.0 where multiple components can be run using ez.run syntax. This new feature came with an unforeseen consequence where these components could possibly (and with startling frequency tended to) have duplicate names, resulting in multiple subscribers/publishers sharing the same topic names. This has been addressed with a change to the ez.run call signature, requiring unique names for every component to be run by ezmsg. Additionally, when run this way in v3.3.0, every component was run in its own process which could be circumvented by using force_single_process but ultimately did not give developers the flexibility they needed when running multiple components. To address this, we have added a process_components keyword argument to the ez.run call signature which specifies which components are to reside in dedicated processes. By default, all components passed to ez.run will now run in the same process (unless sub-components specify their own process_components).
Example v3.3.1 call signature for ez.run
ez.run(
# All components are specified as name: component pairs.
# This construction requires all components to have unique names at this level
# All caps are used for these names in this example just for convention.
REPLAY = replay,
PREPROC = preproc,
SEQ_COLL = sequence_collector,
FILT_COLL = filt_collector,
TERM = term,
# Use the root_name keyword argument to have all of these units reside under a designated root collection
root_name = "PROJECT",
# Connections between these components and streams can be specified just as before
connections = [
(replay.OUTPUT_MESSAGE, preproc.INPUT_SIGNAL),
(preproc.OUTPUT_SEQUENCE, sequence_collector.INPUT_MESSAGE),
(preproc.OUTPUT_FILTERED, filt_collector.INPUT_MESSAGE),
(sequence_collector.OUTPUT_MESSAGE, term.INPUT)
],
# Here, you can specify which of the components passed into ez.run should have a dedicated process.
process_components = [preproc]
)
If you used to call ez.run with a single component, you should consider adapting your code fromez.run(system) to ez.run(SYSTEM = system). Although your code will still function when passing a single component, you should receive a little warning from your typechecker, and this usage is now considered deprecated, and you will receive a DeprecationWarning
Additional features introduced:
ezmsg.util.messagelogger.MessageLogger now logs timestamps for every message in the text file
ezmsg.util.messagereplay.MessageReplay can be run dynamically by publishing Path objects; message rate can be controlled to replay files as fast as possible, at specified message rates, or "as recorded" using these new timestamps
NOTE:MessageLogger has a new message at the start of every log file that just indicates the start time of the log file. This is currently used by MessageReplay when replaying messages "as recorded". If you've been using ezmsg.util.messagecodec.MessageDecoder to directly decode json messages from these files, you may now notice a LogStart message in your file that you didn't put there. To address this, ezmsg.util.messagecodec.message_log has been introduced as a generator that opens the log file, decodes the messages using MessageDecoder and skips over this LogStart message.
Bugfixes
Typing for process_components only requires a typing.Collection[Component] instead of a typing.Tuple[Component, ...]
ezmsg.util.messages.axisarray.AxisArray:
isel allows integers, slices, and index arrays
Users can specify indexers keyword argument, or just use kwargs to specify indexers, just like xarray
ezmsg.util.messagereplay.MessageLoder is gone. It was not a meaningful addition to the core.
ezmsg.core 3.3.1
This pull request primarily addresses a bug introduced in v3.3.0 where multiple components can be run using
ez.run
syntax. This new feature came with an unforeseen consequence where these components could possibly (and with startling frequency tended to) have duplicate names, resulting in multiple subscribers/publishers sharing the same topic names. This has been addressed with a change to theez.run
call signature, requiring unique names for every component to be run byezmsg
. Additionally, when run this way in v3.3.0, every component was run in its own process which could be circumvented by usingforce_single_process
but ultimately did not give developers the flexibility they needed when running multiple components. To address this, we have added aprocess_components
keyword argument to theez.run
call signature which specifies which components are to reside in dedicated processes. By default, all components passed toez.run
will now run in the same process (unless sub-components specify their ownprocess_components
).Example v3.3.1 call signature for
ez.run
If you used to call
ez.run
with a single component, you should consider adapting your code fromez.run(system)
toez.run(SYSTEM = system)
. Although your code will still function when passing a single component, you should receive a little warning from your typechecker, and this usage is now considered deprecated, and you will receive aDeprecationWarning
Additional features introduced:
ezmsg.util.messagelogger.MessageLogger
now logs timestamps for every message in the text fileezmsg.util.messagereplay.MessageReplay
can be run dynamically by publishingPath
objects; message rate can be controlled to replay files as fast as possible, at specified message rates, or "as recorded" using these new timestampsMessageLogger
has a new message at the start of every log file that just indicates the start time of the log file. This is currently used byMessageReplay
when replaying messages "as recorded". If you've been usingezmsg.util.messagecodec.MessageDecoder
to directly decode json messages from these files, you may now notice aLogStart
message in your file that you didn't put there. To address this,ezmsg.util.messagecodec.message_log
has been introduced as a generator that opens the log file, decodes the messages usingMessageDecoder
and skips over thisLogStart
message.Bugfixes
process_components
only requires atyping.Collection[Component]
instead of atyping.Tuple[Component, ...]
ezmsg.util.messages.axisarray.AxisArray
:isel
allows integers, slices, and index arraysindexers
keyword argument, or just use kwargs to specify indexers, just likexarray
ezmsg.util.messagereplay.MessageLoder
is gone. It was not a meaningful addition to the core.tests/test_perf.py
message channels buffers testing.