Open greg-latuszek opened 3 years ago
Generic assumption in searching for solution: we don't want to overcomplicate State Machine (we will use SM abbreviation) of existing devices.
Analized solutions:
Since command knows it's states it might inject them into SM of hosting device.
Moler devices use https://github.com/pytransitions/transitions as machinery to implement their SMs. Implementation is a bit tricky since we want:
goto_state() is not the must for interactive commands. Currently envisioned SM for interactive command is simple, just 3 states: NOT_STARTED, INTERACTIVE, END. Of cause, in future, there might appear interactive commands with more states. However, it looks like they might be implemented in pure pytransition API. That would constitute new type of SM, different from SM of devices. Such SM would work as sub-SM inside current state of device SM. Device would treat it as running command. Interactive command would be still conceptually a command, however, more complicated one.
Firing interactive command would create new device being SM (in same form as current SMs of moler). Hosting device would go into "transparent mode". So, conceptually hosting device would tread interactive command as single state being nested SM with its own states. That would not require to modify states of hosting device but would require to refactor its code. Since interactive command may happen in multiple states device must be able to programmatically jump into "virtual state" of running interactive command.
Reusing current SM for interactive cmd (on example of openssl):
Selecting solution 3 as most beneficial even if it requires refactoring code of moler devices SM.
Majority of changes will happen inside moler/device/textualdevice.py
It is property of TextualDevice
. If we jump into sub-SM it should return state of nested device prefixed with device class name.
Like: OpenSsl.INTERACTIVE
. So for hosting device it is like "I'm in state OpenSsl" with sub state 'INTERACTIVE'.
We don't want to use _
as substate separator since it would be ambiguous where is state and where is substate (moler already uses _
in state names like UNIX_REMOTE
). See same discussion at https://github.com/pytransitions/transitions#hsm
This returned value doesn't come from prompts observers detection, nor from states listed on hosting device SM configuration. It is just returned when device detects "I'm working in transparent mode".
So, we will need new member: self._transparent_SM_mode
Nested SM will see something like NOT_STARTED
--> INTERACTIVE
--> END
Hosting SM will see OpenSsl.INTERACTIVE
They will work as they are. We will not add nested SM prompts (like OpenSsl>
) into prompt observers of hosting SM. Old responsibility remains - host SM only knows own prompts. So, for example it can detect dropped connection to UNIX_REMOTE
.
Nested SM will have prompt observers only for own prompts (like OpenSsl
). So, it has no means to move it into END
state. It is not its responsibility. It will be done by hosting SM since it knows prompt/state which nested SM started from.
So, we need to change prompt observers callback _prompts_observer_callback
. Besides setting state it should:
END
def _validate_prompts_uniqueness()
We need to know prompts available in sub-SM and compare it against prompts of hosting SM. Otherwise we may have 2 prompts observers keeping eye on same prompt. That would lead to risky code with races.
That may become limiting factor nesting level of devices (nested SM being host SM for next-level nested SM)
def get_prompt()
It is used by get_cmd()
to fill cmd_params["prompt"]
of cmd to create. Called only if prompt for new command is not directly provided. That prompt is used by cmd to detect when it is done.
If we are in transparent mode and requesting device to create some subcommand of interactive command then:
get_prompt()
to nested SM get_prompt()
get_prompt()
from inside of get_cmd()
However, since get_prompt()
is public API it's better to utilize proxy when self._transparent_SM_mode
.
If we jump between states of current moler device it checks what type of newline is defined per state. Since it may happen then local console has different line endings then remote console.
However, as we think about host SM and nested SM, they both share same connection. Nested SM being interactive command runs inside same shell of some device. In most cases they would share same newline. But nested SM should not know from which state it starts (from which console).
So, as a default "newline of current console" should be passed down from hosting SM into nested SM.
It is also possible that nested SM changes mode of that console handling and uses different newline. But in such case nested SM knows it from its internal code.
def _get_newline()
Above analysis shows a need to refactor def _get_newline()
. This method depends on property self.current_state
that requires refactoring. Moreover, it uses self._newline_chars[state]
and we don't want to pollute host SM states&newlines with those of nested SM.
Caution: simple proxing into nested SM without passing "newline of current shell" into construction of nested SM might accidentally change newline of host SM - since default returned from _get_newline()
is "\n"
(what if host SM was not "\n" when started sub SM?)
_collect_observer_for_state()
, _collect_cmd_for_state()
, _collect_event_for_state()
and relatedNO CHANGE NEEDED
These methods return command/event names available for given state - so, they return dict. Names are fully qualified names like moler.cmd.unix.ip_addr.IpAddr
used by instance loader.
They are used inside _load_cmdnames_for_state()/_load_eventnames_for_state()
which are used by _collect_cmds_for_state_machine()/_collect_events_for_state_machine()
.
They build those dicts based on _get_available_states()
which should return only states of hosting SM and not nested SM nor "virtual" states of form OpenSsl.INTERACTIVE
.
_get_available_states()
NO CHANGE NEEDED
It just returns self.states
which is modified by _update_SM_states()
called from _add_transitions()
which is used by derived classes (like ProxyPc
, UnixLocal
) to create states and transitions based on SM configuration.
That means, it will work since we are taking here no nested states nor "virtual" ones.
get_cmd()
, get_event()
, get_observer()
Main focus is on get_cmd
but same code modification should be made for get_event
- for parity and because nested SM may also posses some events.
We need to analyse 2 cases, starting from simpler one:
In such case hosting SM doesn't know commands of nested SM.
get_observer()
uses _load_cmdnames_for_state()
(building dict described above) and then builds command object of specified name. So, for example for IpAddr
it searches its fully qualified name moler.cmd.unix.ip_addr.IpAddr
inside dict created by _load_cmdnames_for_state()
.
F.ex.: hosting SM won't find fully qualified name for s_client
since it is known only to nested SM of openssl
device.
Concluding hosting SM get_cmd()
should proxy towards get_cmd()
of nestes SM in case hosting SM is in transparent mode.
openssl
That should run whole machinery of:
Interactive command should be new type of generic command derived from CommandChangingPrompt
. Proposed class name CommandCreatingSM
. Constructor of such class should get:
openssl
device)
openssl
cmd --> Openssl
dev) oropenssl_cmd.started_devname()
(return "OpenSsl"
)Thanks to closure parameter the command won't hold device directly but will have opaque method to just call when command succeedes. In reality reference to hosting device and newly created nested device resides inside that closure method. However, interactive command can do nothing with it besides calling it so, we have narrow dependency in the form of well known pattern "dependency injection". That should also help in testing.
That closure may also have a form of context manager. That way its entry
would be responsible for all the stuff related to making host SM transparent and exit
would be responsible for restoring hosting SM into normal mode. Thanks to context manager as generator we can have setup/cleanup code side-by-side. That would help in maintenance.
goto_state()
State in current moler SM implementation may be changed in 2 ways:
on_connection_made()
into CONNECTED
stateon_connection_lost()
into NOT_CONNECTED
stategoto_state()
If hosting SM is in transparent mode it means it is in "nested SM as my one big state".
So, in transparent mode - any state change caused either by prompt observers, connection callcacks or via jumps inside goto_state()
should:
However, there might appear two implementations for "finalize yourself". Hard and soft one. It comes from nature of state change inside hosting SM:
END
state, etc).goto_state()
is different. It is request "take me from current state towards another one; you have timeout time to do it". So, it may wait for active command inside nested SM to complete. Another words it is SOFT shut down request.Implementation may lead us to same code. That would be great to not have a need to differentiate the two cases - lower maintenance cost. However, above is a reasoning that justifies two different code pieces.
END
state inside nested SMMaybe having END
state in nested SM is a way to go for implementing "shut down". It is not adding any new API this way. Such requirement for all nested SMs (or maybe for all SMs) builds chance for polymorphic behavior. Hosting SM doesn't need to know type of nested SM. It just knows that whenever it wants to shut down nested SM it just says "goto END state".
All devices have API def register_device_removal_callback(self, callback)
and def remove(self)
. Registration is used by DeviceFactory
. Factory performs caching - when you call get_device(name='DEV1')
next time and 'DEV1' has been already created, it is returned to you without recreation. This way we keep devices "all time open" to save connection establishment time. Of cause you can close that device via DeviceFactory.remove_device(name='DEV1')
and/or uncache it via DeviceFactory.forget_device_handler()
.
The consequence of caching is that device with same name point to same device object inside factory cache. If you want to use same name to refer another device you need to uncache it first.
remove_device()
finds cached device of given name and performs dev.remove()
on it. That allows device to do its own cleanup.
dev.register_device_removal_callback()
is used by DeviceFactory
during device creation. A a callback it sets forget_device_handler
. So, whenever device is removed it is also uncached.
Moreover, removal device callback may be called multiple times - it can store multiple callbacks to be called when device is removed.
That functionality may also be used for hosting/nested SMs cleanup. Besides uncaching device we may also do some "nesting SM cleanup" just by registering another callback.
dev.remove()
just calls all callbacks that have been registered on device via dev.register_device_removal_callback()
TextualDevice.remove()
TextualDevice
overwrites basic functionality of AbstractDevice
depicted above. Besides calling all registered remove-callbacks it also performs state and connection cleanup:
NOT_CONNECTED
state using goto_state()
WE DON'T WANT it for nested device. We don't wan't returning from nested SM to cause closing connection on hosting SM (they both share same connection). Besides it, nested SM has no NOT_CONNECTED
state. It has END
state.
So, dev.register_device_removal_callback()
requires refactoring to catch up with transparent mode device.
Why we need it?
If we face interactive command like openssl: https://www.openssl.org/docs/manmaster/man1/openssl.html https://wiki.openssl.org/index.php/Command_Line_Utilities (search "interactive mode") then we may end up with something that may be started in multiple states of typical Moler device. openssl may run inside UNIX_LOCAL, PROXY_PC, UNIX_REMOTE. Moreover, running interactive command introduces its own prompt (OpenSSL> for example). It finishes running openssl command but after it you can't run normal linux commands, just those of openssl. So, it constitutes state of device State Machine. We might introduce new state into existing devices but:
Concluding: such solution would generate exponential effort, unreadable code and maintenance nightmare.
Comments following below are results of days lasting discussions and analysis. It will discuss: