sfoster1
commented
4 years ago Option 2 does seem like the best one to me. In addition to your notes about the options, it doesn't seem impossible to me that the two liquid tracking components might at some point want to diverge - the goal of the python API in the long term is to enable arbitrary complex protocols with integration for external hardware and so on, and it might end up requiring different liquid handling. Keeping the two modules separate therefore means you don't have to have a bunch of switching logic in the shared module that would rapidly end up with option 2 anyway. And once these are two separate modules, we should always have consistency within a stack.
I agree with the rest of your points here, but I'd like to bring up something else: this could be a solid impetus to execute planned refactors around making the python protocol API and the json protocol executors true siblings. Specifically, factoring the control code and deck coordinate handling into an executor layer that is shared by the protocol API and json protocols, which doesn't have its own liquid tracking - in favor of the upstream interface providing it directly - and can be leaner and faster.
In the specific case of liquid tracking, we make the json protocol handlers and the python protocol API responsible for liquid tracking on their own. That lets the json protocol handlers just look at the data the protocol designer already calculated and avoid duplicating work. There's no extra information the runtime liquid level tracking can give because we don't actually have feedback.
IanLondon
pantslakz
umbhau
theosanderson
yfukai
b-cooper
bndo
rick-osmo
Overview
This is an RFC on our approach to introducing basic well liquid/volume tracking to the Python protocol api.
The Python protocol api currently only tracks volume as far as the pipette, so the hardware controller knows how much to move the pipette's plunger for any given atomic pipetting action.
The recently introduced Thermocycler Module needs to know the volumes of the samples in its loaded labware in order to precisely heat and cool its wells. Now is the right time to consider how to track the position and volume of liquids on the deck of the OT2 at runtime.
Requirements
A successful first implementation should address the following goals:
The implementation should also consider the following longer-term feature additions:
Proposal
This RFC proposes that we implement basic well-level volume tracking in the protocol api.
In
opentrons/protocol-designer/step-generation(currently only used by Protocol Designer), lives all of the logic to track the state of liquids on the deck of the OT2 across protocol commands. It is important that whatever Python implementation we use not functionally diverge from the Javascript implementation. We can enforce this relationship in a number of ways:1. Create a method-to-method carbon-copy of
step-generationin Python. Write robust tests that consume the same artifacts and enforce identical outputs.2. Create a Python module that accomplishes the same goals in a style that is closer to the rest of our Python logic. Write robust tests that consume the same artifacts and enforce identical outputs.
3. From the existing Python code, execute the
step-generationlogic in a Javascript interpretter, probably in a different thread running Node.Given these three rough approaches, I think Option 2 is the best fit for our needs at the moment and will serve us going forward.
Option 3 increases the testing area of all our robot software, and has some serious implications surrounding tight coupling and implicit co-versioning across, PD, Run App, and robot software. Option 1 on the other-hand is less tightly coupled, but still has a hazy contract of implementation details. Our JS and Python have enough conventional differences, that I suspect we would be fighting from both sides to compromise whenever possible.
Option 2 is only as strong as the test suite that ensures parity between the two implementations. This feels like a fine place to assert sameness, as each implementation may develop as its domain requirements grow. The tests can serve as the contract that protects against cross-stack breaking changes. This brings up another relevant point. How does this effect our various versioning mechanisms?
As the tests in Option 2 only assert behavioral uniformity, not necessarily structural similarity, each project should be able to append functionality without "breaking" that contract. Editing existing functionality is a little bit tougher. For instance: if the JS
step-generationwere to be updated in a way that effected multichannel access rules, we'd want to make sure that the Python implementation would mirror that update. We're then faced with the possibility of people creating protocols in PD that simulate liquid state in the UI (viastep-generation) differently than the robot software's run/simulation time representation of liquid state.Without adding a new version for the step-generation module to the mix, we may be able to use PD's existing versioning with some new UI in the Run App. In PD, a Major or Minor bump constitutes a change that may be breaking (currently at the PD import file migration level). We could establish the convention that a change to step-generation that gets included into a release of PD always requires at least a Minor version bump. On the Run App side, we can surface an error (or warning), at simulate time, to express that the protocol will still function properly, but may differ from your expectations based on the PD's rendering of the protocol.
Discussion/Future Work
At its smallest scope this python implementation of liquid tracking would just have to cover
aspirate,dispense,blowout, and potentiallydroptip. This approach would be sufficient in many cases to supply the thermocycler with accurate volume information at runtime. Those cases that it wouldn't cover, (e.g. initial liquid volume in TC wells > 0), are common and not adequately addressed with the minimal solution.Because of this, I propose that our first implementation aim for basic parity with Protocol Designer's underlying
step-generationliquid tracking. This similarly only requires coverage ofaspirate,dispense,blowout, anddroptip. The main difference is that instead of just tracking gross volume in wells, we also introduce the concept of liquid identity. This is the mechanism that allows PD to track mixtures/locations of reagents across a protocol. This sort of information paired with liquid metadata would allow for a rich runtime experience that is much more adaptable and extendable for features that are down the road.This approach gives us much more leverage for creating connective tissue between our two protocol creating api's. For example, using a PD-esque UI to populate your starting deck state, and exporting that to be referenced in a Python protocol. This would save us from strict reliance on the inevitable suite of utility functions required for an average Python user to populate their deck and its labware with reagents at the beginning of every protocol.
The opposite direction could also be fruitful by allowing the robot software to send live updates of the current liquid state at runtime to clients in a consistent format to the way it is described under the hood in PD.
┆Issue is synchronized with this Wrike Task by Unito