Switch to inferred drop motion

[EvanKirshenbaum]

While trying to add mirroring to the wombat platform (#60), it dawned on me that this would be a lot simpler if rather than trying to pretend we only had half the pads and have them show up on the monitor twice, we actually made separate pads that were ganged together (#61) In order for this to work correctly with drops, however, what we'd want to do is get rid of all of the explicit drop motion/merge/split/etc. code and have the board itself figure out what happens as part of the call to finish_update() for the board.

This would have the additional benefit that the drop inference model can be updated as we get more understanding of what happens in real life.

Considerations:

• I was going to say that this needed to happen between the callbacks returned by communication requests and those registered by add_monitor(), but nobody actually seems to call that, so I may just get rid of that and work it in after finish_update().
• The actual logic for drops on the board would be very similar to that used in the monitor for interactive clicks: drops can be pulled to a neighboring pad, split, and merged, based on the contents and state of their neighbors.
• The board will have to hold a set of drops that are on its pads.
• Keeping track of drop identity across merges and splits will be tricky, and code will probably want to explicitly look at the resulting pad for the drops to make sure it gets the right one.
  • For merges immediately followed by splits, the "disappeared" drop can be associated with the "still-present" one, and it can be reconstituted on the split.
  • We may want to just drop the off-board drops if a merged drop moves.
  • If we keep track of drop position before the merge, we can put the identical (merged-and-split) drops back in their starting points.
• Care will need to be taken when new reagents are assigned.
  • We may need to distinguish between "These reagents together always become this" and "This is an idiosyncratic name for this particular result".
  • It's possible that the drop may need an annotation of "If you combine with this (as I expect you will), call the result this".
  • It would be nice to be able to model the interaction with well gates as well. This might require the well gate also carrying a drop (and possibly well pads in general flagged as having liquid).
  • I'm not sure how we deal with wells that haven't been asserted to have fluid in them in the interactive case. Perhaps if the well's pads are activated, we assume it has fluid. (as part of the fix-up).
• For extraction ports, the port itself can carry "I just added V of R" and "I just removed V" annotations that get used and then cleared in clean-up.

  Migrated from internal repository. Originally created by @EvanKirshenbaum on Jan 09, 2022 at 6:30 PM PST. Closed on Feb 21, 2022 at 12:28 PM PST.

[EvanKirshenbaum]

This issue was referenced by the following commits before migration:

• 55da5f47bf67376c559a6c9cde6b9c5245ab38de by @EvanKirshenbaum on Jan 17, 2022 at 9:33 AM PST
• 1c776dad9a6c6c0032f1d3b88bb90a56f3aa73f2 by @EvanKirshenbaum on Jan 21, 2022 at 1:38 PM PST
• e435fb41ac86430a09f6f79da8dcbe1656557910 by @EvanKirshenbaum on Jan 24, 2022 at 4:50 PM PST
• 3df4e27d3fd24b84cf0eee655ff1895a9a5cdc3a by @EvanKirshenbaum on Jan 25, 2022 at 11:29 AM PST
• d20babf3f915ebda3950a2b10e691640f3590a92 by @EvanKirshenbaum on Jan 30, 2022 at 3:51 PM PST
• 1ea15ac418308cf3dc6e6f203ad16d95bce0eb55 by @EvanKirshenbaum on Feb 16, 2022 at 2:05 PM PST
• bdab2c1b42f1934e5e7ac5681ef10a4b339ca82e by @EvanKirshenbaum on Feb 16, 2022 at 3:35 PM PST
• 2fb70f0a89a76e5236add0aa518d6936e4854011 by @EvanKirshenbaum on Feb 17, 2022 at 12:20 PM PST

[EvanKirshenbaum]

I'm starting to think that well gates should be pads (rather than well pads), complete with a location just off the main board.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jan 10, 2022 at 11:26 AM PST.

[EvanKirshenbaum]

Time to get my thoughts down. After playing with a couple of approaches, what I'm going to try now is a somewhat radical reimagining of how drops work, but (1) it will look the same most of the time, and (2) it should be able to be made to work right for the cases in which drops merge together with neighboring drops to make weirdly shaped chains (like the ones Corey is seeing). What I'm going to describe here only deals with fluid on the pads proper, but it can probably be made to work for fluid within the wells as well. The nice thing about having fluid motion inferred is that the model can change later without having to fix it in lots of places.

The big change is that Drops will no longer contain liquid themselves. Rather, they will indirect through Blobs, which can involve multiple pads. I'm thinking of something like

class Drop:
   blob: Blob
   pad: Pad
   @property def contents(self) -> Liquid:
      return self.blob.contents

class Blob:
   pads: Final[set[Pad]]
   contents: Liquid  # Final?
   _stashed_drops: Final[dict[Pad, Drop]]
   @property def singleton(self) -> bool:
      return len(self.pads) == 1
   @property def drops(self) -> Sequence[Drop]: ...
   @property def neighbors(self) -> set[Pad]: ...
   # Blobs also probably delegate reservation to their pads, so they all get reserved and unreserved together.

After fix-up:

• every blob will be either pinned or unpinned. It is pinned if all of the pads it contains are on, and it is unpinned if all of them are off.
  • That is, after fix-up, no blob will have a mix of pinned and unpinned pads.
• no two blobs will include neighboring pads.
  • That is, there will always be a buffer of at least one pad between blobs

The fix-up process will look something like this:

1. First, deal with liquid that's entered or left the board.
   1. If a volume has been removed from an extraction point, take it from the blob of the drop at its pad (which mus exist). If this exhausts the blob, remove the blob and all of its drops.
      • Yes, this does set up a race condition with other blobs that might be merging in. I'll note that and deal with it later.
   2. If a liquid has been added via an extraction point,
      1. If there is a drop at the extraction point's pad, mix the liquid into the drop's blob.
      2. Otherwise, create a new drop in a singleton blob at that pad.
   3. If a well has liquid on its gate, do the same (as extraction point delivery) with respect to the well's exit pad.
      • If the gate is off, remove the "have liquid" flag, otherwise leave it. 2, Partition the blobs on the board into pinned and unpinned.
   4. For any blob that contains a mixture of on and off pads, split the blob into connected components that are all on (pinned) and all off (unpinned).
   5. Partition the blob's stashed drops among the new blobs, biasing toward the closest blob, preferring unpinned.
2. For each on neighbor of each unpinned blob:
   1. If the pad has a drop, merge that drop's (necessarily different) blob into this blob:
      1. Mix the two liquids together.
      2. Combine the pad sets and stashed sets
      3. Update all of the other blob's drops to point to this blob.
      4. Forget about the other blob.
      5. (Figure out what to do about reservations)
   2. Otherwise, it's empty.
      1. Add the neighbor pad to the blob.
      2. If there's a stashed drop for that pad or, failing that, for a neighbor pad, put it on the board at the pad. Otherwise create a new drop there. In any case, the drop points to this blob.
      3. Add on neighbors of this pad to set of neighbors to consider.
         • Ignore if they have a drop in this blob.
3. For each unpinned blob, compute the set of on neighboring pad
   1. If there are none, there's nothing to do. The unpinned blob will stay floating.
   2. Otherwise, for each on neighbor,
      1. If it has a drop, add the drop's blob to a set.
      2. If not, compute the connected component of its on neighbors (note that this may involve other neighbors to the original blob) and create a blob for it and add it to the set.
   3. Split the blob among the neighbor blobs with each getting a proportion based on the relative size of the neighbor blobs
      • Viktor and I decided that the appropriate measure is to sum 1/(dx^2+dy^2) for all the pads in each component.
   4. For any newly-created blob, move the drops in the original blob to the closest neighbor pad and create new drops for all others.
   5. Add any non-moved drops to the stashed drops in the closest blob.
   6. As a (very common) optimization, if all of the on neighbors form a single new connected component, rather than creating a new blob, just update the current one, modifying the set of pads, moving drops (probably just one), and possibly unstashing drops.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jan 12, 2022 at 2:09 PM PST.

[EvanKirshenbaum]

Thinking more about it, a few changes:

1. Rather than analyzing all blobs, I'm going to work from a journal of changes (pads turned on, pads turned off, liquid added, liquid removed), focusing on just what changed since the last tick.
2. I'm thinking that blobs are associated with pads rather than with drops and that we can have blobs that don't actually have any liquid in them (i.e., blobs of on pads that don't have drops on them).
3. This means that before (and after) motion inference
   1. All on pads are in a blob and all off pads that have drops on them are in a blob
   2. Blobs are maximum extensions of pads with the same condition
   3. Blobs do not abut. That is, there is always an empty off pad between any two pads in blobs.
4. Blobs carry whether (as of the last tick) they are pinned or unpinned.
5. To manage the journal, I can use the state-change callbacks on the pads, maintaining two sets (turned_on and turned_off). If a pad changes state, remove it from the old set (if it's there) or add it to the new one. (Ignore if new and old are the same.)
   • For liquids added and removed, maintain dictionaries of pad to liquid added and pad to volume removed.
6. For the actual flow:
   1. Process liquid added and removed.
      1. If the pad is not in a blob (and necessarily off and empty),
         1. If any neighbors are in unpinned blobs, merge them and add this pad. Otherwise create a new unpinned blob with just this pad.
         2. If any neighbors are in pinned blobs and not just turned off, add the neighbor pad to the list of turned on pads (so that we will process its pull on this blob).
      2. Then add or remove the liquid from the blob.
         • If a removed blob loses all of its content, schedule the pads to be turned off.
   2. Next, for each turned off pad (necessarily in a pinned blob, possibly without liquid)
      1. If all of the pads in the blob are turned off, note the blobas unpinned and remove all pads from the turned off set, go to the next.
         • Or leave the other pads, but ignore a turned off pad if it's in an unpinned set.
      2. Otherwise partition the blob into pinned and unpinned components (we may be able to do this most easily by putting each on pad in its own blob, putting them all in the turned on set, and letting them coalesce again). Any on pad next to one turned off in the blob should get added to the turned on set.
   3. For each turned on pad (which may or may not be in a blob, pinned or unpinned);
      1. If it's in an unpinned blob, we'll have to do a similar sort of partitioning of the blob.
      2. For each neighbor in a pinned blob, merge the pinned blobs and make sure this pad is in the merger. This can be done one by one. (Neighbors in the same blob, of course, require no merger.)
      3. For each neighbor in an unpinned blob, note that this blob pulls on that neighbor pad.
   4. Finally, for each unpinned blob that's being pulled on
      1. Compute the pull from each pulling pinned blob (based on iterating over contents and taking the smallest distance to each pulling point) and derive the relative pull from each pulling blob.
      2. Add that fraction of the unpinned blob's content to the pinned blob, creating drops if necessary (and pulling them from stash if possible)
      3. Remove the drops from the unpinned blob (stashing them in the nearest pulling blob) and the unpinned blob itself.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jan 13, 2022 at 5:36 PM PST.

[EvanKirshenbaum]

Journal based blobs appear to work, although I haven't tried actually running one of the protocols. (I've just been testing with mouse clicks and macro commands.) I haven't really integrated wells yet (a bit of code in the dispense sequence). What I'm going to try now is to refactor things a bit to abstract out the part of Pad that holds blobs and drops and has neighbors and location as DropLoc and extend it to WellPad. Then I'll make blobs hold droplocs rather than pads. This will allow me to model motion in the well, as well. (I don't think I'm going to actually put drop monitors on the display for well pads, at least not for interior pads.)

The problem I see is that my current pull logic says that what gets pulled off an unpinned blob is a fraction of the volume of the blob, and this won't work for dispensing. So what I think I'm going to do is to say arbitrarily that

• When a gate pulls on an off well pad, it arbitrarily gets the dispensed volume (unless that's more than is in the well, not counting what's on the gate).
• When an interior well pad is turned on and creates a new blob, that blob gets content (I really don't care how much) of the reagent in the well. The important thing is that if I turn off a blob inside a well, it should disappear if there's no content in the well, but stick around if there is.

• Migrated from internal repository. Originally created by @EvanKirshenbaum on Jan 23, 2022 at 11:20 AM PST.

[EvanKirshenbaum]

The combinatorial synthesis example protocol now works with inferred drop motion, so I'm going to declare this done.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Feb 21, 2022 at 12:28 PM PST.