itctools integration with pyrobot

[jhprinz]

I went through the itctools by accident and realized that the functionality for the TecanEVO is very similar to what I did in the pyrobot package. This is especially for Labware.py and Materials.py I think I would make sense if we would integrate this and since itctools seems to be more tested we should build upon the materials.py and labware.py.

For comparison I have used the following scheme: class Mixture: ''' A mixture is a description for a combination of one or more liquids with their respective concentrations '''

this would be equal to SimpleMixture. I have also a way to compute the needed mixture amounts to achieve a target mixture. Like bosutibib 5mM in 100% DMSO, 100%DMSO and Buffer as sources and the target should be 0.5mM Bosutibin in 5% DMSO and you get the amounts of the three parts needed.

For Labware.py I have split this into

class Plate:
'''A plate is a movable location that can contain wells'''

class Well:
'''
A well is a liquid holding site on a plate. It contains specifications about the liquid mixture and its associated position on a plate
'''

class WellSet(list):
'''
A WellSet is a set of wells on one or more plates. Effectively this is a python list of Wells
'''

Most of it is for convenience like you can assign specific wells on a plate using

p = Plate()
p.wells.filter('A,C,E:1-12') # get a list of wells of rows A,C,E
p.wells.filter('A-P-2:1-24-2') # get every second well on a plate

and then you can assign ranges of concentrations, etc. It's mostly to easily build the scheme of a plate.

Lastly there is a class

class Worklist(object):
'''
Worklist generator class to construct a worklist string by using standard commands for EVO worklists
'''

that allows to simplify the worklist generation and

class Task(object):
class Scheduler(object):

that can compute all necessary pipetting steps to create a target plate by only saying what sources you have and what target plates you want. This is not a real optimization, but need to be guided with a few commands, but then it produces quite optimal pipettings.

In essence this is very similar to what you are doing now and the final question is how can be use this to simplify the general making of wordlists for EVO

[jhprinz]

I forgot. I have also running a plate library, which we might use to construct Labware easily with all necessary parameters. Here mainly the working volumes and rows and columns in plates. Might throw an error if the working volume requirements are not met...

[jchodera]

Thanks for noticing this! I agree we should try to standardize.

Perhaps the best approach to this is:

• decide which library to standardize around
• make this conda installable to.make it easy for us to all use. We can have travis-ci push dev versions to binstar.
• add issues to incorporate any functionality that the other library (eg bas's) requires but is currently missing
• implement these through PRs
• slowly adapt all our execution codes to actually use this standard library

If we want other people to be able to use this library too, do we need to qualify the namespace (eg choderalab.tecan.evo?) or are generic names like pyrobot OK?

[bas-rustenburg]

Thanks for browsing through it! We just had a quick discussion around the office. I'm going to try and word what we agreed on, but I think it's basically what John said.

• It would be good to have a standardized library, probably have modules for each part of our automation system. Generic names are probably a bad idea.
• When we need protocols that aren't in the library, add that functionality to the library and keep it consistent.
• Pull requests, and pull requests on your pull requests. People need to be aware what everyone is working on so we don't end up implementing the same code twice.

I think as a first step, it would be great if we could merge pyrobot and the itctools. At least the EVO parts should be one library. We could have an overarching library called choderalab. Within there we could have an automation module with submodules per instrument, to help keep track of classes and functions. We could also have an analysis module, that contain analysis methods (e.g. bayesian itc) .

[jchodera]

WIP pull requests will be essential to keeping everybody synchronized, as Bas makes clear.

Because there are multiple developers involved, the following steps should be the way new features get implemented:

• An new issue is posted on the issue tracker that says "I need feature X for my project Y"
• Someone opens a work-in-progress PR titled "[WIP] Implementing feature X" and posts in the issue thread "I have opened PR #XXX to implement feature X"
• When the implementation is complete, the "[WIP]" is removed from the PR title, and the implementor posts that this is complete and ready to review.
• The PR gets merged, and the issue gets closed.

We should bias ourselves to implementing small features and quick turnaround for PRs, if possible.

If anyone checks in a PR called "Here are 30 features I've implemented on my local branch over the last three weeks, and this breaks everybody else's code", then they will probably get hunted down and beaten senseless. :)

[kyleabeauchamp]

Maybe when the dust settles on this someone (tag team?) can give a group meeting to update us on the usage and status of all this software (Robot, ITC, etc)?

[jchodera]

Maybe when the dust settles on this someone (tag team?) can give a group meeting to update us on the usage and status of all this software (Robot, ITC, etc)?

Good idea!

I've tentatively scheduled Bas + JHP + Sonya to give robots group meeting on Wed 17 Sep.

[bas-rustenburg]

Should we set up a new repository to start gathering this, or work it into the robots repo?

[jchodera]

Might be good to work it into the robots repo, unless it's easiest to start a new repo!

[bas-rustenburg]

I've browsed through the pyrobot code and I think I have some ideas for an initial merge with the itctools. Since I'm currently trying to optimize the ITC work, it is probably good if I take a stab at this first.

@jhprinz pointed out that the labware tools have a lot of overlap. From what I can tell, pyrobot has more functionality when it comes to plate types and specifying wells.

The mixtures/materials side also have overlap, but their structure has some differences. I'm not immediately sure what the best way would be to merge it. The itctools do seem to be more detailed in when it comes to the specification of materials. Pyrobot seems a little bare at first hand, but it does have a very easy to understand structure. Maybe materials.py's mixture class is a little dense. Let's have a discussion about this at some point.

In short here is what I intend to do.

Plan:

1. I will make a new module directory (e.g. choderalab.automation) to which we will slowly start moving code from the other places until everything we need is implemented. Name suggestions are welcome.
2. Integrating pyrobot code into the ITC setup.
   • I will try to remove the specification of wells in itctools.procedures from ITCExperimentSet(and it's child class HeatOfMixingExperimentSet) and use the pyrobot setup. It has more features to it and can probably handle anything that is currently in the ITC setup.
   • I will adjust the current ITC scripts, to make sure they work with this new setup.
     • Plates will be specified using pyrobot (or its new equivalent in choderalab.automation)
     • Wells will still be optimized within the ExperimentSet, but will make use of the pyrobot routines.
3. Once that is complete, we should look at the materials.py / Mixture.py overlap. We will discuss the best way to merge this later, either me or @jhprinz could do this.

Does this sound reasonable?

[jhprinz]

HI...

Thanks Bas for the detailed discussion. I agree that the materials.py side is much better and understandable and we should keep this mostly. Also, of course, the use of simtk.units should be used throughout the project. I had written something that parses strings in a convenient way so that something like '20.0 ul' or '3 nM' would be understood, but with units this is more stable, flexible and established.

In general we should start a new python project in the robots repo maybe

clautomation for ChoderaLabAutomation or clauto or klaatu which sounds similar :)

The pyrobot is still in a very changing way. I tried to clean up all the ideas when merging all my scripts into pyrobot, but it far from being easily usable. For this reason we migh spend a little of time to discuss the basic layout of the classes and their functionality before fixing and merging something.

The easiest way to start would be to using the Worklist class since it is just simplifing the construction of a worklist, nothing else.

The Plate stuff might be worth thinking about a little more. Especially the way Wells are assigned to plates and the other way around. Maybe that can be simplified. I will tried to work out a basic layout today and we could discuss from there.

Plan 1: I fully agree. Start a new module and start adding code. I would mostly keep the itctools stuff though. The pyrobot was actually a first attempt to combine my code from the plate type database, momentum scripting, infinite analysis script and the eve worklist generator.

For the naming I would keep a structure similar to the one I started with: Keep python modules for all instruments/components separate. Have one for labware and one for materials that can be used by all instruments and have maybe a separate module for experiment types that makes use of the instruments/components. Last, have a module for examples/Tutorials and a module for utils/tools which should contain everything that is useful but not experiment or like you also did in itctools, too.

What do you think?

[jhprinz]

The group meeting on Sep 17th is a good idea. We should fix that!