getting error in assaytools parser.py when running quickmodel

[grundye]

@jchodera @steven-albanese

I'm attempting to run the single well data from Friday through assaytools. Mehtap was able to show me how to use the updated version of inputs_single_well.py in the singlemodel4quickwell branch of assaytools. I'm inputting into the command line: quickmodel --inputs 'inputs_single_well'

But I'm getting the following error: Traceback (most recent call last): File "/anaconda3/bin/quickmodel", line 11, in <module> load_entry_point('assaytools==0.3.0', 'console_scripts', 'quickmodel')() File "/anaconda3/lib/python3.6/site-packages/assaytools/scripts/quickmodel.py", line 265, in entry_point quick_model(inputs, nsamples=args.nsamples, nthin=args.nthin) File "/anaconda3/lib/python3.6/site-packages/assaytools/scripts/quickmodel.py", line 37, in quick_model [complex_fluorescence, ligand_fluorescence] = parser.get_data_using_inputs(inputs) File "/anaconda3/lib/python3.6/site-packages/assaytools/parser.py", line 90, in get_data_using_inputs name = "%s-%s-%s%s"%(protein,inputs['ligand_order'][int(i/2)],protein_row,buffer_row) IndexError: list index out of range

I believe I'm using the correct version of assaytools (0.3.1), which I installed using pip after downloading the assaytools master branch. It's totally possible I'm doing something wrong. I think the .json file generated from the new inputs_single_well.py is correct, in any case here's a snippet of the changes I made in order to analyze the data

[grundye]

Oops, forgot to attach the xml files

infinite_results.zip

[MehtapIsik]

The problem seems to be about selection of protein and buffer wells. Quickmodel works if you make a selection of wells as below:

• one type of protein and one type of ligand
• one type of protein with 3 type of ligands (or 3 replicates of the same ligand)
• 2 type of proteins with 2 type of ligands (or 2 replicates of the same ligand)

However, it doesn't work if we select:

• 2 type of proteins with 3 type of ligands ( or 3 replicates of the same ligand)

I don't understand why.

This is how I installed Assaytools to a new Python 3.6 environment and run quickmodel.

Clone Assaytools repo from GitHub. 
Switch to `singlewell4quickmodel` branch (commit: 0465bb2) and install:
$ cd ../Assaytools
$ git checkout singlewell4quickmodel
$ conda install numpy
$ python setup.py install
$ conda install lxml
$ conda install pymc
$ conda install seaborn
$ conda install pymbar
$ quickmodel --inputs 'inputs_single_well' --type 'spectra' --nsamples 100

@grundye I think for now you can analyse the data by making 3 well selections (1 protein, 3 replicates of 1 ligand) at a time. This is the inputs file that works: inputs_single_well_1prot_3replicates.py.zip $ quickmodel --inputs 'inputs_single_well_1prot_3replicates' --type 'spectra' --nsamples 100

I used 100 samples here but when running on the cluster you can increase the samples to 20000.

@sonyahanson Would you be able to look into what is the problem with parsing? I used the xml files Erin posted above. I don't understand why quickmodel works analysing a series of 2 protein with 2 ligands (inputs_single_well_2replicates.py.zip) works but not 2 protein with 3 ligands (inputs_single_well_3replicates.py.zip)

This is the error message I get:

$ quickmodel --inputs 'inputs_single_well_3replicates' --type 'spectra' --nsamples 100
bad selection
Traceback (most recent call last):
  File "/Users/isikm/opt/anaconda/envs/py36_assaytools_singlewell4quickmodel_0465bb2/bin/quickmodel", line 11, in <module>
    load_entry_point('assaytools==0.3.1', 'console_scripts', 'quickmodel')()
  File "/Users/isikm/opt/anaconda/envs/py36_assaytools_singlewell4quickmodel_0465bb2/lib/python3.6/site-packages/assaytools-0.3.1-py3.6.egg/assaytools/scripts/quickmodel.py", line 277, in entry_point
    quick_model(inputs, nsamples=args.nsamples, nthin=args.nthin)
  File "/Users/isikm/opt/anaconda/envs/py36_assaytools_singlewell4quickmodel_0465bb2/lib/python3.6/site-packages/assaytools-0.3.1-py3.6.egg/assaytools/scripts/quickmodel.py", line 38, in quick_model
    [complex_fluorescence, ligand_fluorescence] = parser.get_data_using_inputs(inputs)  
  File "/Users/isikm/opt/anaconda/envs/py36_assaytools_singlewell4quickmodel_0465bb2/lib/python3.6/site-packages/assaytools-0.3.1-py3.6.egg/assaytools/parser.py", line 134, in get_data_using_inputs
    wavelength = '%s' %inputs['wavelength'])[protein_well]))
  File "/Users/isikm/opt/anaconda/envs/py36_assaytools_singlewell4quickmodel_0465bb2/lib/python3.6/site-packages/assaytools-0.3.1-py3.6.egg/assaytools/platereader.py", line 168, in select_data
    if datatypes[0] == dict and wavelength != None:
IndexError: list index out of range

[grundye]

Thanks that works for most of my wells. But if I put into the input file a well that's 'X10', 'X11', 'X12' I get the following error"

Traceback (most recent call last): File "quickmodel.py", line 280, in <module> entry_point() File "quickmodel.py", line 277, in entry_point quick_model(inputs, nsamples=args.nsamples, nthin=args.nthin) File "quickmodel.py", line 38, in quick_model [complex_fluorescence, ligand_fluorescence] = parser.get_data_using_inputs(inputs) File "/anaconda3/envs/assaytools/lib/python3.6/site-packages/assaytools-0.3.1-py3.6.egg/assaytools/parser.py", line 134, in get_data_using_inputs wavelength = '%s' %inputs['wavelength'])[protein_well])) File "/anaconda3/envs/assaytools/lib/python3.6/site-packages/assaytools-0.3.1-py3.6.egg/assaytools/platereader.py", line 168, in select_data if datatypes[0] == dict and wavelength != None: IndexError: list index out of range

which appears to be the same (?) as the one Mehtap got above

[steven-albanese]

I've been able to reproduce the error @grundye is seeing as well!

[sonyahanson]

I think you will need 6 Bosutinibs, since you have six wells to analyze. e.g. 'ligand_order' : ['Bosutinib','Bosutinib','Bosutinib','Bosutinib','Bosutinib','Bosutinib']

If that doesn't fix it, I can look into it more carefully.

[sonyahanson]

@grundye were you able to see if this fixed your problem?

[grundye]

@sonyahanson

I just tried to redo the input file. I used the wells for bosutinib with Src GK mutant, which were B10, B11, and B12 with the buffer wells A2, A3, and A4. I inputted 'ligand_order' : ['Bosutinib','Bosutinib','Bosutinib','Bosutinib','Bosutinib','Bosutinib'] but I got an error that the shapes are incompatible.

$ python quickmodel.py --inputs 'inputs_single_well'
Src GK Mutant-Bosutinib-AB
Traceback (most recent call last):
  File "quickmodel.py", line 280, in <module>
    entry_point()
  File "quickmodel.py", line 275, in entry_point
    quick_model(inputs, nsamples=args.nsamples, nthin=args.nthin)
  File "quickmodel.py", line 66, in quick_model
    assay_volume=inputs['assay_volume'], DG_prior='uniform')
  File "/anaconda3/lib/python3.6/site-packages/assaytools/pymcmodels.py", line 492, in make_model
    size=[N], observed=True, value=top_complex_fluorescence) # observed data
  File "/anaconda3/lib/python3.6/site-packages/assaytools/pymcmodels.py", line 103, in LogNormalWrapper
    stochastic = pymc.Normal(log_prefix + name, mu=mu, tau=tau, size=size, observed=observed, value=np.log(value))
  File "/anaconda3/lib/python3.6/site-packages/pymc/distributions.py", line 288, in __init__
    shape_error()
  File "/anaconda3/lib/python3.6/site-packages/pymc/distributions.py", line 285, in shape_error
    (shape, init_val_shape, parents_shape))
ValueError: Shapes are incompatible: value (17,), largest parent (12,), shape argument (17,)

:(

[sonyahanson]

So I think my original suggestion was not very well thought-out, but I think I figured it out, now.

Thanks @grundye and @MehtapIsik for your detective work on this, it seems Erin's point that wells with 'X10', 'X11', 'X12' result in an error was dead on.

I think I figured out what was happening and it was super dumb, hopefully my latest commit (https://github.com/choderalab/assaytools/pull/133/commits/833405c1976a374792f4bc12d904f3ae6ce6d846) fixed it!

[sonyahanson]

Closing!