search

tyjo / luminate

Denoising longitudinal microbiome data.
7 stars 2 forks source link

compositional_lotka_volterra.py TypeError: 'int' object is not iterable #4

Closed apascualgarcia closed 4 years ago

apascualgarcia commented 4 years ago

Hi @tyjo,

I am trying to estimate LV parameters from the output of python main.py estimate and it gives me first some warnings:

src/compositional_lotka_volterra.py:79: RuntimeWarning: divide by zero encountered in log alr = (np.log(p[:,numer]).T - np.log(p[:,denom])).T Testing regularization parameter set 1 of 375 src/compositional_lotka_volterra.py:461: RuntimeWarning: invalid value encountered in subtract AgB_term1 += np.outer( (xt - xt0) / delT, pgu) src/compositional_lotka_volterra.py:338: RuntimeWarning: invalid value encountered in subtract x_stacked = x[t] - x[t-1] /src/compositional_lotka_volterra.py:342: RuntimeWarning: invalid value encountered in subtract x_stacked = np.vstack((x_stacked, x[t] - x[t-1])) pkg/anaconda3/lib/python3.6/site-packages/scipy/integrate/_ivp/common.py:94: RuntimeWarning: invalid value encountered in true_divide d0 = norm(y0 / scale) pkg/anaconda3/lib/python3.6/site-packages/numpy/core/_methods.py:26: RuntimeWarning: invalid value encountered in reduce return umr_maximum(a, axis, None, out, keepdims)

Then it starts estimating the regularization parameter, and when it finishes it halts with the following error:

... Testing regularization parameter set 375 of 375 Running parameter estimation... Traceback (most recent call last): File "main.py", line 297, in train(Y, U, T, event_names, denom, input_dir, output_dir, otu_table, bootstrap_replicates, use_pseudo_count) File "main.py", line 54, in train clv.train(verbose=False) File "src/compositional_lotka_volterra.py", line 151, in train self.alpha, self.r_A, self.r_g, self.r_B = estimate_elastic_net_regularizers_cv(self.X, self.P, self.U, self.T, self.denom, folds=folds, no_effects=self.no_effects, verbose=verbose) TypeError: 'int' object is not iterable

which sounds like a format error but I do not see any obvious problem. The head of the file is (time runs up to 8.5 every 0.5, three replicates per time point):

id,Beads_Agarose_C,Beads_Agarose_B,Beads_Agarose_A,Beads_Agarose_A,Beads_Agarose_C,Beads_Agarose_B time,0,0,0,0.5,0.5,0.5 OTU_1,0.00105784937096,0.001295563097097,0.000164072970087,0.000660546433594,0.000562045232967,0.000373841350603 OTU_2,0.00237704405768,0.001845613222092,0.000201184874943,0.000455385384357,0.000447590916824,0.000390458910692 OTU_3,0.000459683403489,0.000784831400031,0.000174441566038,0.000111506466749,0.000334358103339,0.000605413257002

Thanks in advance

tyjo commented 4 years ago

It looks like the code is failing on the construction of the additive log-ratio transformation (ALR). I suspect that one of the "denoised" relative abundances from the previous step is being rounded to zero, which would cause the "divide by zero encountered in log" warning. The downstream warnings follow as a result. If you're willing to email your OTU table to me (don't post it here), I will look into the issue. Alternatively, you can bypass the denoising step altogether using the --use-pseudo-counts flag.

I should say that the cLV experiments were from an earlier revision of the manuscript, and were replaced by a different set of experiments for the final version. We haven't thoroughly evaluated how the denoised estimates impact cLV estimates. Nonetheless, I'm happy to chat about this component if it is something you're interested in.

apascualgarcia commented 4 years ago

I will prepare the tables for you, thanks for your help.

tyjo commented 4 years ago

I added a check for zero relative abundances that replaces them with a small value. This should fix the error you're getting.

apascualgarcia commented 4 years ago

It looks it's having an issue to find timeout.py now:

Traceback (most recent call last): File "/main.py", line 10, in from src.compositional_lotka_volterra import CompositionalLotkaVolterra, choose_denom File "src/compositional_lotka_volterra.py", line 6, in from src.timeout import * ModuleNotFoundError: No module named 'src.timeout'

tyjo commented 4 years ago

Yes, this was my fault. I removed this piece of code, but forgot to remove the import. It should be fixed now.

apascualgarcia commented 4 years ago

It is working now, thanks!