segrelab / cometspy

Python interface for running COMETS simulations and analyzing the results
GNU General Public License v3.0
11 stars 9 forks source link

Models input order affect metabolites flux and biomass. #41

Open Shlomit840 opened 10 months ago

Shlomit840 commented 10 months ago

When the order of models input to COBRA and then to COMETS changes, the results are changed. cobra.io.read_sbml_model(model_1) cobra.io.read_sbml_model(model_2) has different results from: cobra.io.read_sbml_model(model_2) cobra.io.read_sbml_model(model_1)

Why is that? How can it be eliminated? I use COBRA version 0.22.1 COMETSpy version 0.4.16 Python version 3.9.7

dukovski commented 10 months ago

The order does make difference only if a cell has very low nutrient that will be completely consumed in a single time step. In other words, the first model will take all of the nutrient, so there is none left for the second. We do randomize the order of uptakes, but if your time step size is very high, there will be a difference. Lower the time step and you will lower the effect.

jeremymchacon commented 10 months ago

@dukovski I'm not sure that's the case anymore, I think by default COMETS does simultaneous updating now with the "two-step" process @djbajic and I put together. @Shlomit840 can you share the rest of your script? Also, is there any chance both models have the same id?

Shlomit840 commented 10 months ago

Hi I have a folder with 17 different models, in alphabetical order. The script in the attached file shows how I load them. When we change the models' order, the results are different. Attached is a comparison plot of one compound, using the different orders. it is clearly different. This is true for all compounds.

comparison for_git_file_order.txt

Shlomit840 commented 10 months ago

Can you fix the problem?

dukovski commented 10 months ago

Can you fix the problem?

Yes, let me try to work on it this week and I'll get back to you.

jeremymchacon commented 10 months ago

I don't think there is a bug, per se. I looked through the script and the optimization is using the default "MAXIMIZE_OBJECTIVE", i.e. not pFBA ("MAX_OBJ_MIN_FLUX" or something, I'm away from the repo).

There are many more solutions to the regular objective function (MAX) than pFBA, and changing the model order will change how the seed affects which solution gets chosen. In other words, I believe this is expected behavior. I would repeat with the pFBA variant and see if the number of solutions goes down.

More importantly, as Ilija pointed out, these differences will get smoothed over to a large extent by reducing the time step to a reasonably small size.

On Tue, Sep 19, 2023 at 7:48 AM dukovski @.***> wrote:

Can you fix the problem?

Yes, let me try to work on it this week and I'll get back to you.

— Reply to this email directly, view it on GitHub https://github.com/segrelab/cometspy/issues/41#issuecomment-1725449011, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLGCRINNK4QLCROERJ676TX3GICVANCNFSM6AAAAAA4EL6ERM . You are receiving this because you commented.Message ID: @.***>

--

Jeremy M. Chacón, Ph.D.

he / him / his

Bioinformatics Analyst RI Bioinformatics Minnesota Supercomputing Institute University of Minnesota

jeremymchacon commented 10 months ago

To be just a bit clearer; it is impossible for us to guarantee a single solution to a multi-species problem with model rearrangements between runs, because of the stochastic nature of FBA.

On Tue, Sep 19, 2023 at 8:48 AM Jeremy Chacón @.***> wrote:

I don't think there is a bug, per se. I looked through the script and the optimization is using the default "MAXIMIZE_OBJECTIVE", i.e. not pFBA ("MAX_OBJ_MIN_FLUX" or something, I'm away from the repo).

There are many more solutions to the regular objective function (MAX) than pFBA, and changing the model order will change how the seed affects which solution gets chosen. In other words, I believe this is expected behavior. I would repeat with the pFBA variant and see if the number of solutions goes down.

More importantly, as Ilija pointed out, these differences will get smoothed over to a large extent by reducing the time step to a reasonably small size.

On Tue, Sep 19, 2023 at 7:48 AM dukovski @.***> wrote:

Can you fix the problem?

Yes, let me try to work on it this week and I'll get back to you.

— Reply to this email directly, view it on GitHub https://github.com/segrelab/cometspy/issues/41#issuecomment-1725449011, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLGCRINNK4QLCROERJ676TX3GICVANCNFSM6AAAAAA4EL6ERM . You are receiving this because you commented.Message ID: @.***>

--

Jeremy M. Chacón, Ph.D.

he / him / his

Bioinformatics Analyst RI Bioinformatics Minnesota Supercomputing Institute University of Minnesota

--

Jeremy M. Chacón, Ph.D.

he / him / his

Bioinformatics Analyst RI Bioinformatics Minnesota Supercomputing Institute University of Minnesota