Different return values from local vs. remote (GitHub workflow) runs

[yalinli2]

Hey @yoelcortes , I'm trying to set up the GitHub workflow for QSDsan, but in doing so I found there's one TEA-related test that the remote won't be able to pass, the thing is that the values I get from local runs are different from the remote runs

I looked into it and feel like it's related to BioSTEAM's TEA, so I added a trial method (see here) and found that again, the local results are different from remote ones (I don't think it's because of the package versions, I tried to match the ones on remote), the complete log is here: logs_11.zip

here's the test codes:

        >>> import biosteam as bst
        >>> from biosteam import Chemical, Chemicals, Stream, System
        >>> from biorefineries import lipidcane as lc
        >>> TEA = lc.ConventionalEthanolTEA
        >>> Water = Chemical('Water')
        >>> NaCl = Chemical('NaCl')
        >>> Methanol = Chemical('Methanol')
        >>> Ethanol = Chemical('Ethanol')
        >>> chems = Chemicals((Water, NaCl, Methanol, Ethanol))
        >>> bst.settings.set_thermo((Water, NaCl, Methanol, Ethanol))
        >>> salt_water = Stream('salt_water', Water=2000, NaCl=50, units='kg/hr')
        >>> methanol = Stream('methanol', Methanol=20, units='kg/hr')
        >>> ethanol = Stream('ethanol', Ethanol=10, units='kg/hr')
        >>> M1 = bst.MixTank('M1', ins=(salt_water, 'recycled_brine', methanol, ethanol))
        >>> P1 = bst.Pump('P1', ins=M1-0)
        >>> H1 = bst.HXutility('H1', ins=P1-0, T=350)
        >>> S1 = bst.Splitter('S1', ins=H1-0, split={'Methanol': 1, 'Ethanol': 1})
        >>> M2 = bst.Mixer('M2', ins=S1-0, outs='alcohols')
        >>> S2 = bst.Splitter('S2', ins=S1-1, outs=(1-M1, 'waste_brine'), split=0.2)
        >>> sys = System('sys', path=(M1, P1, H1, S1, M2, S2))
        >>> sys.simulate()
        >>> tea = TEA(system=sys, IRR=0.15, duration=(2018, 2038),
        ...           depreciation='MACRS7', income_tax=0.35,
        ...           operating_days=200, lang_factor=3,
        ...           construction_schedule=(0.4, 0.6),
        ...           WC_over_FCI=0.05, labor_cost=2.5e6,
        ...           fringe_benefits=0.4, property_tax=0.001,
        ...           property_insurance=0.005, supplies=0.20,
        ...           maintenance=0.01, administration=0.005)
        >>> for i in range(10):
        ...     sys.simulate()
        ...     print(tea.NPV)

expected (i.e., local) results:

        -25246652.833673354
        -25248292.333276972
        -25248617.3071441
        -25248682.187191017
        -25248695.15862941
        -25248697.752734397
        -25248698.271548085
        -25248698.37531053
        -25248698.396063007
        -25248698.400213514

remote results:

     -25247164.98735794
    -25248804.495846584
    -25249129.471471302
    -25249194.351868946
    -25249207.323377453
    -25249209.91749646
    -25249210.43631295
    -25249210.54007596
    -25249210.560828548
    -25249210.56497907

the absolute diff between local and remote results are pretty consistent, so probably not a convergence thing, any ideas on why this would be the case? Thanks!!!

[yoelcortes]

Hi Yalin,

I'm getting the same results as the remote results:

-25247164.98735794
-25248804.495846584
-25249129.471471302
-25249194.351868946
-25249207.323377453
-25249209.91749646
-25249210.43631295
-25249210.54007596
-25249210.560828548
-25249210.56497907

Even if I disable numba (I have numba==0.54.0), I also get (almost exactly) the same results:

-25247164.987357937
-25248804.49584659
-25249129.471471302
-25249194.351868954
-25249207.323377457
-25249209.917496465
-25249210.43631295
-25249210.540075958
-25249210.56082855
-25249210.56497907

I believe it may have to do with your chemicals/thermo dependency, I am using the latest from the master branches (and the remote is using the latest PyPI versions). Caleb has recently added Joback estimation methods, new data, and fixed accuracy/reproducibility of results; so maybe it has to do with your local versions.

Let me know if updating works, Thanks!

[yoelcortes]

FYI, if you ever think it is just numba acting up, you can disable numba with:

>>> import os
>>> os.environ["NUMBA_DISABLE_JIT"] = '1' # Before running anything

[yalinli2]

Ah I forgot thermo!!! Thanks for the tips!