search

msmobility / silo

SILO Model Java Code
GNU General Public License v2.0
18 stars 27 forks source link

SiloMucTest doesn't consistently pass #360

Open jakobrehmann opened 1 year ago

jakobrehmann commented 1 year ago

I sometimes get assertion errors when I run SiloMucTest, e.g.: 

junitx.framework.ComparisonFailure: dwellings are different. Line [3] expected:<...,834,...> but was:<...,831,...> ['1']

I seems like, at least on my local machine, that SILO is not quite deterministic: if run multiple times, the same test potentially produces different results. At first I thought this had to do with multi-threading. However, even when I added the option number.of.threads = 1 to ./test/muc/siloMucTest.properties, the test still failed occasionally.

paulheinr commented 5 months ago

Our analysis so far is:

The difference in the output data shows, that dwelling prices differ. Prices are updated in de.tum.bgu.msm.models.realEstate.pricing.PricingModelImpl#updateRealEstatePrices. The prices rely on the vacancy rate of the respective region. Whether a dwelling is vacant or not is determined by de.tum.bgu.msm.models.relocation.moves.MovesModelImpl#searchForNewDwelling in the move model.

Those two lines seem to be problematic: 

double util = housingStrategy.calculateHousingUtility(household, dwelling);
double prob = housingStrategy.calculateSelectDwellingProbability(util);

If we set the probability to a constant (e.g. 0.1), tests didn't fail. So something at the calculation of housing utility seems to be non deterministic.