thekingofkings
commented
6 years ago Guess 2: the naive method stops early
There is another stopping criteria, which is the following: https://github.com/thekingofkings/chicago-partition/blob/64a2e5389082a6018362b8e4a3ebcd654b80e5a5/MCMC.py#L205
Namely, for Naive MCMC and Softmax MCMC, if we cannot find a better solution after M=100 times rejections, the search stops.
Evidence: I notice that for Naive MCMC, the following codes are not executed (no print screen info) for some rounds.
if isConvergent(epsilon,f_series):
# when mae converges
print "converge in {} samples with {} acceptances \
sample conversion rate {}".format(iter_cnt, len(mae_series),
len(mae_series) / float(iter_cnt))
CommunityArea.visualizeCAs(iter_cnt=None,fname=project_name+"-CAs-iter-final.png")
CommunityArea.visualizePopDist(iter_cnt=None,fname=project_name+'-pop-distribution-final')
break
Issue: Naive MCMC outperforms other methods for house prediction.
Intuitively, the Naive MCMC should not outperform other methods. What causes the counter intuitive observations?
Guess 1: population variance term.
The penalty term dominates the optimization objective. Temporary solution: predict crime alone and ignores any penalty.
The following are the results of only minimizing the house price prediction error, namely set the weight for penalty term as 0. It is clear that the naive method still gives better prediction error.