pysal / tobler

Spatial interpolation, Dasymetric Mapping, & Change of Support
https://pysal.org/tobler
BSD 3-Clause "New" or "Revised" License
146 stars 30 forks source link

update docs #39

Closed knaaptime closed 4 years ago

knaaptime commented 4 years ago

i'm not getting an errors locally, but have a failure from xgboost in the cloud

tobler/tests/test_area_tables_area_interpolate.py .                      [ 25%]
tobler/tests/test_vectorized_raster_interpolation.py ..F                 [100%]
=================================== FAILURES ===================================
_ VectorizedRasterInterpolation_Tester.test_VectorizedRasterInterpolation_XGBoost _
self = <test_vectorized_raster_interpolation.VectorizedRasterInterpolation_Tester testMethod=test_VectorizedRasterInterpolation_XGBoost>
    def test_VectorizedRasterInterpolation_XGBoost(self):
        local_raster_path = fetch_quilt_path("nlcd_2011")
        nlcd_raster = rasterio.open(local_raster_path)

        s_map = gpd.read_file(libpysal.examples.get_path("sacramentot2.shp"))
        df = s_map[["geometry", "TOT_POP"]]
        df.crs = {"init": "epsg:4326"}

        # Shrinking the spatial extent
        df = df[
            (
                (df.centroid.x < -121.4)
                & (df.centroid.x > -121.5)
                & (df.centroid.y > 38.5)
                & (df.centroid.y < 38.8)
            )
        ]
        df = df.set_geometry("geometry")

>       weights = return_weights_from_xgboost(df, local_raster_path, "TOT_POP")
tobler/tests/test_vectorized_raster_interpolation.py:151: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tobler/area_weighted/vectorized_raster_interpolation.py:409: in return_weights_from_xgboost
    shap_values = explainer.shap_values(X)
../../../miniconda/envs/TEST/lib/python3.6/site-packages/shap/explainers/tree.py:217: in shap_values
    self.assert_additivity(out, model_output_vals)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <shap.explainers.tree.TreeExplainer object at 0x7f50f6a79390>
phi = array([[-5.44528961e+01,  1.48020420e+01, -6.31824402e+02,
         4.01767700e+02],
       [-5.54114609e+01,  1.29951...  2.72606903e+02],
       [ 2.13440582e+02,  5.97311340e+02,  2.57922632e+03,
        -7.69384460e+01]], dtype=float32)
model_output = array([3789.8262 , 3586.1118 , 3867.163  , 3685.5225 , 3495.4353 ,
        565.17303, 4210.1196 , 5690.48   , 7483.479... 4767.215  , 7372.5723 , 4449.741  , 7483.479  ,
       5092.618  , 5114.677  , 5283.8247 , 7372.5723 ], dtype=float32)
    def assert_additivity(self, phi, model_output):
        err_msg = "Additivity check failed in TreeExplainer! Please report this on GitHub."
        if self.feature_dependence != "independent":
            err_msg += " Consider retrying with the feature_dependence='independent' option."
        if type(phi) is list:
            for i in range(len(phi)):
                assert np.max(np.abs(self.expected_value[i] + phi[i].sum(-1) - model_output[:,i])) < 1e-3, err_msg
        else:
>           assert np.max(np.abs(self.expected_value + phi.sum(-1) - model_output)) < 1e-3, err_msg
E           AssertionError: Additivity check failed in TreeExplainer! Please report this on GitHub. Consider retrying with the feature_dependence='independent' option.
../../../miniconda/envs/TEST/lib/python3.6/site-packages/shap/explainers/tree.py:394: AssertionError
----------------------------- Captured stdout call -----------------------------
knaaptime commented 4 years ago

shap appears to be the root cause:

AssertionError: Additivity check failed in TreeExplainer! Please report this on GitHub. Consider retrying with the feature_dependence='independent' option.
knaaptime commented 4 years ago

not totally sure about how to fix... any thoughts @renanxcortes ?

knaaptime commented 4 years ago

this is resolved by pinning to shap>=0.33, which i've included in this PR. The tests will continue to fail until 0.33 is up on conda-forge, but after that this should be good to go

knaaptime commented 4 years ago

this also resolves #38

renanxcortes commented 4 years ago

this is resolved by pinning to shap>=0.33, which i've included in this PR. The tests will continue to fail until 0.33 is up on conda-forge, but after that this should be good to go

Ok! Excellent! Thanks!

knaaptime commented 4 years ago

well i hadn't intended to keep working on this branch but whatever. this now also addresses #34