search

jgrss / mpglue

A Python library for image and vector processing
MIT License
8 stars 3 forks source link

Error in applying classification #7

Open bpstewar opened 5 years ago

bpstewar commented 5 years ago

I am encountering the following error when running classify, any suggestions?

(spfeasenv) root@90c5febdf5dd:/mnt/data# classify -i spfeas/103001007FA97400_stackedRGBNDSV.vrt -o spfeas/103001007FA97400_stackedRGBNDSV_classified.tif -s spfeas/103001007FA97400_sampleData/trainingData_points__103001007FA97400_stackedRGBNDSV_SAMPLES.txt --classifier-info "{'classifier': 'RF'}"
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
15:12:25:INFO:146:classify.main:
Start date & time --- (Tue Nov  6 15:12:25 2018)

15:12:29:INFO:4795:classification._train_model:  Training a RF model with 120,374 samples and 153 variables ...
15:12:33:INFO:5312:classification.predict:  Predicting class labels from spfeas/103001007FA97400_stackedRGBNDSV.vrt and writing to spfeas/103001007FA97400_stackedRGBNDSV_classified.tif ...
15:12:34:INFO:5561:classification._predict:  Block 1 of 77 ...
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/pysal/__init__.py:65: VisibleDeprecationWarning: PySAL's API will be changed on 2018-12-31. The last release made with this API is version 1.14.4. A preview of the next API version is provided in the pysal 2.0 prelease candidate. The API changes and a guide on how to change imports is provided at https://migrating.pysal.org
  ), VisibleDeprecationWarning)
Traceback (most recent call last):
  File "/opt/conda/envs/spfeasenv/bin/classify", line 11, in <module>
    load_entry_point('MpGlue==0.2.4', 'console_scripts', 'classify')()
  File "/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/MpGlue-0.2.4-py2.7-linux-x86_64.egg/mpglue/classification/classify.py", line 194, in main
    n_jobs_vars=args.n_jobs_vars)
  File "/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/MpGlue-0.2.4-py2.7-linux-x86_64.egg/mpglue/classification/classify.py", line 33, in predict
    self.cl.predict(input_image, output_image, **kwargs)
  File "/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/MpGlue-0.2.4-py2.7-linux-x86_64.egg/mpglue/classification/classification.py", line 5372, in predict
    self._predict()
  File "/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/MpGlue-0.2.4-py2.7-linux-x86_64.egg/mpglue/classification/classification.py", line 5945, in _predict
    out_raster_object.write_array(np_dtype(mdl.predict(features).reshape(n_rows,
  File "/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/sklearn/ensemble/forest.py", line 538, in predict
    proba = self.predict_proba(X)
  File "/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/sklearn/ensemble/forest.py", line 578, in predict_proba
    X = self._validate_X_predict(X)
  File "/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/sklearn/ensemble/forest.py", line 357, in _validate_X_predict
    return self.estimators_[0]._validate_X_predict(X, check_input=True)
  File "/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/sklearn/tree/tree.py", line 373, in _validate_X_predict
    X = check_array(X, dtype=DTYPE, accept_sparse="csr")
  File "/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/sklearn/utils/validation.py", line 453, in check_array
    _assert_all_finite(array)
  File "/opt/conda/envs/spfeasenv/lib/python2.7/site-packages/sklearn/utils/validation.py", line 44, in _assert_all_finite
    " or a value too large for %r." % X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float32').
jgrss commented 5 years ago

The error stems from Scikit-learn not liking your input image data (must be some missing data). I added a check for nans and infs in mpglue.classification. Pull the latest version to implement the changes.

bpstewar commented 5 years ago

The issue is with some of the edge images - somehow I calculated spfeas for areas outside of the image, so some of my spfeas tiles have consistent 0s throughout the stack. I can now run a classification against the good tiles, but I am losing my edge tiles. Is it possible to apply a mask to the image for classification? I couldn't quite figure it out through the classify -h prompt.

jgrss commented 5 years ago

You can currently use --mask-background, which accepts an image name or an array. Any zero in the mask is set as zero in the predictions. The drawback of this is that it first makes predictions on all pixels.

I am open to any enhancement suggestions. Maybe what you want is to not predict on pixels under a mask?