search

usc-isi-i2 / dsbox-cleaning

The data cleaning TA1 component of DSBox
MIT License
6 stars 4 forks source link

Pandas warning: A value is trying to be set on a copy of a slice from a DataFrame #36

Closed remram44 closed 6 years ago

remram44 commented 6 years ago

https://github.com/usc-isi-i2/dsbox-cleaning/blob/7b4336fe752cfcbaa06065651cacba32cf05ab15/dsbox/datapreprocessing/cleaner/encoder.py#L282-L283

/.../_venvs/d3m/src/dsbox-datacleaning/dsbox/datapreprocessing/cleaner/encoder.py:283: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  col[col.notnull()] = col[col.notnull()].apply(chg_t)

You should be using .loc, for the left side, otherwise you might not update the data at all.

luofanghao commented 6 years ago

thanks so much for pointing out! it should be solved now: https://github.com/usc-isi-i2/dsbox-cleaning/commit/78b4e520307fe17314838685a4616951d30f1cd2

remram44 commented 6 years ago

See also: https://github.com/usc-isi-i2/dsbox-cleaning/blob/c3b7930f3bc31c09ac7a96e166dba768cd72901a/dsbox/datapreprocessing/cleaner/unary_encoder.py#L229-L236

RqS commented 6 years ago

Solved