search

fchollet / deep-learning-with-python-notebooks

Jupyter notebooks for the code samples of the book "Deep Learning with Python"
MIT License
18.17k stars 8.53k forks source link

6.3 Standardization of validation and test data #100

Closed gmohandas closed 5 years ago

gmohandas commented 5 years ago

I have 2 concerns:

  1. The training data is standardized using the mean and standard deviation of the training set. Shouldn't the validation and test set be standardized using its respective mean and standard deviation as well? I can't see this done anywhere.
  2. Wouldn't it be better to refer to the procedure as standardizing and not normalizing? If nothing else, this will conform to scikit-learns terminology.
gmohandas commented 5 years ago

I withdraw the issue as I understand now that it is standard practice to standardize or normalize the validation and test set with the mean and standard deviation of the training set. This is, so that, the same (inverse) transformations can be applied to the validation and test sets with the prediction results. My bad!