Closed tolandwehr closed 3 years ago
amahendrakar
commented
3 years ago @tolandwehr, I was able to run the given code snippet without any issues, please find the gist of it here.
In order to expedite the trouble-shooting process, could you please provide the complete code to reproduce the issue reported here. Thanks!
tolandwehr
commented
3 years ago Okay. Gist is a little short, as it misses the
cross_val_predict, where the error happens. My description might have left this unclear, sry. Here the notebook, which is a little messy, though, as it also involves other tasks:
import tensorflow as tf
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = '0' # Set to -1 if CPU should be used CPU = -1 , GPU = 0
gpus = tf.config.experimental.list_physical_devices('GPU')
cpus = tf.config.experimental.list_physical_devices('CPU')
if gpus:
try:
# Currently, memory growth needs to be the same across GPUs
for gpu in gpus:
tf.config.experimental.set_memory_growth(gpu, True)
logical_gpus = tf.config.experimental.list_logical_devices('GPU')
print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs")
except RuntimeError as e:
# Memory growth must be set before GPUs have been initialized
print(e)
elif cpus:
try:
# Currently, memory growth needs to be the same across GPUs
logical_cpus= tf.config.experimental.list_logical_devices('CPU')
print(len(cpus), "Physical CPU,", len(logical_cpus), "Logical CPU")
except RuntimeError as e:
# Memory growth must be set before GPUs have been initialized
print(e)
#from __future__ import print_function, division
import plotly.express as px
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import keras
import sys
import os
import tsaug
from tsaug.visualization import plot
from tsaug import TimeWarp, Crop, Quantize, Drift, Reverse, Convolve, AddNoise, Dropout, Pool, Resize
import statsmodels
import datawig
import impyute
import missingpy
from missingpy import KNNImputer,MissForest
from impyute.imputation.cs import mice
from datawig import SimpleImputer
from statsmodels import robust
from operator import itemgetter,attrgetter
from functools import partial
from scipy import stats
from pylab import rcParams
from tpot import TPOTRegressor
from sklearn import preprocessing
from sklearn.decomposition import PCA
from sklearn import model_selection
from sklearn.preprocessing import StandardScaler, MinMaxScaler, Normalizer, LabelEncoder, RobustScaler, QuantileTransformer
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split,cross_val_score, cross_val_predict, cross_validate, GridSearchCV, RandomizedSearchCV, TimeSeriesSplit, KFold
from sklearn.metrics import mean_squared_error, r2_score, explained_variance_score, make_scorer,median_absolute_error, mean_absolute_error,max_error,explained_variance_score
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.utils.validation import check_array, check_is_fitted
from sklearn.experimental import enable_iterative_imputer # noqa
from sklearn.impute import IterativeImputer, SimpleImputer
from tensorflow.keras.wrappers.scikit_learn import KerasClassifier, KerasRegressor
from kerastuner import HyperModel
# import tensorflow as tf
# import cProfile
# tf.enable_eager_execution()
# tf.executing_eagerly()
from tensorflow.python.keras.layers import InputLayer, TimeDistributed, Lambda, Dense, Dot, Reshape,Concatenate, Embedding, Activation, Conv1D, Conv2D, Cropping2D, MaxPooling2D, Flatten, Dropout, LSTM, GRU, Bidirectional, Input, LeakyReLU,Conv2DTranspose, ZeroPadding2D, ZeroPadding1D, UpSampling2D, UpSampling1D,multiply,AveragePooling1D # components of network
from tensorflow.python.keras.models import Model, Sequential # type of model
from tensorflow.python.keras.layers import BatchNormalization
from tensorflow.python.keras.optimizers import Adam, RMSprop, SGD, Nadam, Adadelta, Adamax
from tensorflow.python.keras.regularizers import l2
from tensorflow.python.keras.callbacks import EarlyStopping, ModelCheckpoint, TensorBoard
#from SpectralNormalizationKeras import DenseSN, ConvSN2D
from ctgan import CTGANSynthesizer
#import tensorflow.keras.backend as K
from tensorflow.python.keras.backend import expand_dims, squeeze
from tqdm import tqdm
######loading and preparing data
Heisei_T=pd.read_excel('Heisei_Whoa.xlsx')
Heisei_T=Heisei_T.drop(Heisei_T.columns[0], axis=1)
Year_Frame = 18
Set_Len = len(Heisei_T)
Sample_Len = int(Set_Len/Year_Frame)
Set_Width = len(Heisei_T.columns)
Heisei_Heads=Heisei_T.columns
Heisei_TR=np.array(Heisei_T).reshape(Sample_Len,Year_Frame,Set_Width)
np.random.shuffle(Heisei_TR)
Heisei_T=Heisei_TR.reshape(Sample_Len*Year_Frame,Set_Width)
def split_sequences(sequences, n_steps):
X, y = list(), list()
for i in range(len(sequences)):
# find the end of this pattern
end_ix = i + n_steps
# check if we are beyond the dataset
if end_ix > len(sequences):
break
# gather input and output parts of the pattern
seq_x, seq_y = sequences[i:end_ix, :-1], sequences[end_ix-1, -1]
X.append(seq_x)
y.append(seq_y)
return np.array(X), np.array(y)
n_steps = 3
X_full=[]
y_full=[]
for x in range(0,Sample_Len):
X,y=split_sequences(Heisei_T[x*Year_Frame:(x+1)*Year_Frame,:],n_steps)
X_full.append(X)
y_full.append(y)
X_full=np.array(X_full)
y_full=np.array(y_full)
X_full.shape
X_full=X_full.reshape(Sample_Len*16,n_steps,Set_Width-1)
y_full=y_full.reshape(Sample_Len*16,1)
#pd.DataFrame(y_full).head(90)
X_full.shape
#y_full.shape
print(pd.DataFrame(y_full).max())
y_test=np.array(pd.DataFrame(y_full).tail(round(0.10001671123*y_full.shape[0])))
y_train=np.array(pd.DataFrame(y_full).head(round((1-0.10001671123)*y_full.shape[0])))
X_test=np.array(pd.DataFrame(X_full.reshape(X_full.shape[0],X_full.shape[1]*X_full.shape[2])).tail(round(0.10001671123*X_full.shape[0]))).reshape(round(0.10001671123*X_full.shape[0]),X_full.shape[1],X_full.shape[2])
X_train=np.array(pd.DataFrame(X_full.reshape(X_full.shape[0],X_full.shape[1]*X_full.shape[2])).head(round(1-0.10001671123*X_full.shape[0]-1))).reshape(round((1-0.10001671123)*X_full.shape[0]),X_full.shape[1],X_full.shape[2])
Train_Len=len(X_train)
Test_Len=len(X_test)
print(y_train.shape,y_test.shape,X_train.shape,X_test.shape)
X_test_scaler = preprocessing.StandardScaler()
y_test_scaler = preprocessing.StandardScaler()
X_train_scaler = preprocessing.StandardScaler()
y_train_scaler = preprocessing.StandardScaler()
y_test=y_test_scaler.fit_transform(y_test)
y_train=y_train_scaler.fit_transform(y_train)
X_test=X_test_scaler.fit_transform(X_test.reshape(Test_Len*n_steps,Set_Width-1))
X_train=X_train_scaler.fit_transform(X_train.reshape(Train_Len*n_steps,Set_Width-1))
X_test=X_test.reshape(Test_Len,n_steps,Set_Width-1)
X_train=X_train.reshape(Train_Len,n_steps,Set_Width-1)
#########Model and SKlearn Cross_Val_Predict
def twds_model(layer1=32, layer2=32, layer3=16, dropout_rate=0.5, optimizer='Adam'
, learning_rate=0.001, activation='relu', loss='mse'):#, n_jobs=1):layer3=80,
model = Sequential()
model.add(Bidirectional(GRU(layer1, return_sequences=True),input_shape=(X_train.shape[1],X_train.shape[2])))
model.add(AveragePooling1D(2))
model.add(Conv1D(layer2, 3, activation=activation, padding='same',
name='extractor'))
model.add(Flatten())
model.add(Dense(layer3,activation=activation))
model.add(Dropout(dropout_rate))
model.add(Dense(1))
model.compile(optimizer=optimizer,loss=loss)
return model
twds_model=twds_model()
print(twds_model.summary())
def CustomVarious(y_true, y_pred):
y_true=y_true.reshape(len(y_true[:,1])*Heisei_TR.shape[1],)
if np.isnan(y_pred).any():
result=-1000000
MAD= 1000000
else:
y_pred=y_pred.reshape(len(y_pred[:,1])*Heisei_TR.shape[1],)
MAD=median_absolute_error(y_true, y_pred)
print(MAD)
return MAD
scorer = make_scorer(CustomVarious, greater_is_better=False
model_twds=KerasRegressor(build_fn=twds_Model, batch_size=256,epochs=6)
############# PLACE OF THE ERROR ############
twds_Pred=cross_val_predict(model_twds,
X_train,
y_train,
n_jobs=1,
cv=4,
verbose=2)
@tolandwehr,
On running the code I am facing an error stating FileNotFoundError: [Errno 2] No such file or directory: 'Heisei_Whoa.xlsx'. Could you please provide all the necessary files to run the code?
Also, could you please remove the dependencies and get the example down to the simplest possible repro? That will allow us to easily debug the issue. Thanks!
tolandwehr
commented
3 years ago @amahendrakar
Whoa contains data that I'm not allowed to pass, unfortunately ^^'. But it was checked with
.isnull().sum().sum()to be free on NaNs.
You can dense the code down to:
import tensorflow as tf
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = '0' # Set to -1 if CPU should be used CPU = -1 , GPU = 0
gpus = tf.config.experimental.list_physical_devices('GPU')
cpus = tf.config.experimental.list_physical_devices('CPU')
if gpus:
try:
# Currently, memory growth needs to be the same across GPUs
for gpu in gpus:
tf.config.experimental.set_memory_growth(gpu, True)
logical_gpus = tf.config.experimental.list_logical_devices('GPU')
print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs")
except RuntimeError as e:
# Memory growth must be set before GPUs have been initialized
print(e)
elif cpus:
try:
# Currently, memory growth needs to be the same across GPUs
logical_cpus= tf.config.experimental.list_logical_devices('CPU')
print(len(cpus), "Physical CPU,", len(logical_cpus), "Logical CPU")
except RuntimeError as e:
# Memory growth must be set before GPUs have been initialized
print(e)
#from __future__ import print_function, division
import plotly.express as px
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import keras
import sys
import os
import tsaug
from tsaug.visualization import plot
from tsaug import TimeWarp, Crop, Quantize, Drift, Reverse, Convolve, AddNoise, Dropout, Pool, Resize
import statsmodels
import datawig
import impyute
import missingpy
from missingpy import KNNImputer,MissForest
from impyute.imputation.cs import mice
from datawig import SimpleImputer
from statsmodels import robust
from operator import itemgetter,attrgetter
from functools import partial
from scipy import stats
from pylab import rcParams
from tpot import TPOTRegressor
from sklearn import preprocessing
from sklearn.decomposition import PCA
from sklearn import model_selection
from sklearn.preprocessing import StandardScaler, MinMaxScaler, Normalizer, LabelEncoder, RobustScaler, QuantileTransformer
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split,cross_val_score, cross_val_predict, cross_validate, GridSearchCV, RandomizedSearchCV, TimeSeriesSplit, KFold
from sklearn.metrics import mean_squared_error, r2_score, explained_variance_score, make_scorer,median_absolute_error, mean_absolute_error,max_error,explained_variance_score
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.utils.validation import check_array, check_is_fitted
from sklearn.experimental import enable_iterative_imputer # noqa
from sklearn.impute import IterativeImputer, SimpleImputer
from tensorflow.keras.wrappers.scikit_learn import KerasClassifier, KerasRegressor
from kerastuner import HyperModel
from tensorflow.python.keras.layers import InputLayer, TimeDistributed, Lambda, Dense, Dot, Reshape,Concatenate, Embedding, Activation, Conv1D, Conv2D, Cropping2D, MaxPooling2D, Flatten, Dropout, LSTM, GRU, Bidirectional, Input, LeakyReLU,Conv2DTranspose, ZeroPadding2D, ZeroPadding1D, UpSampling2D, UpSampling1D,multiply,AveragePooling1D # components of network
from tensorflow.python.keras.models import Model, Sequential # type of model
from tensorflow.python.keras.layers import BatchNormalization
from tensorflow.python.keras.optimizers import Adam, RMSprop, SGD, Nadam, Adadelta, Adamax
from tensorflow.python.keras.regularizers import l2
from tensorflow.python.keras.callbacks import EarlyStopping, ModelCheckpoint, TensorBoard
from ctgan import CTGANSynthesizer
#import tensorflow.keras.backend as K
from tensorflow.python.keras.backend import expand_dims, squeeze
from tqdm import tqdm
X_full = np.random.rand(1200,18,15)
y_full = np.random.rand(1200,18 )
y_test=np.array(pd.DataFrame(y_full).tail(round(0.10001671123*y_full.shape[0])))
y_train=np.array(pd.DataFrame(y_full).head(round((1-0.10001671123)*y_full.shape[0])))
X_test=np.array(pd.DataFrame(X_full.reshape(X_full.shape[0],X_full.shape[1]*X_full.shape[2])).tail(round(0.10001671123*X_full.shape[0]))).reshape(round(0.10001671123*X_full.shape[0]),X_full.shape[1],X_full.shape[2])
X_train=np.array(pd.DataFrame(X_full.reshape(X_full.shape[0],X_full.shape[1]*X_full.shape[2])).head(round(1-0.10001671123*X_full.shape[0]-1))).reshape(round((1-0.10001671123)*X_full.shape[0]),X_full.shape[1],X_full.shape[2])
Train_Len=len(X_train)
Test_Len=len(X_test)
print(y_train.shape,y_test.shape,X_train.shape,X_test.shape)
X_test_scaler = preprocessing.StandardScaler()
y_test_scaler = preprocessing.StandardScaler()
X_train_scaler = preprocessing.StandardScaler()
y_train_scaler = preprocessing.StandardScaler()
y_test=y_test_scaler.fit_transform(y_test)
y_train=y_train_scaler.fit_transform(y_train)
X_test=X_test_scaler.fit_transform(X_test.reshape(Test_Len*n_steps,Set_Width-1))
X_train=X_train_scaler.fit_transform(X_train.reshape(Train_Len*n_steps,Set_Width-1))
X_test=X_test.reshape(Test_Len,n_steps,Set_Width-1)
X_train=X_train.reshape(Train_Len,n_steps,Set_Width-1)
#########Model and SKlearn Cross_Val_Predict
def twds_model(layer1=32, layer2=32, layer3=16, dropout_rate=0.5, optimizer='Adam'
, learning_rate=0.001, activation='relu', loss='mse'):#, n_jobs=1):layer3=80,
model = Sequential()
model.add(Bidirectional(GRU(layer1, return_sequences=True),input_shape=(X_train.shape[1],X_train.shape[2])))
model.add(AveragePooling1D(2))
model.add(Conv1D(layer2, 3, activation=activation, padding='same',
name='extractor'))
model.add(Flatten())
model.add(Dense(layer3,activation=activation))
model.add(Dropout(dropout_rate))
model.add(Dense(1))
model.compile(optimizer=optimizer,loss=loss)
return model
twds_model=twds_model()
print(twds_model.summary())
def CustomVarious(y_true, y_pred):
y_true=y_true.reshape(len(y_true[:,1])*Heisei_TR.shape[1],)
if np.isnan(y_pred).any():
result=-1000000
MAD= 1000000
else:
y_pred=y_pred.reshape(len(y_pred[:,1])*Heisei_TR.shape[1],)
MAD=median_absolute_error(y_true, y_pred)
print(MAD)
return MAD
scorer = make_scorer(CustomVarious, greater_is_better=False)
model_twds=KerasRegressor(build_fn=twds_Model, batch_size=256,epochs=6)
############# PLACE OF THE ERROR ############
twds_Pred=cross_val_predict(model_twds,
X_train,
y_train,
n_jobs=1,
cv=4,
verbose=2)
Or actually you could leave all the data preshaping and just feed in with
X_train = np.random.rand(1200,18,15)
X_train = np.random.rand(1200,18,1 )@tolandwehr Thank you for the update. I was able to reproduce the issue with TF v2.3.
Whereas on running the code with TF-nightly, I am facing a different error stating ValueError: The first argument to Layer.call must always be passed. Please find the attached gist. Thanks!
amahendrakar
commented
3 years ago --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-603-37b55dfd53fd> in <module> ----> 1 GridLSTM.fit(X_train, y_train) ~\Anaconda3\envs\Tensorflow\lib\site-packages\sklearn\utils\validation.py in inner_f(*args, **kwargs) 70 FutureWarning) 71 kwargs.update({k: arg for k, arg in zip(sig.parameters, args)}) ---> 72 return f(**kwargs) 73 return inner_f 74
@tolandwehr,
Looking at the error log, it seems like the error is thrown from a sklearn\utils\validation.py module and Tensorflow is the name of your environment. Please correct me if I am wrong, thanks!
tolandwehr
commented
3 years ago @amahendrakar
Exactly, the environ was called 'Tensorflow', the module is 'sklearn\utils\validation.py'.
amahendrakar
commented
3 years ago @gowthamkpr,
Running the code with TF v2.3, throws an error stating TypeError: can't pickle _thread.RLock objects.
However with TF-nightly, the error changes to ValueError: The first argument to Layer.call must always be passed.. Please find the attached gist. Thanks!
rmothukuru
commented
3 years ago @tolandwehr, Can you please refer this Article which comprises Using Keras Wrapper in Scikit Learn. Thanks!
rmothukuru
commented
3 years ago @tolandwehr, Can you please respond to the above comment. Thanks!
tolandwehr
commented
3 years ago @rmothukuru Sry, the last weeks are very busy on releasing some topics, I will come back to the issue next week.
rmothukuru
commented
3 years ago @tolandwehr, Can you please respond to the above comment. Thanks!
rmothukuru
commented
3 years ago Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!
google-ml-butler[bot]
commented
3 years ago 
anoldmaninthesea
commented
3 years ago I've just opened this related issue, but now with data freely available. https://github.com/tensorflow/tensorflow/issues/47324
For Neural-Network regression prediction task cross_val_predict from SKlearn throws the error (full error further below, below the model used)
when utilizing it with input data of form
and the following in NN
and
The error was mentioned at SKlearn, but it was contemplated that the error is likely to be on the Tensorflow site or that it might be caused by third party content - however, I have no idea where this content might be involved.
The complete error:
The versions used: