Open omar-sadiqi opened 6 years ago
LSTM with DIST Keras : the problem is with preprocessing the sparkDF for the model : schema of the DF : each line have two matrix : FALSE and TRUE ( autocorrection )
schema = StructType([StructField('TRUE',ArrayType(ArrayType(IntegerType()), True)) , StructField('FALSE',ArrayType(ArrayType(IntegerType()), True))])**
reshapetransformer = ReshapeTransformer("TRUE", "TRUE", (MAX_LEN , LEN)) reshapetransformer = ReshapeTransformer("FALSE", "FALSE_", (MAX_LEN , LEN))
trainer = AEASGD(model, optimizer, loss, metrics=["accuracy"], num_workers=num_workers, batch_size=120,featurescol="FALSE", labelcol="TRUE", num_epoch=2,communication_window=32, rho=5.0, learning_rate=0.01)
weights doesn't get updated HELP
LSTM with DIST Keras : the problem is with preprocessing the sparkDF for the model : schema of the DF : each line have two matrix : FALSE and TRUE ( autocorrection )
schema = StructType([StructField('TRUE',ArrayType(ArrayType(IntegerType()), True)) , StructField('FALSE',ArrayType(ArrayType(IntegerType()), True))])**
reshapetransformer = ReshapeTransformer("TRUE", "TRUE", (MAX_LEN , LEN)) reshapetransformer = ReshapeTransformer("FALSE", "FALSE_", (MAX_LEN , LEN))
trainer = AEASGD(model, optimizer, loss, metrics=["accuracy"], num_workers=num_workers, batch_size=120,featurescol="FALSE", labelcol="TRUE", num_epoch=2,communication_window=32, rho=5.0, learning_rate=0.01)
weights doesn't get updated HELP