langnico / GRAINet

8 stars 4 forks source link

initialize_weights function #4

Open gzegers opened 1 year ago

gzegers commented 1 year ago

Hi,

I started using GRAINet however I wasn't able to install the versions of the packages stated in the instructions. So I installed the same packages with the updated versions. By doing this I needed to change a few details in the code to make it work with the updated versions of keras and tensorflow. Particularly the function initialize_weight is not working anymore as the "kernel.initializer.run" function is not available anymore.

I create a new function called "initialize_weigh2 (attached)" to solve this problem, and now I am able to run GRAINet with the updated packages. However, it seems that I am getting higher val_loss values (Results figure attached). I think this is associated with the selected method to initialize the weights, however, I could find in the code with method is calling the "kernel.initializer.run".

Does anyone know wich method should work better to initialize the weights?

Thanks,

Gerardo

Fig1

def initialize_weights2(model, layer_name=None): """ Re-initialize the weights before starting a new experiment/training :param model: keras model :param layer_name: if None initialize all model layers, else only specified layer :return: """ session = K.get_session() for layer in model.layers:

layer.kernel_initializer = glorot_uniform(seed=None)

    layer.kernel_initializer =RandomNormal(mean=0.0, stddev=0.01, seed=None)
    #layer.kernel_initializer = he_normal(session=session)