search

lululxvi / deepxde

A library for scientific machine learning and physics-informed learning
https://deepxde.readthedocs.io
GNU Lesser General Public License v2.1
2.61k stars 734 forks source link

#Set same training points for repeating the training identically #680

Open TiToLiOn opened 2 years ago

TiToLiOn commented 2 years ago

Hi Lu,

I've a problem while I was trying to finding best hyperparametres for my model. I wanted to hold the same training points for each net configuration, to obtain the same result if I decided to repeat the process in the future. For doing that, after setting a seed, I tried to obtain the same distribution of training points for each configuration, but I couldn't. The I decided to create a list of <deepxde.data.pde.TimePDE at 0x2680f9b72b0>, and I realise that, when I plot the training points, I got different results. I think that when I extract them with .train_points(), I got the same result in each column, but combined both columns with diferents order. Here is the part of my code I'm referring to. How could I solve that?

Thanks Lu!

capas=[4,5,6] neuronas=[16,32,64,128,256] N_int=[64,128,256,512] N_sb=[32,64,128] N_tb=[32,64,128]

opt=["adam","L-BFGS"]

lr=[0.001,0.01]

dde.config.set_random_seed(666)

datas=[] #ESTO LO HAGO PORQUE EL SET SEED NO ME FUNCIONA BIEN PARA EL SAMPLEO DE PUNTOS DE ENTRENAMIENTO for i in capas: for j in neuronas: for k in N_int: for l in N_sb: for m in N_tb: for n in lr: data = dde.data.TimePDE( geomtime, edp_difusion, [bc, ic], num_domain=k, num_boundary=l, num_initial=m, solution=sol, ) datas.append(data)

figure() for x in datas[0].train_points(): plot(x[0],x[1],color="red",marker="o")

1

figure() for x in datas[0].train_points(): plot(x[0],x[1],color="red",marker="o")

2
lululxvi commented 2 years ago

set_random_seed sets the random seed, i.e., the random points will be same for running the same code multiple times. If you remove the for loop, it would work. Otherwise, you can generate the points by yourself and pass via anchors.

TiToLiOn commented 2 years ago

Hi Lu,

Thanks for the answer, but it remains giving me a diferent result. This is my code now:

dde.config.set_random_seed(666)

data = dde.data.TimePDE(geomtime, edp_difusion, [bc, ic], num_domain=62, num_boundary=62, num_initial=100, solution=sol, )

a1=data.train_points() a2=data.train_points()

figure() for x in a1: plot(x[0],x[1],color="red",marker="o")

1

figure()
for x in a2: plot(x[0],x[1],color="red",marker="o"

2

)

I have a pair of questions.

  1. About the mecanic of the method .train_points(), at the moment when I call it, it generates the points, isn't it? Why does the set_random_seed doesn't let me get the same result.
  2. Setting the same seed affects to the overall random computations in the net, such as the Glorot uniform initialization of the weights or the gradient algorithm I'm using, ie, setting a seed and running the code of the neural net, I had to get the same results (in term of residuals in the case that I use the same training points), cause I dont get the same results and I don't know if it's due to the generation of training points is different althought I'm using the same seed, or if also been afecting other changes that occurs randomly inside the working of the net.

Thanks for the help proffesor!

lululxvi commented 2 years ago

This is as expected. Consider the following pseudocode:

numpy.set_random_seed(0)
a = np.random()
b = np.random()

Should a and b be the same or not?