example ogbg-mol-esol_batch with GINconv please

[thegodone]

I am trying to implement ESOL model but with GINconv and got lot of issues:

So can you give a clear example on how to merge tud_disjoint & ogbg-mol-esol_batch examples please ?

• is A must be sparse ?
• ...

[danielegrattarola]

Hi,

have you looked at this example? With the following changes

• use 'ogbg-molesol'
• change EdgeConditionedConv to GINConv
• copy-paste the evaluator from the ogbg-mol-esol_batch example

you should be good.

Keep in mind that ogbg-molesol has edge attributes but GINConv cannot process them. So you should either

1. Keep everything as-is and simply not use the E_in input layer (this is the fastest way in terms of coding).
2. Change the code to not use edge attributes (basically remove any object/variable that starts with E_).

Also, yes. The adjacency matrix for GINConv should be sparse.

Cheers

[thegodone]

Thanks Daniele,

Model compiles and converges now. (I took option 2 removing all the E_ in the code)

But after the training part, my evaluation is strange cause y_pred are changing each time I run it and the value (rmse) is not very good compare to the model validation mse:

I have to use this method cause I need the I_ values in the models: ################################################################################

EVALUATE MODEL

################################################################################ print('Testing model') evaluator = Evaluator(name=dataset_name) y_pred = [] batches_test = batch_iterator([X_te, A_te], batch_size=batch_size) for b in batchestest: X, A, I = numpy_todisjoint(*b) A = ops.sp_matrix_to_sptensor(A) p = model([X, A, I_], training=False) y_pred.append(p.numpy())

y_pred = np.vstack(y_pred) print(y_pred) model_loss = loss_fn(y_te, y_pred) ogb_score = evaluator.eval({'y_true': y_te, 'y_pred': y_pred})

print('Done. Test loss: {:.4f}. rmse: {:.2f}'.format(model_loss, ogb_score['rmse']))

Any idea why the training=False parameters looks not used, or I made a mistake ?

[thegodone]

another related question on GIN code def call(self, inputs): X, A, E = self.get_inputs(inputs) output = self.mlp((1.0 + self.eps) * X + self.propagate(X, A, E))

why to pass E there ?

here the Build part I used: ################################################################################

BUILD MODEL

################################################################################ X_in = Input(shape=(F,)) A_in = Input(shape=(None,), sparse=True) I_in = Input(shape=(), dtype=tf.int64)

output = GINConv(90, activation='relu')([X_in, A_in]) output = GINConv(90, activation='relu')([output, A_in]) output = GINConv(90, activation='relu')([output, A_in]) output = GlobalAvgPool()([output, I_in]) output = Dense(90, activation='relu')(output) output = Dropout(0.2)(output) output = Dense(n_out, activation='linear')(output)

Build model

model = Model(inputs=[X_in, A_in, I_in], outputs=output) opt = Adam(lr=learning_rate) loss_fn = MeanSquaredError()

@tf.function( input_signature=(tf.TensorSpec((None, F), dtype=tf.float64), tf.SparseTensorSpec((None, None), dtype=tf.float64), tf.TensorSpec((None,), dtype=tf.int32), tf.TensorSpec((None, n_out), dtype=tf.float64)), experimental_relax_shapes=True) def trainstep(X, A, I, y): with tf.GradientTape() as tape: predictions = model([X, A, I], training=True) loss = lossfn(y, predictions) loss += sum(model.losses) gradients = tape.gradient(loss, model.trainable_variables) opt.apply_gradients(zip(gradients, model.trainable_variables)) return loss

[danielegrattarola]

Good to know that it works, although I am not sure what is giving you different predictions every time. training=False should be picked up automatically by TF to disable dropout for inference. What happens if you remove Dropout from the model?

As for why there is E in GinConv: I honestly don't remember why I thought it would be a good idea to keep it there, since it will be None anyway. You can safely ignore it without problems, I'll fix it in a later release.

Cheers

[thegodone]

same issue without Dropout. I was suspect it but it does not solve the problem

Le ven. 13 nov. 2020 à 10:15, Daniele Grattarola notifications@github.com a écrit :

Good to know that it works, although I am not sure what is giving you different predictions every time. training=False should be picked up automatically by TF to disable dropout for inference. What happens if you remove Dropout from the model?

As for why there is E in GinConv: I honestly don't remember why I thought it would be a good idea to keep it there, since it will be None anyway. You can safely ignore it without problems, I'll fix it in a later release.

Cheers

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/danielegrattarola/spektral/issues/115#issuecomment-726645905, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJBWYTEVIWHONZKP6SKTJDSPT2KLANCNFSM4TR44U3A .

[danielegrattarola]

sorry, I forgot about this issue. How different are the embeddings? Can you run a np.allclose(embedding_1, embedding_2)? It could be that the non-determinism of some TF ops is leading to different results, but they shouldn't be that far if the input is the same.

Cheers

[thegodone]

Could you make a real example on mol-esol with the new tf 2.0 and last version ?