About pre-training model.

[XuhaoWan]

Hi authors, For the pre-trained model, three pre-training tasks have three losses. so does the final optimal pre-trained model achievee better performance on all three tasks? Does this mean that the training process needs to be tried continuously. For example, after performing well on task a, use the same parameter model to see how it performs on task bc. If the performance is poor, the entire model will be retrained. Thanks!

[Yeonghun1675]

Thank you for your interest in our MOFTransformer.

As I understand it, your question is that when you perform each of the 3 pre-training tasks, you are asking if it might not be a good fit for the other tasks. (Is this correct?) We did not train the 3 loss functions independently, but together so that they can be trained simultaneously. So we don't go through the process of doing task A and then seeing if it fits B C .

Of course, a model trained simultaneously is not necessarily better than a model trained separately, but since our goal is to ensure that Transformer includes local/global information about MOFs, we focused on training a model that performs well overall, rather than one that performs well on one task.

If you have any further questions, please let us know in the comments.

[XuhaoWan]

Thank you for your response. So the model was trained simultaneously for three tasks with the same loss function. So for the total loss, is it the sum of three sub-task losses?

[Yeonghun1675]

Yes, your explanation is correct. We represented the total loss as the sum of the three subtasks. You can check this in "training_step" in the module of the code!