Closed lv2020 closed 3 months ago
Hi, lv2020, two key questions.
Thanks for your quick reply!
For the first question, my concern is whether there is label leakage in this case as only the shape is different. And if there is a new type of subgraph, then TLM needs to be retrained?
We recommend retraining, which is the simplest and most effective way. TLM training is very fast. We found that pre-training on 4 V100s for 2 hours is almost perfect; if you load the previous checkpoint for pre-training, it will be quicker. I guess it only takes 30 minutes. Of course, it is not necessary to retrain. TLM's generalization ability comes from subgraph-type tokens and other tokens (such as subgraph shape and hardware specifications). Even if the subgraph type token is [UNKNOWN], it still has a certain generalization ability.
Thank you for your patience and detailed answer!
Thanks for sharing your code and dataset! I have two main questions regarding the generalization performance of TLM: