search

cornell-zhang / HOGA

Hop-Wise Graph Attention for Scalable and Generalizable Learning on Circuits
BSD 3-Clause "New" or "Revised" License
22 stars 2 forks source link

About the code implementation on the QoR task #2

Open Jedges opened 3 months ago

Jedges commented 3 months ago

I'm sorry to bother you again. I implemented the code and tested it according to the previous discussion, but the results do not seem to be consistent with those in the paper, so I have a few questions.

  1. HOGA needs to calculate multi-hop features in advance. In the QoR task, its initial feature is a 3D vector, which is then mapped to a high-order vector through an embedding layer. At which step does Hoga calculate multi-hops? If it is the latter, it must be recalculated in each epoch?
  2. I used the evalute function in OpenABC-D to calculate the result statistics. Did you also use this function? It used mean_absolute_percentage_error to calculate MAPE, and the result I got was two orders of magnitude larger than the one reported in your paper. I wonder if you have encountered this problem. This is the test result on some data image
  3. If it is convenient, could you provide the model weights or training code? I'm not sure if my implementation is correct.

I sincerely hope to get your reply, thanks very much.

Chenhui1016 commented 2 months ago

Hi,

Here are the answers to your questions:

  1. We calculate the hop-wise features based on the initial 3D vector.
  2. Your results are actually consistent with ours. Please refer to our note for details.
  3. We have provided all the necessary scripts and sources for reproducing our results (which seems you have already done). Additionally, we provided a simple script to quickly reproduce all MAPE scores on test set and plots of HOGA predictions vs. ground truth. Since I’m fully occupied with other work, we do not plan to add additional scripts in the near future.
Jedges commented 2 months ago

Thanks very much for your reply, which answered my question about MAPE calculation. It seems that I have successfully reproduced the results of HOGA on the QoR task. I wish you all the best in your future work.