xyfffff / rethink_mcts_for_tsp

[ICML'24 Oral] Rethinking Post-Hoc Search-Based Neural Approaches for Solving Large-Scale Traveling Salesman Problems
20 stars 1 forks source link

why the inference time is zero #2

Open yimengmin opened 1 month ago

yimengmin commented 1 month ago

Hi I am a little bit confused that why your inference time on TSP-1000 in your paper is 0.00 minutes. FYI, the inference time in UTSP is inference + I/O operation, does the inference time in your paper includes the I/O? if just the for inference, UTSP is ~ 1s.

Thank you

xyfffff commented 3 weeks ago

Hi,

Thank you for your interest in our work. I also came across your commentary, "Comment on paper: Position: Rethinking Post-Hoc Search-Based Neural Approaches for Solving Large-Scale Traveling Salesman Problems," and I found it somewhat lacking in substance and humorously misguided.

You seem to believe that the inference time for UTSP includes I/O operations while SoftDist does not, thereby questioning the validity of our conclusions. This perspective is fundamentally flawed for several reasons:

  1. Performance Comparison: As detailed in Table 1 of our paper, which provides comprehensive experimental results across various methods, even if we disregard the I/O time for UTSP, its performance remains the poorest. Furthermore, UTSP fails to scale to TSP-10000 instances. Hence, our original conclusion stands robust: UTSP lags behind SoftDist in both speed and performance metrics.

  2. Misinterpretation of Table 3: Your assertion that “Table 3 shows SoftDist failing to outperform UTSP” is an overreach. The intent of our Table 3 experiment is to demonstrate that, with finely-tuned MCTS parameters, the impact of the heatmap diminishes. Importantly, it was not designed to prove that SoftDist surpasses other methods. As Table 3 shows, even when using a Zero heatmap for TSP-500, the final results surpass those of UTSP. Therefore, in a scenario where MCTS parameters are optimized, what practical value does your UTSP algorithm hold?

Moreover, I believe your commentary is riddled with errors and demonstrates a clear lack of thorough reading of our paper. I would recommend that you carefully review our work in its entirety before drawing any conclusions.

In light of these points, it's clear that our conclusions are well-founded. Your arguments fail to undermine the validity of our findings and reflect a misunderstanding of the nuances within our experimental design.

Best regards, Yifan

yimengmin commented 1 day ago

do you run the UTSP inference part or not?

xyfffff commented 1 day ago

Did you read our paper or not? The answer to your question is clearly explained in the description of Table 1.

yimengmin commented 1 day ago

I have carefully reviewed your paper and would like to confirm some details.

Since my previous question was not addressed, I am assuming that you did not run any inference on your GPU.

Additionally, your Table 1 does not clearly clarify whether inference was performed or where the total time originates very clearly. To be specific, the caption mentions "corresponding to heatmap generation and MCTS runtimes," but it appears that this was directly taken from the UTSP paper. Therefore, the time actually includes heatmap generation, I/O time, and MCTS runtimes, which means your caption and the time column are incorrect.
That's my understanding.

Also, you mentioned "† indicates methods utilizing heatmaps provided by the original authors", however, I did NOT remember I authorized you to publish a paper with that, this may lead to some potential Ethical issues: using someone else's data without their consent can be considered unethical. It's generally good practice to obtain explicit permission from the original authors, particularly if their data or methods are central to your research. I love open source, but I believe you should at least inform me about where you plan to use the data before proceeding.