The comment highlights two main points, as stated in the abstract:
We identify two major issues in the SoftDist paper (Xia et al.): (1) the failure to run all steps of different baselines on the same hardware environment, and (2) the use of inconsistent time measurements when comparing to other baselines. These issues lead to flawed conclusions. When all steps are executed in the same hardware environment, the primary claim made in SoftDist is no longer supported.
Therefore, the two questions are:
1. Did you run all steps of different baselines (inference + I/O + search) on the same hardware environment?2. Were the time measurements consistent (did you account for I/O) when comparing to other baselines?
some other minor includes:
You extensively fine-tuned your model, but did not tune any other GNN models, correct?
Did you run the inference part of other models? Since the local CPU/GPU environment has changed, you should correspondingly tune the model to ensure that it performs well on your local environment.
The comment highlights two main points, as stated in the abstract:
We identify two major issues in the SoftDist paper (Xia et al.): (1) the failure to run all steps of different baselines on the same hardware environment, and (2) the use of inconsistent time measurements when comparing to other baselines. These issues lead to flawed conclusions. When all steps are executed in the same hardware environment, the primary claim made in SoftDist is no longer supported.
Therefore, the two questions are:
1. Did you run all steps of different baselines (inference + I/O + search) on the same hardware environment? 2. Were the time measurements consistent (did you account for I/O) when comparing to other baselines?
some other minor includes: