The issue of significant disparities in the results between the maxpressure and sotl algorithms.

[yifan-dadada]

Thank you very much for your team's contribution to LibSignal, providing a benchmark for comparison. However, while using the maxpressure and sotl algorithms on the Hangzhou 4x4 dataset, I have observed significant discrepancies compared to the results reported by your team in the paper. I would like to inquire whether you are aware of the reasons behind these differences and if you could provide assistance in resolving this issue. Your guidance would be highly appreciated.

Furthermore, I have tested other road networks, and the issue persists in the maxpressure algorithm. I hope to receive your assistance in resolving this matter.

[derekmei233]

Hi yifan, Thanks for your inquiry. Are you having this problem on master branch? I will look into the problem and let you know this weekend.

[yifan-dadada]

Hi yifan, Thanks for your inquiry. Are you having this problem on master branch? I will look into the problem and let you know this weekend.

Thank you very much for your reply. I noticed the response in question, so the branch I used is the tutorial branch. Will using the master branch yield different results? Which branch do you recommend using? I will try using other branches for testing.

Thank you again for your help.

[yifan-dadada]

Hi yifan, Thanks for your inquiry. Are you having this problem on master branch? I will look into the problem and let you know this weekend.

Hello Derek,

Based on your suggestion, I conducted a new experiment on the master branch and found that the results were normal with an acceptable margin of error.

maxpressure --- hanghzou4x4

sotl --- hanghzou4x4

In order to clarify the reason, I tested the results of the fixed time algorithm between two branches and found that there are also differences. And I have ruled out the possibility of data differences. So, are there significant differences between different branches? Which branch should I use as the final result?

fixedtime-30 --- hangzhou4x4

fixedtime-10 --- hangzhou4x4

In addition, I also found that the results in the fixed time are opposite to those given in the paper.

Thank you again for your reply. I would greatly appreciate your help!

[yifan-dadada]

Hi yifan, Thanks for your inquiry. Are you having this problem on master branch? I will look into the problem and let you know this weekend.

Hello Derek,

Based on your suggestion, I conducted a new experiment on the master branch and found that the results were normal with an acceptable margin of error.

maxpressure --- hanghzou4x4

sotl --- hanghzou4x4

In order to clarify the reason, I tested the results of the fixed time algorithm between two branches and found that there are also differences. And I have ruled out the possibility of data differences. So, are there significant differences between different branches? Which branch should I use as the final result?

fixedtime-30 --- hangzhou4x4

fixedtime-10 --- hangzhou4x4

In addition, I also found that the results in the fixed time are opposite to those given in the paper.

Thank you again for your reply. I would greatly appreciate your help!

Hello Derek,

I compared the statistical methods of the master branch and the tutorial branch, and found two differences in the (world_sumo. py) code. The first point is the difference in statistical distance (max_distance), which is 1000 in the tutorial branch and 200 in the master branch. The second point is that there are differences in the code for calculating Travel Time. Upon inspection, I found that the code in the master branch should be correct.

Are there any other differences in these two branches that affect the results?

Thank you very much for responding to my question. Thank you again for your help.

[wingsweihua]

@derekmei233 Follow up on this.