Open zimonitrome opened 2 years ago
This is an interesting find! I have tried this now and am seeing a similar phenomenon:
Running torchsort_2
, _3
, _4
(all just calling the exact same function repeatedly) seem to have very similar performance, but it is interestingly worse than the initial trial. I suppose a more substantial warmup might be able to remedy this, but I am not quite sure why the performance would be worse - I'll keep looking into it.
I wanted to try to improve/modify the torchsort code a little so I tried making a copy of the SoftSort class and the soft_sort function.
Running some benchmarks I got the following results:
![benchmark_custom_cuda](https://user-images.githubusercontent.com/17656709/154806979-4d1ad2da-e6c1-43f3-a397-c1d58a346955.png)
Which was worrying. The carbon copy diverges at a similar point to the figure in the readme:
I then re-ran the benchmark with the exact same function twice (not even a copy) and got the same results.
That code can be found here:
Any idea what this might depend on?