mgravell / SortOfProblem

A demo project discussion sorting
29 stars 3 forks source link

Performance of indexed sorted data? #3

Open GotoFinal opened 6 years ago

GotoFinal commented 6 years ago

I wonder if there will be any visible difference when iterating over indexed sorted data, as instead of simple sequential reads from memory we need to jump to random places in memory to get each element. Can you maybe test if there will be any difference when iterating over all that sorted elements with some simple operation inside loop? Would be interesting if cost of accessing that data would be similar to what we saved by not sorting real data and just using indexes.

Sorry that I didn't just make a PR, but currently I don't have enough time to play with this, especially that C# isn't my language so it will take some time for me to find and learn some good benchmark tools for this language.

mgravell commented 6 years ago

Well, if you're iterating over all the data, most aggregates don't care about the sort, so you would just ignore the index and go via the data...

I understand what you're asking, and I can test it, but: in reality it isn't a huge major scenario. The main thing that might require a sorted full scan is an ordered export.

On 21 Jan 2018 5:40 pm, "Bartłomiej Mazur" notifications@github.com wrote:

I wonder if there will be any visible difference when iterating over indexed sorted data, as instead of simple sequential reads from memory we need to jump to random places in memory to get each element. Can you maybe test if there will be any difference when iterating over all that sorted elements with some simple operation inside loop?

Sorry that I didn't just make a PR, but currently I don't have enough time to play with this, especially that C# isn't my language so it will take some time for me to find and learn some good benchmark tools for this language.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mgravell/SortOfProblem/issues/3, or mute the thread https://github.com/notifications/unsubscribe-auth/AABDsD2GSyCzBGDAAXK3Y8BnOzkDOvO6ks5tM3Z2gaJpZM4Rl5bn .