Curious about your position

[Ataraksia]

Greetings and salutations. First off, let me say that I have been incredibly impressed with all you've been able to accomplish with this project in such a short period of time(by project I mean automatic in general, although I am a great fan of this extension in particular as well). I honestly have no idea how you have been able to maintain such a ridiculously high rate of commits, and quality commits at that, new functionality appearing almost everyday it seems; you are an absolute machine, my friend, and I am excited to see how the project continues to develop.

Now as to the reason for my message here today, I am curious about where you stand on an issue related to this particular project. You see, I have been enjoying benchmarking my system in order to see how high I could push things; it's been good fun experimenting with each new enhancement you push out. At the moment, I am managing an ever so slight lead for the top spot on the charts (I am the so cleverly named 'me'). These last few days I have been enjoying the new token merging addition, and only finally today thought to make use of it for benchmarking as well. As usual, for the sake of satisfying my curiosity, I switched all its settings to their maximum throughput options, and was amazed to see a massive 10it/s increase from 54 to 64. I haven't uploaded my results yet because I have to admit, at this level, it feels like it's entirely about gaming the system with all pretense of providing any actually useful benchmark data totally abandoned; I would never actually employ these particular settings in practice.

Now, at the end of the day do a few outlier datapoints really matter all that much, probably not, but nevertheless I was somewhat curious to hear your position on the matter; do you think that users should only benchmark with settings that they would actually employ in practice, should they benchmark to the highest degree that they are able, or are you completely indifferent to this topic and are angrily wondering to yourself why I'm wasting your time with such unimportant nonsense? Honestly, if you don't have the time, or inclination, to provide a reply to this, I don't really mind if you ignore it. It was more of an idle curiosity on my part at the end of the day anyway.

[vladmandic]

angrily wondering to yourself why I'm wasting your time with such unimportant nonsense

haha! not at all :)

I am managing an ever so slight lead for the top spot on the charts

nice! i need to get a better hardware :)

...it feels like it's entirely about gaming the system

yes, it is gaming the system, but as long as it is above the board, that is perfectly fine. ideally, i should store all such settings in the benchmark database, but that would be a lot to gather and with continuous development of the core repo, i'd have to update what to collect quite frequently.

so if you want to play fair, do the benchmark and write in the description what you did - doesn't have to be long, just a quick note. i think others may find that both interesting and useful.

btw, i'm going to close the issue as its not really an issue, but feel free to comments further...

[Ataraksia]

Haha, no worries, I took your advice about leaving a note to explain my jump in score, and I had nothing really of note to add here so I was just going to leave you to your wizardry. However, during a benchmark I was running today, I recorded an absurdly high 78it/s, which I of course immediately uploaded without thinking to double check the score first. After coming to my senses a bit, I decided to redo the benchmark to make sure that the score was actually reproducible and not merely the result of a strange glitch, which, as it turns out, I think it actually may have been. In all the tests I've run since, I consistently score around 68it/s, which I'm still extremely happy with, but that very much makes me doubt that initial 78it/s that I recorded. Assuming it's not too much trouble, I think it's probably best to to remove that likely errant score, the ID for which is 1591926582438719490.

Additionally, while I've got you haha, I actually had an issue a week or so ago where one of my scores posted twice for some reason. There are two IDs that correspond to that single duplicated entry, each with a different timestamp. I assume that aspect though is normal anyway glancing through the database file. 1590566428745052170 1590566030256812033

Very much appreciated :). I shall endeavor to not break the benchmarking system anymore moving forward :P.

[vladmandic]

I'll remove the entries, no worries.