Smart contract is selecting the wrong winner

[BlockchainML]

Dear Danku Organizers,

On the 23rd of March we submitted the highest scoring valid model (best accuracy within the Ethereum gas limit) here https://etherscan.io/tx/0xefa6d4f6d815b6.... , which was given the model ID: 10.

It achieves 83% accuracy on the test set. We called evaluate_model in this transaction https://etherscan.io/tx/0x79a33b1900140... and was surprised to find that the smart contract still thinks that model 24 should win (current state of best_submission_index). While model 24 also scores 83% on the test set, it was submitted many days after ours and the whitepaper clearly states that "If there are two submissions with the same solution, and this solution is the best one, the first submitter will get paid out instead if they’re both evaluated".

We hope you can help us resolve this problem - we spent many hours and much gas on this competition.

In particular, it would be unfair to award the price to 0x48cd... as they at many times were outright copying our models: https://etherscan.io/tx/0xe474b7576e19... (posted by us at block 5363565), https://etherscan.io/tx/0xc24e11df57edb1ca2c0619ae4edf... (posted by 0x48cd at block 5363824 ~1 hour later). The latter uses the same network layout and many of the parameters are exactly the same (check the bias parameters at the end 0x1388, 0x2710, 0x0, 0x5, etc). It didn't bother us at the time because we assumed our model would still take priority because it was submitted earlier.

Thanks in advance, 0x31E518d6C491aF158b5a3d41667e79287d1E9a2E

--- Message Signed ---
{ "address": "0x31e518d6c491af158b5a3d41667e79287d1e9a2e",
  "msg": "Dear Danku Organizers,\n\nOn the 23rd of March we submitted the highest scoring valid model (best accuracy within the Ethereum gas limit) here https://etherscan.io/tx/0xefa6d4f6d815b6.... , which was given the model ID: 10.\n\nIt achieves 83% accuracy on the test set. We called evaluate_model in this transaction https://etherscan.io/tx/0x79a33b1900140... and was surprised to find that the smart contract still thinks that model 24 should win (current state of best_submission_index). While model 24 also scores 83% on the test set, it was submitted many days after ours and the whitepaper clearly states that \"If there are two submissions with the same solution, and this solution is the best one, the first submitter will get paid out instead if they’re both evaluated\".\n\nWe hope you can help us resolve this problem - we spent many hours and much gas on this competition.\n\nIn particular, it would be unfair to award the price to 0x48cd... as they at many times were outright copying our models: https://etherscan.io/tx/0xe474b7576e19... (posted by us at block 5363565), https://etherscan.io/tx/0xc24e11df57edb1ca2c0619ae4edf... (posted by 0x48cd at block 5363824 ~1 hour later). The later uses the same network layout and many of the parameters are exactly the same (check the bias parameters at the end 0x1388, 0x2710, 0x0, 0x5, etc). It didn't bother us at the time because we assumed our model would still take priority because it was submitted earlier.\n\nThanks in advance,\n0x31E518d6C491aF158b5a3d41667e79287d1E9a2E",
  "sig": "0x1da1fb9c185a8c3fd4f7a98a64149ca56751703ae4800f1bcc15895f142a9f892ae31583b4053cb6b0f9f329b8bd8d0c4593dd6b73b0a709702a7a74fd5bb8291c",
  "version": "3",
  "signer": "MEW"}
---

[besirkurtulmus]

Thank you for reporting this. We're currently looking into it as we speak.

Please send us an email at info@algorithmia.com. So that we can respond with a resolution.

[besirkurtulmus]

Fixed in commit: 4656d30c9de3287f8713e0533cddd33d68cc039c