google-research / fool-me-twice

Game code and data for Fool Me Twice: Entailment from Wikipedia Gamification https://arxiv.org/abs/2104.04725
Apache License 2.0
17 stars 6 forks source link

User are able to re-answer Fib questions #5

Closed nhatsmrt closed 3 years ago

nhatsmrt commented 3 years ago

In the "Find the Fib" questions, after the player has selected a statement, he/she is asked to select which one is more interesting:

Screen Shot 2021-05-23 at 6 38 57 PM

If, instead of selecting one, the player just escapes to home screen (for e.g by clicking on the leaderboard then click on play again), then he/she can re-answer the question. This effectively allows the user to cheat by re-answer the question, and this could be done an infinite number of times for the same question...

Could you double check to see if the issue occurs on your end as well? Thanks!

eisenjulian commented 3 years ago

Good catch, currently the mechanism that prevents the user from seeing the same task more than once in the Workflow is controlled by the client (so no cheating safe), by updating the object in the database called userStatus here. This occurs whenever the game screen invokes the callback that the game is over. I think it would be a bit more complex to provide another callback that allows for a partial completion.

I think the current way this works is ok as long as we don't award the points multiple times, and it's safer since it will run server side. For that, a simple fix would be to add the following logic the createVote function in firebase/functions/index.js replacing the existing to-do. If this works for you feel free to send a PR or I'll add it directly.

const priorPoints = await Promise.all(fibs.map(
  fibId => db.doc(`fibs/${fibId}/votes/${uid}`).get().then(f => f.exists)
));
// The player already voted on some of this claims so we are not giving votes twice.
if (priorPoints.some(b => b)) {
  res.json({success, points: 0})
  return;
}
nhatsmrt commented 3 years ago

That seems to do the trick! I'll do some further testing, then create a PR. Thank you so much for your help!

nhatsmrt commented 3 years ago

Here you go: https://github.com/google-research/fool-me-twice/pull/6