search

google-research / fool-me-twice

Game code and data for Fool Me Twice: Entailment from Wikipedia Gamification https://arxiv.org/abs/2104.04725
Apache License 2.0
17 stars 6 forks source link

Generate ''Find the Fib'' Questions #4

Open nhatsmrt opened 3 years ago

nhatsmrt commented 3 years ago

When/how are the "find the fib" questions generated to a user? From my understanding of the paper, the flow is that one user will write claims (true and false) about the topics, and then these will show up in another user's menu later, correct? And are the initial "Find the Fib" questions pre-loaded into the database somehow?

eisenjulian commented 3 years ago

Currently newly created claims will not show on other users feed automatically/immediately. We have the concept of a workflow which is a list of pairs of claims to vote on as well as entities to write claims about that is shown to all users at the same time.

A new workflow with the latest data is created and uploaded by running the script python util/create_workflow.py. Then we need to point users running the game to the new workflow id (by default is today's date in YYYY-MM-DD format. This can be done by navigating to https://console.firebase.google.com/u/0/project/PROJECT_ID/config and adding a string parameter kWorkflowID. This parameter will override in realtime the defaultConfig of board/src/config/server.js

Regarding the initial state, the script util/bootstrap.py uploads a few the claims in the FM2 dataset to create an initial workflow, not all of them are being uploaded, but it would be easy to add a flag for that if it's needed.