Credence Game stuck on a single question

[tedks]

I seem to be getting the same question no matter how many times I answer it in the credence game. This is question number 38 in my game, relating to "Which of these fighters was earlier in their first UFC Heavyweight championship victory?"

Contact me if you need my username on predictionbook to further investigate this.

[ghost]

similarly i appear to be stuck at question 137

"Question 137: As of 2015, each of these countries was one of the twenty most populous in the world. Which had a higher population? "

[malob]

I'm stuck on:

Question 31: In 2011, Forbes Magazine ranked each of the following among the best 100 colleges in the United States. Which one did they rank higher?

[malob]

FYI, I'm using Safari 11.0.1 on macOS 10.13.1. I've tried deleting all website data and turning off my content blocker and neither helped. I also tried using a fresh install of Chrome, and had the same issue.

I quickly looked through recent commits and couldn't immediately see any code changes that would have cause this. Though that doesn't mean much since I'm unfamiliar with the codebase and didn't look very hard 😛.

[michaelwebb76]

Thanks @malob. This code has been untouched since it was introduced, with the exception of a few forced gem (dependency) -related updates. This is on my to-do list... but it's a long to-do list.

[jordan-schneider]

+1 stuck on first question given.

[bmillwood]

FWIW I got stuck on the first question on a locally-running deployment. I wouldn't put too much faith in my server being set up correctly, so it could be unrelated... but here's the log on the server's stdout:

Started PUT "/credence_game_responses/1" for ::1 at 2018-07-21 18:48:45 +0800
Processing by CredenceGameResponsesController#update as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"sRK+08lxSBP5cnheCZepMJ1RGWn8oTPZAoj1gnAHDpnI7xVRqg2DZiXtFlynOQZGfUCl0QOI3L8lFsFo4sBpBA==", "response"=>{"answer_credence"=>"99", "given_answer"=>"1"}, "commit"=>"99%", "id"=>"1"}
  User Load (0.3ms)  SELECT  `users`.* FROM `users` WHERE `users`.`id` = 4 ORDER BY `users`.`id` ASC LIMIT 1
  CredenceGameResponse Load (0.2ms)  SELECT  `credence_game_responses`.* FROM `credence_game_responses` WHERE `credence_game_responses`.`id` = 1 LIMIT 1
  CredenceGame Load (0.2ms)  SELECT  `credence_games`.* FROM `credence_games` WHERE `credence_games`.`id` = 1 LIMIT 1
  CredenceGame Load (0.2ms)  SELECT  `credence_games`.* FROM `credence_games` WHERE `credence_games`.`user_id` = 4 LIMIT 1
   (0.1ms)  BEGIN
  SQL (0.3ms)  UPDATE `credence_game_responses` SET `answered_at` = '2018-07-21 10:48:45', `updated_at` = '2018-07-21 10:48:45' WHERE `credence_game_responses`.`id` = 1
   (5.8ms)  COMMIT
  CredenceQuestion Load (0.4ms)  SELECT  `credence_questions`.* FROM `credence_questions` WHERE `credence_questions`.`enabled` = 1 ORDER BY RAND() LIMIT 1
  CredenceAnswer Load (0.3ms)  SELECT  `credence_answers`.* FROM `credence_answers` WHERE `credence_answers`.`credence_question_id` = 49 ORDER BY RAND() LIMIT 5
  CredenceAnswer Load (0.2ms)  SELECT  `credence_answers`.* FROM `credence_answers` WHERE `credence_answers`.`id` = 752 LIMIT 1
  CredenceAnswer Load (0.1ms)  SELECT  `credence_answers`.* FROM `credence_answers` WHERE `credence_answers`.`id` = 718 LIMIT 1
  CredenceQuestion Load (0.2ms)  SELECT  `credence_questions`.* FROM `credence_questions` WHERE `credence_questions`.`id` = 17 LIMIT 1
  CACHE CredenceQuestion Load (0.0ms)  SELECT  `credence_questions`.* FROM `credence_questions` WHERE `credence_questions`.`id` = 17 LIMIT 1  [["id", 17], ["LIMIT", 1]]
Redirected to http://localhost:3000/credence_games/try
Completed 302 Found in 18ms (ActiveRecord: 8.4ms)

Started GET "/credence_games/try" for ::1 at 2018-07-21 18:48:45 +0800
Processing by CredenceGamesController#show as HTML
  Parameters: {"id"=>"try"}
  User Load (0.3ms)  SELECT  `users`.* FROM `users` WHERE `users`.`id` = 4 ORDER BY `users`.`id` ASC LIMIT 1
  CredenceQuestion Exists (0.2ms)  SELECT  1 AS one FROM `credence_questions` LIMIT 1
  CredenceGame Load (0.2ms)  SELECT  `credence_games`.* FROM `credence_games` WHERE `credence_games`.`user_id` = 4 LIMIT 1
  CredenceGameResponse Load (0.2ms)  SELECT  `credence_game_responses`.* FROM `credence_game_responses` WHERE `credence_game_responses`.`id` = 1 LIMIT 1
  Rendering credence_games/show.html.erb within layouts/application
  CredenceGameResponse Load (0.3ms)  SELECT  `credence_game_responses`.* FROM `credence_game_responses` WHERE `credence_game_responses`.`credence_game_id` = 1 AND (answered_at IS NOT NULL) ORDER BY answered_at desc LIMIT 10
  CACHE CredenceGameResponse Load (0.0ms)  SELECT  `credence_game_responses`.* FROM `credence_game_responses` WHERE `credence_game_responses`.`credence_game_id` = 1 AND (answered_at IS NOT NULL) ORDER BY answered_at desc LIMIT 10  [["credence_game_id", 1], ["LIMIT", 10]]
  CredenceQuestion Load (0.3ms)  SELECT  `credence_questions`.* FROM `credence_questions` WHERE `credence_questions`.`id` = 17 LIMIT 1
  CredenceAnswer Load (0.2ms)  SELECT  `credence_answers`.* FROM `credence_answers` WHERE `credence_answers`.`id` = 752 LIMIT 1
  CredenceAnswer Load (0.1ms)  SELECT  `credence_answers`.* FROM `credence_answers` WHERE `credence_answers`.`id` = 718 LIMIT 1
  Rendered credence_games/show.html.erb within layouts/application (7.4ms)
  Response Load (0.3ms)  SELECT `responses`.`prediction_id` FROM `responses` WHERE `responses`.`user_id` = 4 AND (confidence is not null)
  Prediction Load (0.2ms)  SELECT `predictions`.* FROM `predictions` WHERE 1=0 ORDER BY `predictions`.`updated_at` DESC
  Rendered layouts/_messages.html.erb (2.1ms)
Completed 200 OK in 31ms (Views: 25.7ms | ActiveRecord: 2.4ms)

[bmillwood]

here's another earlier log from the same stuck loop:

 Started PUT "/credence_game_responses/1" for ::1 at 2018-07-21 18:48:32 +0800
Processing by CredenceGameResponsesController#update as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"jexz1uDDQG0epM6X8t6N5b42nr7jz0d0eEQEZhs+8cf0EdhUg7+LGMI7oJVccCKTXiciBhzmqBJf2jCMifmWWg==", "response"=>{"answer_credence"=>"99", "given_answer"=>"1"}, "commit"=>"99%", "id"=>"1"}
  User Load (0.4ms)  SELECT  `users`.* FROM `users` WHERE `users`.`id` = 4 ORDER BY `users`.`id` ASC LIMIT 1
  CredenceGameResponse Load (0.3ms)  SELECT  `credence_game_responses`.* FROM `credence_game_responses` WHERE `credence_game_responses`.`id` = 1 LIMIT 1
  CredenceGame Load (0.2ms)  SELECT  `credence_games`.* FROM `credence_games` WHERE `credence_games`.`id` = 1 LIMIT 1
  CredenceGame Load (0.2ms)  SELECT  `credence_games`.* FROM `credence_games` WHERE `credence_games`.`user_id` = 4 LIMIT 1
   (0.1ms)  BEGIN
  SQL (0.3ms)  UPDATE `credence_game_responses` SET `answered_at` = '2018-07-21 10:48:32', `updated_at` = '2018-07-21 10:48:32' WHERE `credence_game_responses`.`id` = 1
   (4.9ms)  COMMIT
  CredenceQuestion Load (0.4ms)  SELECT  `credence_questions`.* FROM `credence_questions` WHERE `credence_questions`.`enabled` = 1 ORDER BY RAND() LIMIT 1
  CredenceAnswer Load (0.4ms)  SELECT  `credence_answers`.* FROM `credence_answers` WHERE `credence_answers`.`credence_question_id` = 35 ORDER BY RAND() LIMIT 5
  CredenceAnswer Load (0.2ms)  SELECT  `credence_answers`.* FROM `credence_answers` WHERE `credence_answers`.`id` = 752 LIMIT 1
  CredenceAnswer Load (0.2ms)  SELECT  `credence_answers`.* FROM `credence_answers` WHERE `credence_answers`.`id` = 718 LIMIT 1
  CredenceQuestion Load (0.2ms)  SELECT  `credence_questions`.* FROM `credence_questions` WHERE `credence_questions`.`id` = 17 LIMIT 1
  CACHE CredenceQuestion Load (0.0ms)  SELECT  `credence_questions`.* FROM `credence_questions` WHERE `credence_questions`.`id` = 17 LIMIT 1  [["id", 17], ["LIMIT", 1]]
Redirected to http://localhost:3000/credence_games/try
Completed 302 Found in 19ms (ActiveRecord: 7.7ms)

Started GET "/credence_games/try" for ::1 at 2018-07-21 18:48:32 +0800
Processing by CredenceGamesController#show as HTML
  Parameters: {"id"=>"try"}
  User Load (0.3ms)  SELECT  `users`.* FROM `users` WHERE `users`.`id` = 4 ORDER BY `users`.`id` ASC LIMIT 1
  CredenceQuestion Exists (0.2ms)  SELECT  1 AS one FROM `credence_questions` LIMIT 1
  CredenceGame Load (0.2ms)  SELECT  `credence_games`.* FROM `credence_games` WHERE `credence_games`.`user_id` = 4 LIMIT 1
  CredenceGameResponse Load (0.2ms)  SELECT  `credence_game_responses`.* FROM `credence_game_responses` WHERE `credence_game_responses`.`id` = 1 LIMIT 1
  Rendering credence_games/show.html.erb within layouts/application
  CredenceGameResponse Load (0.4ms)  SELECT  `credence_game_responses`.* FROM `credence_game_responses` WHERE `credence_game_responses`.`credence_game_id` = 1 AND (answered_at IS NOT NULL) ORDER BY answered_at desc LIMIT 10
  CACHE CredenceGameResponse Load (0.0ms)  SELECT  `credence_game_responses`.* FROM `credence_game_responses` WHERE `credence_game_responses`.`credence_game_id` = 1 AND (answered_at IS NOT NULL) ORDER BY answered_at desc LIMIT 10  [["credence_game_id", 1], ["LIMIT", 10]]
  CredenceQuestion Load (0.3ms)  SELECT  `credence_questions`.* FROM `credence_questions` WHERE `credence_questions`.`id` = 17 LIMIT 1
  CredenceAnswer Load (0.2ms)  SELECT  `credence_answers`.* FROM `credence_answers` WHERE `credence_answers`.`id` = 752 LIMIT 1
  CredenceAnswer Load (0.2ms)  SELECT  `credence_answers`.* FROM `credence_answers` WHERE `credence_answers`.`id` = 718 LIMIT 1
  Rendered credence_games/show.html.erb within layouts/application (7.5ms)
  Response Load (0.3ms)  SELECT `responses`.`prediction_id` FROM `responses` WHERE `responses`.`user_id` = 4 AND (confidence is not null)
  Prediction Load (0.2ms)  SELECT `predictions`.* FROM `predictions` WHERE 1=0 ORDER BY `predictions`.`updated_at` DESC
  Rendered layouts/_messages.html.erb (2.1ms)
Completed 200 OK in 32ms (Views: 25.7ms | ActiveRecord: 2.6ms)

I wonder if it has something to do with the CredenceQuestion caching?

[bmillwood]

No, it's probably not the question cache thing. The problem is the credence game itself not getting e.g. num_answered updated.

I found that my problem was solved if I removed the if saved_changes? part of app/models/credence_game.rb, but I don't yet understand why. (It still might not be the same problem as exists in the production version.)

[Raemon]

Just noting this is still a problem for me (it's stuck on the very first question)

[karlkeefer]

Ditto ^ stuck on question 1 comparing two rivers in South America.

[sonofhypnos]

Same for me

[karlkeefer]

I want to share the credence game with a number of folks, and it's valuable if the predictionbook version was made to work again - how many people's calibration might be improved!?

I'm attaching a totally-unofficial-but-my-handle-is-my-real-name-so-trust-me bounty of $100 to this bug if it gets fixed in the next 30 days.

Tagging @michaelwebb76 since it was on your to-do list 3 years ago - if not you, maybe you can tag another contributor familiar with this chunk of code?

[bmillwood]

@karlkeefer Maybe you're already aware, but there are other versions of the game, e.g. http://acritch.com/credence-game/ and http://www.openphilanthropy.org/calibration

[karlkeefer]

Thanks @bmillwood.

The acritch.com one I've downloaded and played (and it works!) but that is a big barrier vs. just running it in the browser.

The open philanthropy one is great (though I wish it had a dedicated website with some kind of introduction, vs. a mysterious signup wall...) maybe the effort is better spent improving the landing for the openphilanthropy version... Thank you for the links!!!

---

edit: I emailed openPhil to see if they'd be willing to create a dedicated landing page w/ some of the FAQ from within that version hosted before the auth wall.

[jaysonvirissimo]

@karlkeefer did you ever manage to find an alternative, browser-based, calibration game? If not, this may be of some use.