search

Dallinger / Griduniverse

Welcome to the Griduniverse.
7 stars 3 forks source link

Implementation of is_complete() needs improvement #203

Open jessesnyder opened 6 years ago

jessesnyder commented 6 years ago

In Slack channel, @alecpm suggests: "I think we should maybe use a timeout based on time since last activity for any participant once the normal completion logic (networks filled all participants in accepted or rejected state). We also need to make sure we expire any remaining HITs shortly after deciding the experiment is complete."

alecpm commented 6 years ago

Is this huge number of recruits with the older MultiRecruiter code? That didn't really limit the number of recruits in any sensible way, but that should be fixed.

jessesnyder commented 6 years ago

@alecpm I'm not positive I included the relevant console output from Slack. You and @MatthewWilkes were discussing the issue in the context of Matthew's longish list of console excerpts. I somewhat hastily grabbed what I thought was the right one while Vishal and I were meeting yesterday.

alecpm commented 6 years ago

OK, given that, I'm pretty sure that traceback isn't relevant to the issue of marking an over-recruited experiment as complete. The massive over-recruitment issue was a flaw in the MultiRecruiter which should be fixed now.

alecpm commented 6 years ago

Actually, the over-recruitment issue won't be fixed until PR Dallinger/Dallinger#1393 is merged, but it's not related to the is_complete() condition which is GU specific. The default Dallinger is_complete() condition would work fine, but it will sometimes cut off over-recruited HITs before they've been submitted.