More flexible meta-association.

[cscott]

This hunt is the second hunt in the row (as I recall) that had an unusual association of puzzles to metas. In particular, having puzzles associated with more than one meta seems to be common enough that we should support it.

Perhaps letting puzzles be switched to rounds (ie metas) more easily (aka #221) in connection with letting puzzles be associated with multiple rounds (instead of one-to-one) would solve this problem. The puzzle should appear in the rounds-puzzle-list of all rounds which it is associated with. This leads to some duplication on the blackboard, but that duplication seems to be precisely what folks working on the metas would prefer (we manually copied puzzles in the emotions round to achieve this). There can be Yet Another Setting (see settings discussion in #273) to suppress duplicates, for folks who just want to see new open puzzles.

This preserves the "rounds are metas" assumption in the code, changing only the 1-to-1 association of puzzles to rounds/metas.

268 is related, but I think Round Groups are still a useful concept, in fact I would create one more layer of them if possible. This year we had "Recover the Core Memories" -> "Sci Fi Island" -> "Transformers" -> {puzzle}, for example. If I were to make a change w/ the round groups, it would just be to make the # of parents variable: "Events" just had one standalone round, eg, and the "Meals" round group only had two puzzles (which we chose to represent as "metas" with no puzzles.). Not every round needs two layers of round groups above it. But allowing Round Groups to contain Round Groups as well as Rounds would allow adding the extra layer of organization where helpful.

[Torgen]

I'll point out that the test data from the 2011 hunt also has an unusual association of puzzles to metas, both in the Civilization round (where some technologies fed into multiple wonders) and in the Mega Man round, where the robot master metas fed into each other in a Rock-Paper-Scissors loop.

[Torgen]

One issue with your proposal: the Games island, where every puzzle fed into both The Desert and The Robber. Grouping every puzzle into both sets would blow up the size of the blackboard. Here's my hybrid proposal: There are two levels: Rounds (but the collection is called RoundGroups to make old data migratable) and Puzzles. Puzzles gain a boolean field called isMeta, the puzzles array from Rounds, and an array of puzzle _ids called feedsMeta. (I'd rather not have the denormalization, but MiniMongo doesn't have indexes and we're going to want to find metas by puzzle on the client side.) RoundGroups get a puzzles array instead of rounds, and a groupByMetas boolean. If groupByMetas is true, you get the rendering you describe, and puzzles that belong to two metas show up in both spots, with a virtual tag showing the other metas they belong to. Puzzles that belong to no metas in that RoundGroup can show up at the end. (Because puzzles can belong to metas outside their round group, to enable metametas and meta^3s.) If groupByMetas is false, all the metas show up at the top, followed by all the non-metas. The non-metas say all the metas they belong to. There would be a checkbox in each RoundGroup's header that sets something in SessionStorage; if it's set, the groupByMetas value is overridden. To convert from old data, you see if the round groups have rounds or puzzles. If it's rounds, you set groupByMetas to true, copy the puzzle fields from each round into a puzzle, set those puzzles as metas, and put their _ids at the start of the RoundGroup's puzzles array. Then for each puzzle in the round, you add it to the RoundGroup's puzzles array and add the meta's _id to the puzzle's feedsMeta array. Then you delete the Rounds. Sound like a plan?

[cscott]

I don't think it's worth optimizing for every crazy corner case. I can think of a number of reasonable ways of handling the robber/desert scenario: 1) just deal with the duplication, that's what the "hide duplicate puzzles" flag on the whiteboard is for 2) use the same meta solve room for both desert and robber, and record the answer as A / B 3) handle one of them as a "normal puzzle" with a manual meta:true tag set. Since the input set is all puzzles, you don't really need a separate enumeration of answers.

Grouping puzzles by round group is still useful; you seem to have lost a level of organization in your proposal.

Mongo has a very loose semantics for atomicity. The algorithm for inserting and moving puzzles into rounds were quite tricky since you can't do atomic operations in both the puzzle and the round object at the same time.

I'm also nervous about maintaining navigability: right now there's a breadcrumbs that goes blackboard => round => puzzle, and that helps remind you where in the hunt you are, especially when 'round' is 'Pokemon Island'. In your proposal 'round' isn't a full-fledged object; it would have to be replaced with a new page with a list of metas in that round, forcing folks through an extra level of indirection to get to a page where they could actually discuss the meta. (But in my proposal you'd have to have a drop-down for "round" in the breadcrumbs, or keep track of which of the duplicated puzzle entries you clicked through to get there, since there's more than one path from blackboard to puzzle.)

I think I could be convinced by your proposal, but: Keep round groups. Let them contain round groups or rounds. Rounds are no longer puzzles, and don't have dedicated chat or spreadsheets. 'groupByMetas' is a display preference, not stored in the db object. Rounds probably need to create a default hidden "no meta" puzzle that will contain every puzzle that does not otherwise have a meta assigned, in order to efficiently implement the display you describe.

Puzzles have the properties you describe. They don't need an isMeta property, it can just be "contains a nonempty puzzles array", although maybe the indexing by Boolean would be helpful.

You don't have to worry about converting from old data; we wipe the db between years. You can just run the older version of the blackboard software if you want to see old data. (It would actually be nice to have an "archive" routine we could run after each hunt to dump a static html view of that year's hunt, with chat room and oplog as a single static page, but that's a separate feature request.)

I'd also have to be convinced that we could update the puzzles array, the feedsMeta inverse array, and theMeta array atomically. In addition, we'd need an atomic update of the array of puzzles in round, the hidden "no meta" puzzle, and a puzzle to be moved, in order to handle moving a puzzle to a different round.

[Torgen]

Re Robber/Desert

just deal with the duplication, that's what the "hide duplicate puzzles" flag on the whiteboard is for

Does blaze give a good way to record that a puzzle was already rendered in this pass?

use the same meta solve room for both desert and robber, and record the answer as A / B

What does 'bot call in' tell whoever's manning the call-in queue? (This was also an issue with the Zelda round, I imagine. I think I was asleep for most of it.)

handle one of them as a "normal puzzle" with a manual meta:true tag set. Since the input set is all puzzles, you don't really need a separate enumeration of answers.

Then they don't show up in the table at the top, and for the robber, you did want them.

Grouping puzzles by round group is still useful; you seem to have lost a level of organization in your proposal.

My proposal still has the round group. If the metas for a round group are non-overlapping, it gives the same organizational ability as the current system. It just stores the rounds in the Puzzles collection.

The algorithm for inserting and moving puzzles into rounds were quite tricky since you can't do atomic operations in both the puzzle and the round object at the same time.

Which is the tricky part? Ensuring the list you're updating still has the old value in case someone modified it in the meantime when inserting in a specific spot? (Which neither the bot nor the UI let you do.) Or is it that all mutations return immediately on the client regardless of whether they will succeed on the server?

EDIT: just found how you move puzzles within a round by using addPuzzleToRound.

right now there's a breadcrumbs that goes blackboard => round => puzzle, and that helps remind you where in the hunt you are, especially when 'round' is 'Pokemon Island'.

In my proposal "Pokemon island" is the round group, but we can still put it in the breadcrumbs. We can also link the metas from the top left pane, and we can pull information from those metas. For example, you could set a tag on the Anger round that all answers are 9 letters and it would show up for all the puzzles in the round, including those that feed two metas.

The only part of the breadcrumbs I ever used was the Blackboard link, and I only used that when I switched sheets in the spreadsheet so often that the back button was impractical.

I notice both our teams made "Recover the Core Memories" a round group, and it was a mistake for both of us. It would be like if we had made a "Mario" round group and a "Games other than Mario" round group in 2011. But of course, before we saw what was on an island, we didn't know.

You don't have to worry about converting from old data; we wipe the db between years.

I may have been fooled by the 'if MIGRATE_ANSWERS' code in lib/model.coffee.

I'd also have to be convinced that we could update the puzzles array, the feedsMeta inverse array, and theMeta array atomically.

Could we denormalize with a server-side observer instead?

In addition, we'd need an atomic update of the array of puzzles in round, the hidden "no meta" puzzle, and a puzzle to be moved, in order to handle moving a puzzle to a different round.

The advantage of representing not being in a meta by having an empty feedsMeta field is that you don't need to operate on two metas at a time.

[Torgen]

The advantage of the isMeta property is that in edit mode, to add an existing puzzle to another meta, you could give it a dropdown of everything with the isMeta tag. If something is a meta iff puzzles feed into it, then the dropdown has to list every puzzle, or there's no way to be the first puzzle to feed into a meta. Or the meta needs to have a dropdown of every puzzle.

[Torgen]

How will the breadcrumbs work if a puzzle feeds into two metas? Should there be a dropdown to choose among them? If the meta feeds into a meta-meta, should that appear in the breadcrumb trail from the start? (The round group currently doesn't.)

[Torgen]

I sent a link and L/P for my staging server to the mailing list. I probably have a few changes to make before I merge it to my fork's master, and I think because it's so large and intertwined with my other changes, it will depend on multiple of my other PRs to get merged before I can make a cherry-pick PR for it--in fact I think I have another PR I want to send that's in that state that this one depends on.

Anyways, for a sample multi-meta case, the breadcrumbs work like:

Blackboard | Meta: Interstellar Spaceship | v 4 Metas | Puzzle: Technological Crisis at Shikakuro Farms

Where v is a down-caret and 4 Metas is a dropdown menu containing the 4 metas from the Civ round that TC@SF feeds into. If you navigate to one of them, it looks like e.g.:

Blackboard | Meta: Interstellar Spaceship | v + 3 | Meta: Granary of Ur | Puzzle: Technological Crisis at Shikakuro Farms

Re: atomicity, there are already a few cases in the blackboard where you can get a race condition--e.g. if I create a puzzle in a round at the same time as you're deleting the round, my puzzle can exist with no parent, but searching by canonical name will still find it. There's not really a way to fix that until Meteor catches up to MongoDB 4.0 and its multi-document transactions--we can't use them via rawCollection since even Meteor 1.7 doesn't use a Mongo driver recent enough.

[Torgen]

I'm experimenting with a version that uses jcbernack:reactive-aggregate to store the meta memberships normalized in the meta (so that order can be preserved) instead of denormalized in both the meta and the member. Not sure what the performance will be like.