Section 3.4 - Request for re-setting distributions

[ghost]

The question type distribution was reset recently. When this was done, the MA questions barely survived, while some quizzers specialized in them The new focus is much more (and in some opinions too much) focused on references. We would like to increase the number of MA questions, and reduce the number of questions relying on reference numbers. (The reference numbers first appeared in 1557, and may be considered not to be inspired.)

Suggested distributions, as used by NCD:

   N/E |  N/E

  Min | Max
FTV | 3 | 4 INT | 8/9 | 16 MA | 2 | 3 Q | 1 | 3/4 REF | 2 | 4 SIT | 2/0 | 4/0

(I'll have to check if that's correct for the Epistle years. Something seems odd.)

[JoshJetto]

I think the INT min and the Quote max is high by one question each for Epistle years. If you make this adjustment, then you will get a total min of 16 and a total max of 30.

[kclimenhaga]

I don't have strong feelings about the specific numbers. I do think that the current total min of 19 is too high. I think that having quizzes that are too predictable emphasizes strategy over content knowledge, and would prefer to see a min around 16 or so.

[scottpeterson]

Reference questions aren't important because the verse numbers are considered to be "inspired." They're important because they reward a level of depth and precision that isn't required on other types.

Wanting to reduce the number of Refs purely because the chapter/verse divisions aren't "inspired" is a red herring argument.

(there may be other reasons we want more/fewer Refs of course)

[scottpeterson]

I do find the competitive implications of reducing the total of the minimums below 19 very interesting. Less predictability absolutely means severe question type specializing carries more risk (which could be "good", because it encourages more full content?).

Though, one MASSIVE second order effect to consider is, the proportion of question types that DO get asked in those "up for grabs" questions (meaning, currently there's 1 "up for grabs" question since the minimums total 19), will be largely determined by the proportion of WRITTEN questions. Which MAY not be something we want.

This is why I'm not the biggest fan of CR and CVR lumped together under "Reference Questions." Usually the written proportions of these are ~60/40 CR/CVR, which results in 60% of asked Refs being CR. (maybe this IS desired, idk).

[kclimenhaga]

Reference questions aren't important because the verse numbers are considered to be "inspired." They're important because they reward a level of depth and precision that isn't required on other types.

I absolutely agree!

Regarding the second entry, that is a fair point about the proportion of written questions. However, would it not actually be determined by the proportion of extra questions in the maximums? I am not proposing changing the maximums.

I agree that splitting CR and CVR would be a good idea.

[scottpeterson]

Good point. It would depend on how the question set is generated. There are many ways you COULD pick a question type in a question set generating algorithm. And lots of implications in whatever you pick.

[JoshJetto]

I would agree with the idea of splitting the CR and CVR questions into their own separate place in the distribution, and with a more variable question distribution, I could see going 1-3 for both question types, along with going 1-3 for Quotes (N) and 2-3 for Quotes (E). I could also see us dropping the INT min to 6 in Narrative years in the proposed question distribution to arrive at a min of 16 for both Narrative and Epistle years. (See Proposed Question Distribution in issue "3.4 Question Type Distribution - on behalf of Josh Jetto").

As far as the variability goes in terms of question selection beyond the MIN levels, I do think that if there is a way to weight that in the question generator to match the total percentage of questions over the MIN for each question type, that might be the fairest way to go about it. So, of the 14 possible questions above the MIN in a regulation quiz, INTs comprise 8, FTVs 1, Q 2, CR/CVR 2, MAs 1. And actually, I think in terms of the A/B questions, that already happens since the quiz generators typically have 30 questions total in each quiz generated, which takes each question type up to its maximum, and then it's somewhat random within those probabilities as far as whether those questions will actually get used in the quiz based on what questions are answered incorrectly from question 16 on. So, now that I've typed this, I think that since this is already being done at the question generating level, we shouldn't try to change this to somehow make it less random.

Although it is true that CVRs require a level of precision that others do not, they require a level of precision to a prompt that is not a part of the actual text of Scripture and one that for many (probably the majority of people) is not natural, whereas prompts connected to what the Scripture actually says are for the majority of people more natural. From the standpoint of the authors of the New Testament, they quote directly from the Old Testament with mainly general references - "as it is written" - and only get as specific as naming the prophet who wrote what they are quoting. The references are not essential; the text is, and biblically-speaking the references are not a part of the Bible. Additionally, CVR and CR questions tend to be vague "He will what? They will what? Do not what?" And so, the questions themselves give very little text-based information to the average quizzer (or even an International-level non-reference quizzer) in order to be able to answer the question correctly - even when given the whole question. The current, reference-heavy distribution strongly (and I believe wrongly) advantages the reference specializers over the rest of the competitive field. It doesn't make the reference specializers know the material better than they would have known it already, and it eliminates the non-reference quizzers from more of the questions - making quizzing competitively easier for reference quizzers and less motivating for non-reference quizzers, since they are effectively unable to jump on a significant percentage of the questions. And I would argue that by making quizzing competitively easier for reference quizzers, it lessens their motivation to know the material better as well. My concern and guess (though I can't more than anecdotally say) is that the fruit of the current (since 2017), reference-heavy distribution has not been more memorization by more quizzers at the District level, but more unjumped on questions, and a less fun and motivating quizzing experience for the majority of Bible Quizzers.

[JoshJetto]

I should clarify my above response regarding CVR and CR questions is that the greater problem of non-text prompts lies with CVR and not CR question types, though CR questions are still generally more vague than I like questions to be, and often not very helpful to the quizzer from a text-based prompt standpoint.

[kclimenhaga]

In terms of the "reference-heavy" weighting, I believe the only change from the previous weighting was to increase the minimum number of quote questions to 3 from 1. Reference questions had a minimum of 3 in the previous weighting. There are more reference questions in the As and Bs now, but not in the numbered questions. I could see the value in increasing the number of FTVs at the expense of Qs, based on your reasoning, but not of lowering the number of reference questions. INTs can also be equally guilty of being vague, so reference questions can sometimes be easier to answer, especially if quizzers wait to hear the full question. At the IBQ level, of course, the full question will never be read, but I don't think the distribution needs to account for that.

Anecdotally, in my district (which uses the IBQ distribution), I have not seen more sat-on questions since the distribution was changed.

[jswingle]

I'm closing out this identical issue and linking to the discussion:

https://github.com/gryphonshafer/Quizzing-Rule-Book/issues/108

Please note that there is a pretty hefty discussion thread on Issue #108, so please read through the posts therein to be fully informed on this issue.

[jswingle]

Alright, here's my crack at a preferred distribution. Numbers with a / are in Narrative/Epistle format, as it's formatted in the rulebook. CR includes MACR, CVR includes MACVR, and Finish/Quote include all subtypes.

TYPE ---- MIN ----- MAX INT ------- 7/8 ------- 12/14 MA -------- 2 --------- 3 CR -------- 2 --------- 3/4 CVR ------ 2 --------- 3/4 Q ---------- 1/2 ------- 3 F ----------- 3 --------- 4 SIT -------- 2/0 ------ 4/0

Explanations: (1) slightly decreased number of quotes, while maintaining giving them a slight priority on epistle years (2) corrected the overcorrection on MAs. I think 2-3 is exactly perfect, 1 is too few and 4 is too much. There are usually only ~150 good MAs in a given quiz year so we don't need to ask 4 in a quiz (3) finding a way to work in the CR/CVR split. Formerly references had a min of 3 and a max of 6/7. In this proposal, they have a minimum of 4 combined, but only in rare circumstances during epistle years will ever go above 6 (4) maintained the current rulebook's minimum requirements totaling 19 for both narrative and epistle years. I like this setup, as it allows 1 question from 1-20 per quiz to be a "wild card" while allowing a lot of latitude for the alphanumerics and overtime. If you count it up, you'll see that both narrative and epistle still add up to 19. To accomplish this, I had to reduce the INT minimum by 1. (5) I also reduced the INT maximum by quite a lot because 14/16 was overkill. This makes A/B questions WAY MORE likely to be INTs than they ought to be. The total number of max questions used to be 33, but I decreased it to 32 -- this isn't a problem, 30 is already the maximum number of questions that can be asked in a quiz so 32 is still very much unpredictable even when all As/Bs are used

Would definitely like as much feedback and discussion as possible on this - @gryphonshafer @scottpeterson @ZacharyTinker @JoshJetto @jttower @kclimenhaga @josiah-leinbach @ARMediting

[kclimenhaga]

I like most of this. In terms of where we differ, here are my thoughts: -If possible, I would really like to see the distribution the same in all years, obviously with the exception of situation questions. I think the consistency makes it easier to remember, and I don't like a distribution that is too complicated to remember. If others disagree in terms of the minimums, could we at least make the maximums the same every year? I don't see a downside to having a higher number of max questions. -I think one quote question per quiz is too few. I would rather see either 1-3 MAs or a minimum of 6 INTs than only one quote. I don't think they need to be as high as FTVs, but one is too few. If we're saying we need more than one MA because some quizzers specialize in them, then having only one Q also doesn't work.

Other than that, I think the distribution looks good.

[gryphonshafer]

If possible, I would really like to see the distribution the same in all years, obviously with the exception of situation questions.

This. I totally agree, but not because it'll make things easier on us veterans, which of course it would to small degree. Rather, it'll reduce some amount of rule book complexity and thus the associated barrier to entry for new participants, the inverse of which is counter-mission.

Historically, type distribution didn't change between epistle and narrative material seasons. The SIT type was originally set at 0-4, and since humans built quizzes instead of algorithms, narrative seasons nearly always included a minimum of 2 and a maximum of 3.

If we have a singular distribution for both epistle and narrative material seasons (SITs being the exception), does the value of the added complexity of multiple distributions exceed its "cost"?

[jswingle]

@kclimenhaga I think the downside to having too many "max questions" is that the As/Bs at the end of the quiz can easily become dominated by one single question type.

I don't have a diehard opinion on the 1 quote min vs. 2 quote min, but I do tend to agree with @JoshJetto 's opinion that the current type distribution overemphasizes references beyond their actual importance. I would rather have more Finish than Quote, is what I'm trying to say -- and I would rather emphasize SITs during narrative years if we can, since I think they are spiritually significant content for those material years. Again, just my preference, not a super strong opinion.

[jswingle]

Since there's discussion over differing type distributions on Narrative vs. Epistle years, I may as well point out that I think we should add a question type to the epistle years which requires quizzers to identify the Old Testament scripture that a quotation comes from. It would basically be the epistle version of a SIT question.

For example, the "You have made him a little lower than the angels" passage in Hebrews, the quizzer would have to identify that it comes from Psalm 8. And would have to finish the quotation if they prejump.

This obviously belongs in a giant separate issue but you know... worth thinking about.

[jttower]

For example, the "You have made him a little lower than the angels" passage in Hebrews, the quizzer would have to identify that it comes from Psalm 8. And would have to finish the quotation if they prejump.

@jswingle how many of these type of questions would be able to be written in various Epistle years, and how would quizzers know where these quotes are from, would they all be from quotes that are footnoted in a regular NIV Bible? I think this is an intriguing idea, and could solve the differing type distributions question for both years. (This has become an iceberg-type issue!)

Would it be worded something like "SIT question, where in the OT was it said. 'You have made..."? I haven't looked into this, but is it possible we could run into issues where it is quoted in more than one place? Would these questions be formed based on which quotes are footnoted in the NIV, or would there be another way of identifying these? Just wondering how quizzers will be able to study such things since they aren't marked in the quiz books.

[jttower]

This is why I'm not the biggest fan of CR and CVR lumped together under "Reference Questions." Usually the written proportions of these are ~60/40 CR/CVR, which results in 60% of asked Refs being CR. (maybe this IS desired, idk).

I guess I'm not understanding this argument. From what I've read, it sounds like CRs are preferred to CVRs because the verse numbers are not included which makes them less like quote questions. If there are already a higher proportion of CRs in any given question set, then those are more likely to be asked. And I thought some of this discussion started because people wanted less REF questions, not more. If we split the CR/CVR question types, then there will be more REF questions asked than in the current distribution (Current 3-7, with Jeremy's it would be 4-8 combined).

Regarding @jswingle distribution, I agree with @kclimenhaga that the number of quotes is too few. I laid out my arguments in closed issue #108. I suggest this distribution for several reasons:

1. It keeps the number of min/max questions the same every year (with the exception of adding in SITs).
2. It maintains the level of quotes and doesn't overemphasize them in Epistle years.
3. It reduces the number of REF questions.
4. It corrects the over-correction of MA questions.
5. It reduces the MIN levels which adds to more variability in a quiz, which it sounds like people want.
6. I think it evens out the "specialty" questions more.

Narratives / Epistles INT 7-15 / 7-15 FTV 3-4 / 3-4 Q 2-3 / 2-3 REF 2-4 / 2-4 MA 2-3 / 2-3 SIT 2-4 / 0

Tot: 18-33 16-29

Thoughts?

[ARMediting]

My quick-response thought is that a potential of 8 total reference questions in Jeremy's proposal is too many, but a potential 2 total of Jessica's proposal is too few. I think I'd prefer 2-3 CR and 2-3 CVR.

[ARMediting]

Also, I wanted to respond to something @JoshJetto said in #108:

And unlike CVRs, nobody (really) specializes in INTs, because there are far too many of them (and because, unlike CVRs, they can be asked in such a wide variety of ways that it would be a waste of time to try to learn all of the possible text-prompt points to do that). In other words, INTs to MAs is not the same as CVRs to CVRMAs such that you could say to an International quizzer (or one at the local or District level), “Say, while you’re working on specializing in INTs, why don’t you just pick up the MAs at the same time.”

I have specialized in INTs. Studying all the possible starts is nearly impossible, but I studied unique words, two-word phrases, and looked at what kinds of phrases were more likely to come up in W starts. Plus just writing and practicing a ton of them. All of this would translate well to also studying MAs. It seems natural to me to ask a quizzer to study both because they both require the same level of accuracy in answering and a strong overall knowledge of the material.

[jttower]

If we split up CR/CVRs and do 2-3 for both, then we could arrive at a max of 6 REF questions, which puts us at a higher number of REF questions than the 2013 Distribution. Which brings me back to the question, wasn't this issue raised to reduce the number of REF questions?

Above, @JoshJetto had suggested 1-3 each of CR/CVRs, which gives a combined 2-6 REF questions. It sounds like @ARMediting would like more than 2, and I thought @JoshJetto wanted less than 6, since his first proposal had a max of 4.

• We could leave them combined and do 3-4 REF questions which brings the MINs to 19/17 and keeps the MAXs the same in my proposed distribution above.
• We could split 2 CR and 2 CVR, but that brings the Narrative MIN to 20, which no one probably wants. I am not in favor of reducing any of the other MINs in order to accomodate this, because I think 6 INTs is too few and I don't want to go below 2 of Q/S/MA and not less than 3 FTVs. Because of this, I'm not sure how else to accommodate the CR/CVR split.

[kclimenhaga]

If the CR/CVR split is too cumbersome, would it work to say 3-4 REF questions, but specify that at least one will be a CVR and at least one will be a CR? I'm not sure what that would do to quiz generating programs, but it might solve some of the problems we are having with distribution.

[jttower]

I asked Ted about his Quiz Generation program on QuizNet (https://trueword.freehostia.com/), and he is confident he could make it work to have 3-4 REF and require at least one of each to be a CR/CVR.

[jswingle]

@jttower I have created a new issue which tackles the "OT quotation" idea. I know @JoshJetto has also expressed interest in it: Issue #167

If this sort of idea gets implemented, it would mean a lot of changes to the distributions, so for now I'll refrain from further tweaking/discussion on this issue...

[JoshJetto]

I like @kclimenhaga's idea on the CR/CVR piece with 3-4 and specifying that there will be at least one of each in those 3. This would be a case in which I would then be in favor of having 2-3 Q as the Quote distribution each year as it provides a minimum of 3 verse-reference questions for the reference specialists. If we did split CRs and CVRs, then I think 2-3 in both would be the way to go and 1-3 for Q (all years). In both cases, this yields a minimum of 3 verse-reference questions for Q/CVR specialists and makes verse-reference questions on average 15% of the numbered questions, which I think is the right balance.

[24RMiller]

As others have mentioned, I feel that we should have a minimum of 2 MAs, in exchange for decreasing the INT minimum by 1. I believe there would still be plenty of multiple answer questions to choose from, and I don’t see the reason why we have 7-8 interrogatives and 1 MA. MAs are(in my opinion) simply a better version of an interrogative that often requires a slightly longer answer.

[ARMediting]

@kclimenhaga I think the downside to having too many "max questions" is that the As/Bs at the end of the quiz can easily become dominated by one single question type.

Since I really like the idea of having the same distribution for all years (minus SITs), what if we just make mins and maxes apply only to the 20 base questions, and write separate distribution rules for As and Bs? Here's my proposal, which slightly modifies @jttower's:

1-20, numerical only: Narratives / Epistles INT 7-10 / 7-10 FTV 3-4 / 3-4 Q 2-3 / 2-3 REF 3-4 / 3-4* MA 2-3 / 2-3 SIT 2-4 / 0

Totals: 19-28 / 16-24 *At least one CR and one CV

As and Bs: 1 of each F, Q, MA, CR, CV, S 4/5 INT

I like having a few more references to still make those worth studying, and we simply don't need as many INTs filling out the distribution. Is this proposal possible to encode in QuizNet, etc?

[jttower]

Ted said he could rewrite the code for QuizNet to fulfill those parameters if that's what we decide.

[josiah-leinbach]

Reducing the INT minimum by one, increasing the MA minimum by one, and guaranteeing at least one CR and CVR respectively would be the most conservative changes to the distribution that still accomplish most major goals. Reducing the INT minimum from 9 to 8 in an Epistle year will not make that much of a difference in a given quiz, certainly at the local level. The same is likely true at the International level. However, increasing the MA minimum from 1 to 2 all of a sudden makes MAs a specialty worth specializing in, where the payoff could actually be worth the time spent studying for them; and that specialization is what gives them the potential for a higher accuracy compared to INTs. A higher accuracy leads to more correct answers, a good thing.

[AliyaStrasburg]

Increasing MAs seems like a good idea, especially since some people do specialize in them and there seem to be a lot of INTs challenged because they were technically MAs. So long as there are plenty of INTs, I'm good with whatever! 😊

[24RMiller]

After discussion with fellow quizzers, I have reached the following conclusions:

1. MAs as a question type should be removed. They should be added into the question lists with INTs. As a result, you could get 5 MAs in a quiz, or you could get 0, but they are all announced as Ints. (One option I have thought of would be to guarantee at least one MA is still used, although I am not a huge fan of that)

2. Each quiz should be guaranteed to have one CR and one CVR, with every subsequent reference having a 50% chance to be either type. This means if even there have already been 3 CRs and 1 CVR, the next reference question still has a 50/50 chance of being a CVR or CR.

3. As a result of these changes, the minimum for INTs should be increased by 1, and the maximum by 2, to reflect the mins and maxes of the MAs.

4. I have seen others mention that increasing the MA minimum allows for more specialization, but my question is: Is that a good thing? Shouldn’t people be spending their time memorizing the material, rather than specializing in a question type?

5. Also, I realize I said the exact opposite 3 months ago, but in discussing this with other coaches and quizzers, I have changed my mind in regards to the question breakdown of quizzes.

[EfishWGLD]

Having looked at all the proposed question type break downs and read through the thread (minus Issue #108), this is my personal conviction.

1. MAs: I think MAs are a necessary question type. They provide an in for many International quizzers and for competitive teams at the district level. Specializing in a question type, while not explicitly spiritual, does accomplish a lesser-mentioned quizzing goal- teamwork. A well-oiled team has quizzers where each can do anything asked of them- but that don't need to. Rookie or time-constrained International quizzers need question types that help the team succeed, are more manageable than memorizing full content, and have a reasonable impact. In recent years, MA specialization has become a less legitimate specialization, making assigning question types more difficult. I think adding "2" minimum MAs and "1" less INT is very reasonable for this reason.

2. Reference Questions: References were and are very important to me, in and outside of quizzing. Only in the last few years have I began to to truly memorize references, and it makes using Scripture so much quicker, easier, and reliable. If I am witnessing to someone, I would infinitely rather reference "In John 3:16, etc., etc." than "Somewhere in the book of John, it speaks of God's love for the world." Memorizing references makes for easier application post-quizzing. Within quizzing, references made memorizing easier for myself to clearly demarcate the verses I was memorizing, made me study harder on the whole, and created an easy method to practice (a Quote vs. an FTV is easier to quiz yourself on). I was a reference specialist for much of my quizzing career (I sound old, yikes), and CRs and CVRs were both the hardest and most rewarding questions to answer. Few things are more effective at making one learn Scripture verbatim, word for word, than CVRs, which is a central goal of quizzing. For that reason, I think that a "2" Quote minimum and a "3-4" or "3-5" CR and CVR spread, with one CVR and one CR guaranteed per quiz, should be implemented to both legitimize the specialties for each respectively and allow for the weight that reference questions should hold.

And thanks CQLT for all the work you guys do! We appreciate you all very much.

[LukeBraisted]

Quotes: While it is true that the reference numbers are not inspired, they are useful for finding verses within the inspired text. The ability to locate verses is a valuable skill to have when teaching others or defending your beliefs. Additionally, quote questions require strong knowledge of the text of the Bible and not just the reference numbers. I think that we should have a minimum of two quotes in both narrative and epistle years.

Reference Questions: The skills required for CR and CVR questions are quite different. Guaranteeing at least one of each type would keep quizzes more consistent and fair. CRs and CVRs both serve valuable purposes, but I understand how they can be considered less important than some other types. I support the suggestion of a minimum of three REF questions, with at least one CR and at least one CVR.

Multiple Answer Questions: MA questions tend to encourage studying lists rather than studying the material directly. Combining MA and INT questions would simplify the rules and prevent confusion or challenges resulting from debate over whether an answer is single or multiple. MAs and INTs seem similar in both structure and purpose, so I agree with the proposal to remove the MA type and ask questions that have two or more answers as INTs. Along with this, I think CRMAs and CVRMAs should be announced as simply CRs and CVRs.

Narrative/Epistle Years: A similar question distribution for narrative and epistle years would be easier to remember and make the rules simpler. Aside from SIT questions, I see no need to emphasize certain question types in different years. I think it would be best to have the same minimum question distribution in both narrative and epistle years.

INT and Total Minimums: While some consistency is good, I do see value in having less predictability by lowering the total minimums. For this reason, I would prefer to have a minimum of eight INTs (if MAs are combined with INTs). However, if others consider eight of them too low, another INT could be added.

Maximum Numbers: I think it would be good if the maximum numbers of each type were proportionate to the minimum numbers and similar for both narratives and epistles, with a total maximum close to 30.

The following distribution makes sense to me, but I am open to other suggestions:

Narratives / Epistles INT 8-15 / 8-15 FTV 3-6 / 3-6 Q 2-4 / 2-4 REF 3-5 / 3-5 SIT 2-4 / 0

Total: 18-34/16-30

[kclimenhaga]

Regarding re-labelling MAs as INTs:

There are some MAs that would not be valid as non-MA questions, because the two answers are not in the same place in the verse. For example, the CVRMA: "According to Luke 2:4, what town?" The verse reads:

So Joseph also went up from the town of Nazareth in Galilee to Judea, to Bethlehem the town of David, because he belonged to the house and line of David.

If this is asked as a CVR, it is not valid, because there are two possible answers to the question. We could rewrite the description for INTs to allow for this, but I don't see a straightforward way to word this description, and quizzers would have no way of knowing that they need to provide multiple pieces of information. Internationals-level quizzers would figure it out, but these questions would be challenging for newer quizzers. So we would realistically need to throw out questions like this. It isn't a huge number (about 40 in my Acts question file, most of which are reference questions), but it's not insignificant. I also want to keep MAs for the reasons others have specified regarding specializations and accessibility for newer quizzers, but this is another thing to consider.

I don't see any reason that we couldn't include MAs as part of the INT distribution, but still announce them as MAs (similar to how CVRMAs and CRMAs are included in the reference distribution). I would prefer to keep them separate, but I'm not as strongly opposed to this idea as I am to getting rid of MAs entirely.

[jttower]

Trying to summarize people's suggestions here for ease of discussion, I see a few options:

Even out specialty questions (Jessica's proposal): Narratives / Epistles INT 7-15 / 7-15 FTV 3-4 / 3-4 Q 2-4 / 2-4 REF 2-4 / 2-4 (with a minimum of 1 each of CR/CVR) MA 2-3 / 2-3 SIT 2-4 / 0

Total: 18-34 16-30

---

Combine MAs into INTs (Luke's proposal): Narratives / Epistles INT 8-15 / 8-15 (with a minimum of 2 each of INT/MA) FTV 3-6 / 3-6 Q 2-4 / 2-4 REF 3-5 / 3-5 (with a minimum of 1 each of CR/CVR) SIT 2-4 / 0

Total: 18-34/16-30

---

Separate out numerical questions from As/Bs (Alex's proposal): 1-20, numerical only: Narratives / Epistles INT 7-10 / 7-10 FTV 3-4 / 3-4 Q 2-3 / 2-3 REF 3-4 / 3-4 (with a minimum of 1 each of CR/CVR) MA 2-3 / 2-3 SIT 2-4 / 0

Total: 19-28 / 16-24 (max with As/Bs would be 38/34)

As and Bs: 1 of each F, Q, MA, CR, CV, S, and 4/5 INT

[JoshJetto]

So here's my crack at combining @jswingle's distribution with @kclimenhaga's. I am actually now in favor of a tighter MIN than before, mainly because it keeps CVR and Quote questions from becoming too high a percentage of the total number of questions (and also because it made question tracking more meaningful at Internationals as a coach). I am going to list two distributions here. The first would be more in keeping with our norms. The second would allow the A/B question distribution to be more similar to the numeric question distribution.

INT 7-15 / 8-16 MA 2-3 / 2-3 REF 3-4 / 3-4 Q 2-3 / 2-3 FTV 3-4 / 3-4 SIT 2-4 / 0 Tot: 19-33 / 18-30 *In this case A/B questions would be 8 INT and 6 Specialty (Narrative) and 8 INT and 4 Specialty (Epistle) (a bit INT heavy given our numeric question distribution).

Alternative Distribution: INT 7-14 / 8-14 MA 2-4 / 2-4 REF 3-5 / 3-5 Q 2-3 / 2-3 FTV 3-5 / 3-5 SIT 2-4 / 0 Tot: 19-35 / 18-31 *In this case A/B questions would 7 INT and 9 Specialty (Narrative) and 6 INT and 7 Specialty (Epistle). I would even be okay with dropping the MIN on INTs in Epistle years to 7, making the MIN 17 and the MAX 30 and making the INT and Specialties even on the "wild-card" numeric and A/B questions.

I think for me, the beauty of this second Distribution is that it does provide for more variability on the A/B questions and increases the likelihood of an additional specialty question across the board of specialty questions for the "wild-card" numeric questions. And it does this without making the CVR + Quote percentage likely to rise above 20% of the questions in any given quiz, which in my mind, 15-20% (3-4 questions out of 20) is the ideal percentage emphasis for questions that will be jumped on the verse number at Internationals and that will be asked at the District level.

[kclimenhaga]

I like the second of @JoshJetto's proposals except that I don't like having a different minimum of INTs in narrative vs. epistle years. Could I amend it slightly to:

INT 7-14 / 7-14 MA 2-4 / 2-4 REF 3-5 / 3-5 Q 2-3 / 2-3 FTV 3-5 / 3-5 SIT 2-4 / 0 Tot: 19-35 / 18-31

I think this is a good balance of questions, and does give variability on A/B.

I think previous distributions have over-emphasized INTs because there are so many of them, but that doesn't make them the best or most important question type. I'd be happy seeing the minimum INTs drop down to 7.

[Daniel-Braisted]

I have a different opinion on REF questions, one that I haven't seen suggested here. As someone who specialized in CVRs one year and CRs another, I can confirm that they're completely different question types. They're both questions based on a non-unique phrase, but the skill set is completely different between the two. For a CR, all you often need is a general idea of where something is in the material, so you don't need verse references or even word-perfect knowledge. Even at the international level, you answer a CR based on the question that's asked. On the other hand, CVRs require quizzers to know the exact references within a chapter. At Internationals, CVR specialists usually jump on the reference and "quote backwards" through the verse until they're asked for the question. The difference in skill sets isn't just at the International level, though: at the district level, I've seen many quizzers who are willing to jump on CRs, but not on CVRs since they don't know the references.

Because of this, I'm in favor of completely separating the two in terms of distribution. The difference isn't like the difference between Q and Q2V; it's more like the difference between Q and FTV. Having a system of "3 with at least 1 of each" is better than what we have now, but it's more complicated than the distribution we have for any other question types (needlessly so, in my opinion). A quizzer who specializes in one type of reference question isn't going to be good at the other type by default, so reference specialists might only get 1 of their question type in a quiz, which is exactly what we're trying to avoid for MA specialists.

So in the interest of simple rules that better reflect the way REF questions work in practice, I would propose that CRs and CVRs be treated as separate question types, each with a minimum of 2 per quiz. I'm open to different overall distributions that have been proposed so far, so I won't propose a new total distribution, but both of @JoshJetto's proposals have a REF minimum of 3, so 4 (2 of each) isn't that far off from what people are already leaning towards.

[jttower]

What if we do less INT and then separate the REF questions like this:

INT 6-11 MA 2-4 CVR 2-3 CR 2-3 Q 2-4 FTV 3-6 SIT 2-4 / 0 Tot: 19-35 / 19-31

[jttower]

At our meeting on February 19, 2023, the Rules Committee voted to adopt the following question distribution:

Narrative/Epistle INT 7-14 / 7-14 MA 2-4 / 2-4 REF 3-5 / 3-5 Q 2-3 / 2-3 FTV 3-5 / 3-5 SIT 2-4 / 0 Tot: 19-35 / 17-31

At least one of the reference questions will be a CVR or CVRMA, and at least one of the reference questions will be a CR or a CRMA.

---

We are very thankful for everyone's input and discussed all suggestions. We wanted to make the distribution the same across Narrative/Epistle years with the exception of adding in SIT questions. We fixed the over-correction of MAs. We ultimately decided against splitting REF questions because everything we tried skewed the distribution.

[jttower]

On behalf of the Rules Committee:

Since the Rules Committee has voted to approve the change detailed above, a "pull request" will be made - in other words, a change has been officially proposed and voted on by the Rules Committee, and will be given a final vote in 3 months.