Open Theta91 opened 9 years ago
mbentley00
commented
9 years ago I'll be implementing some of this with the packetization refactoring, but probably not all of it. I don't think that making distribution entry super easy is a very high priority scenario. It's not something that people are going to need to do very frequently and so long as there's an okay way to get it right I'd rather focus on things that you do hundreds of times per set than just once.
Theta91
commented
9 years ago In my experience w/ QEMS1.0, it's a pretty high priority. Even a slight change requires a brand new distribution for a set -- it's also one of those things that should not be hard. For example, specifying a distribution is pretty easy and you should be able to do that in QEMS2 pretty easily, but right now it's actually quite literally 100x more intensive to input an entire distribution than to specify it. Though not nearly as intensive, QEMS1.0 had this same problem and it led to a lot of knock-on effects, crazy stuff, and pulling hair.
So, I guess, the question is what do you think isn't important above? I'm willing to revise or argue for if given specifics.
And, this is not to say that everything above needs to be implemented immediately as there are certainly some more pressing concerns in different areas. But I think it's better to have it right in the end.
Theta91
commented
9 years ago The biggest things that aren't implementation specific that I think should be had here are: subcategories have a semantic relationship w/ a TLC and don't exist outside of it; percentage specification for subcategories but number specification for TLC; combining TLCs to form units of 1 question; default TLCs and subcategories associated with them so they don't have to be entered fresh every single time.
mbentley00
commented
9 years ago Some comments:
There needs to be a standard set of top-level categories (TLCs) from which you can select. There should be no manual typing of a category name unless it is not present in the system. This should take the form of a drop-down box, alphabetically sorted.
Agree. UI needs to be made for this.
To start, TLCs present in the system should include Literature, History, Science, Fine Arts, RMP, Geography, Current Events, Trash, and Social Science. One might even separate RMP into individual categories given the differences in how sets have been handling the category.
Sure, we can pre-populate.
A TLC is given a min/max on tossups and bonuses per packet as in the current distribution entry screen. ONLY the TLC. There should be no "Subcategory" box next to it.
I'm treating all types of categories the same. They're going to have min tossups, max tossups, min acf bonuses, max acf bonuses, min vhsl bonuses, max vhsl bonuses, min total questions and max total questions. These are all per-period requirements. You set the total number of questions in a separate field on the distribution screen.
Instead of a "Subcategory" box, there should be a "Manual Entry" link that turns the dropdown box into a text field, so that you can specify a custom category name. (clicking "Manual Entry" is then replaced with "Pre-configured Entry" which brings back the dropdown when clicked.)
It would be great if this was interactive like this in the future. V1 I think is a separate screen where you can add a new category or a new child (i.e. add a sub category or sub category). On that separate screen you choose from existing cat/sub/subsub or can create a new one.
As now, the distribution entry screen should start with 1 line of entry; there should be a plus button between the line of entry and "Submit changes" that performs "add a new category" -- this should allow you to specify a new TLC.
I think there should be a + button next to every existing category and subcategory so you can add children to it.
Every TLC entry should have a plus button to the left of the category name that performs the function "add a subcategory"
Agree. This works the same way for a sub-category (but not a sub-sub category since we are only letting you go 3 deep).
All of the standard TLCs should have an associated set of standard subcategories. For example, in Literature, this would be "American", "British", "European", "World".
Yes, we can pre-populate this.
Pressing plus on a standard TLC automatically creates all its standard subcategories, which can be removed if so desired.
I think the way this is solved is by having the "add new category/subcategory/subsubcategory" dialog let you multi-select from existing items.
The subcategory distribution for a TLC is specified by % for tossup and bonus. Whether there should be a max/min % I haven't thought about, but I'd lean yes.
Disagree. It's just too much code and logic to have a separate way of calculating things for categories compared to subcategories. We can have some UI that shows what the percentage ends up being to make it easier for users entering new distributions. And distributions are already not completely generic since you need to give it a total number of questions. We could do everything in percentages but that poses its own problems with rounding and guaranteed numbers of questions.
There should be no instance where summing a subcategory's needs does not yield exactly its TLC needs.
Agree
There should be no instance where you have extra questions or less questions than you need when constructing packets. This means packet templates with filled-in needs need to be generated whenever the packet number for a set is changed.
Right now I'm checking if you meet the minimum total number of questions for all categories. If you don't, you get a warning. It can optionally fill in the set with placeholder questions. You would either edit these questions with real questions or delete them and manually add new questions in their place (Although I need to think through this latter scenario some more, it might cause problems).
I think my plan for now is that if you change the properties of a distribution, then you will have to rebuild the set requirements and repacketize. If you had filler questions in your set, ideally we'd delete them before the next packetization and prompt you again if you still need them.
At some near-future point, support needs to be added for combinations of standard TLCs. For example, combining Geography, Current Events, and Trash to form 1 question per packet. Clicking plus by a combined TLC automatically creates subcategories for all the TLCs combined into 1 TLC. This is an extremely important feature as, near-universally, TLCs should occur in units of 1 question per packet (which is not the case for subcategories, obviously). This then eliminates most of the need for the "Set-wide distribution" page.
So I think the way that I'd like to solve this is by just allowing for fractions in all types of categories. And you can combine this with maximum number of questions per packet to ensure that it doesn't pick 2 geography questions for one round and 2 current events questions for the next round.
mbentley00
commented
9 years ago Also, regarding the comments about top-level categories being integer based, that isn't going to work for certain formats. For instance, an ACF packet might have 1 tiebreaker question evenly distributed from the rest of the set. So you'd need to have something like 0.1 literature tossups per tiebreaker period.
Theta91
commented
9 years ago A very long response:
Some comments:
There needs to be a standard set of top-level categories (TLCs) from which you can select. There should be no manual typing of a category name unless it is not present in the system. This should take the form of a drop-down box, alphabetically sorted.
Agree. UI needs to be made for this.
In retrospect, putting all this in 1 issue was a very bad idea. I'm going to create a few separate issues for beta / rc-1 integration, like this one. I'm partial to UI stuff, so I will work on this down the line.
To start, TLCs present in the system should include Literature, History, Science, Fine Arts, RMP, Geography, Current Events, Trash, and Social Science. One might even separate RMP into individual categories given the differences in how sets have been handling the category.
Sure, we can pre-populate.
Excellent.
A TLC is given a min/max on tossups and bonuses per packet as in the current distribution entry screen. ONLY the TLC. There should be no "Subcategory" box next to it.
I'm treating all types of categories the same. They're going to have min tossups, max tossups, min acf bonuses, max acf bonuses, min vhsl bonuses, max vhsl bonuses, min total questions and max total questions. These are all per-period requirements. You set the total number of questions in a separate field on the distribution screen.
Thoughts for the far future (as in rc-1 future). Ideally, we would transition to having a two-form distribution entry screen. An upper portion defining the name of the distribution, number of questions in each period, and the question types included in the distribution. Submitting this refreshes the lower portion which is our current table (w/ separate submit button) w/ the category entry formset. Conceivably, this could be implemented as a UI thing: min/max tossups/bonuses/vhsl bonuses are simply not shown and set to 0 in the Distribution model (or wherever this is now -- PeriodWideEntry?).
Instead of a "Subcategory" box, there should be a "Manual Entry" link that turns the dropdown box into a text field, so that you can specify a custom category name. (clicking "Manual Entry" is then replaced with "Pre-configured Entry" which brings back the dropdown when clicked.)
It would be great if this was interactive like this in the future. V1 I think is a separate screen where you can add a new category or a new child (i.e. add a sub category or sub category). On that separate screen you choose from existing cat/sub/subsub or can create a new one.
I'll start a new issue for this.
As now, the distribution entry screen should start with 1 line of entry; there should be a plus button between the line of entry and "Submit changes" that performs "add a new category" -- this should allow you to specify a new TLC.
I think there should be a + button next to every existing category and subcategory so you can add children to it.
Agree.
All of the standard TLCs should have an associated set of standard subcategories. For example, in Literature, this would be "American", "British", "European", "World".
Yes, we can pre-populate this.
That's great news.
Pressing plus on a standard TLC automatically creates all its standard subcategories, which can be removed if so desired.
I think the way this is solved is by having the "add new category/subcategory/subsubcategory" dialog let you multi-select from existing items.
Yes, that's fantastic.
The subcategory distribution for a TLC is specified by % for tossup and bonus. Whether there should be a max/min % I haven't thought about, but I'd lean yes.
Disagree. It's just too much code and logic to have a separate way of calculating things for categories compared to subcategories. We can have some UI that shows what the percentage ends up being to make it easier for users entering new distributions. And distributions are already not completely generic since you need to give it a total number of questions. We could do everything in percentages but that poses its own problems with rounding and guaranteed numbers of questions.
Perhaps we can have a separate issue w/ some technical discussion of what is feasible here and how to implement it (hopefully including Jerry). I think it's very important that a distribution model take subcategories as percents of categories as this is the best way to scale across packets and eliminate some of the issues w/ category entry that were present in QEMS1 and that are frustrating about QEMS2. As an application example, VHSL requires 30 tossups while ACF requires 20. However, HSAPQ's distribution, for instance, changes across levels of competition, not by set type. So things like changing Science to 5 questions per packet for VHSL shouldn't necessitate recalculating all of the subcategories. I imagine this is also something needed by NHBB. For a universal application example, percentages allow a much finer control of the questions in a set without getting into decimal representations -- they are more human friendly.
Along with the associated trappings needed for Distribution/Period entry, here is how I envision the structure of categories:
class Category(models.Model):
name = models.CharField(max_length=100)
class SubCategory(models.Model):
name = models.CharField(max_length=100)
category = models.ForeignKey(Category)
class SubSubCategory(models.Model):
name = models.CharField(max_length=100)
category = models.ForeignKey(SubCategory)There should be no instance where you have extra questions or less questions than you need when constructing packets. This means packet templates with filled-in needs need to be generated whenever the packet number for a set is changed.
Right now I'm checking if you meet the minimum total number of questions for all categories. If you don't, you get a warning. It can optionally fill in the set with placeholder questions. You would either edit these questions with real questions or delete them and manually add new questions in their place (Although I need to think through this latter scenario some more, it might cause problems).
I think my plan for now is that if you change the properties of a distribution, then you will have to rebuild the set requirements and repacketize. If you had filler questions in your set, ideally we'd delete them before the next packetization and prompt you again if you still need them.
This is a good plan.
At some near-future point, support needs to be added for combinations of standard TLCs. For example, combining Geography, Current Events, and Trash to form 1 question per packet. Clicking plus by a combined TLC automatically creates subcategories for all the TLCs combined into 1 TLC. This is an extremely important feature as, near-universally, TLCs should occur in units of 1 question per packet (which is not the case for subcategories, obviously). This then eliminates most of the need for the "Set-wide distribution" page.
So I think the way that I'd like to solve this is by just allowing for fractions in all types of categories. And you can combine this with maximum number of questions per packet to ensure that it doesn't pick 2 geography questions for one round and 2 current events questions for the next round.
This did not work in QEMS1, and I don't think it will work in QEMS2 (at least currently). The problem is that you'd need a separate parameter for "maximum number of questions per packet", not just tossups/bonuses, because you don't want both a G tossup and a G bonus in a packet, but currently the max for both G TU and G B would be 1. To be fair, the "units of 1 question per packet" doesn't solve it out-of-the-box either, but I think it provides a better framework for it? In that (under the above revamp to categories/subcategories/subsubcategories), subcategories and categories have a semantic relationship that would make it easier to set-up exclusionary requirements (comments welcome here, as I wouldn't be surprised to be very wrong...). Moving to a generic question model (#201) may also make this more feasible..
Also, regarding the comments about top-level categories being integer based, that isn't going to work for certain formats. For instance, an ACF packet might have 1 tiebreaker question evenly distributed from the rest of the set. So you'd need to have something like 0.1 literature tossups per tiebreaker period.
Tiebreakers do present a problem in their total unlikeness to the rest of the packet. To be honest, I think tiebreakers should basically be implemented analogously to Categories (this is going to sound weird). I.e. a Tiebreaker Period has x number of questions per packet (an integer). And the Categories in it are specific as percentages (as that's how people normally think about Tiebreakers). This may also be a discussion for a separate issue since it would complicate things, but having Tiebreakers be totally different from other periods would really help w/ flexibility (and allow things like changing the number of tiebreakers w/o having to change the entire distribution for that period).
mbentley00
commented
9 years ago This did not work in QEMS1, and I don't think it will work in QEMS2 (at least currently). The problem is that you'd need a separate parameter for "maximum number of questions per packet", not just tossups/bonuses, because you don't want both a G tossup and a G bonus in a packet, but currently the max for both G TU and G B would be 1. To be fair, the "units of 1 question per packet" doesn't solve it out-of-the-box either, but I think it provides a better framework for it? In that (under the above revamp to categories/subcategories/subsubcategories), subcategories and categories have a semantic relationship that would make it easier to set-up exclusionary requirements (comments welcome here, as I wouldn't be surprised to be very wrong...). Moving to a generic question model (#201 https://github.com/grapesmoker/qems2/issues/201) may also make this more feasible..
Yes, there is going to be a max/min questions per packet field as well as a fraction for specific question types. i.e. you could have a top level category of Trash. It has 0.5 tossups per packet, 0.5 bonuses per packet, a min number of questions per packet of 0 and a max number of questions per packet of 1. And your geography top level category could have the same thing.
I don't think moving to a generic question model actually makes this particular problem any easier or harder. (Also, given the amount of new code in the packetization, I don't suggest doing the question change until that is done).
Tiebreakers do present a problem in their total unlikeness to the rest of the packet. To be honest, I think tiebreakers should basically be implemented analogously to Categories (this is going to sound weird). I.e. a Tiebreaker Period has x number of questions per packet (an integer). And the Categories in it are specific as percentages (as that's how people normally think about Tiebreakers). This may also be a discussion for a separate issue since it would complicate things, but having Tiebreakers be totally different from other periods would really help w/ flexibility (and allow things like changing the number of tiebreakers w/o having to change the entire distribution for that period).
I don't like having a separate system for tiebreakers--it means that now we need a different algorithm to complete them. Again, we can show percentages (and maybe even let you enter them directly) in the UI, but behind the scenes they're going to be hard numbers.
Perhaps we can have a separate issue w/ some technical discussion of what is feasible here and how to implement it (hopefully including Jerry). I think it's very important that a distribution model take subcategories as percents of categories as this is the best way to scale across packets and eliminate some of the issues w/ category entry that were present in QEMS1 and that are frustrating about QEMS2. As an application example, VHSL requires 30 tossups while ACF requires 20. However, HSAPQ's distribution, for instance, changes across levels of competition, not by set type. So things like changing Science to 5 questions per packet for VHSL shouldn't necessitate recalculating all of the subcategories. I imagine this is also something needed by NHBB. For a universal application example, percentages allow a much finer control of the questions in a set without getting into decimal representations -- they are more human friendly.
But with percentages you start having rounding problems. I don't think that changing this is a very frequent operation (you guys probably don't do this more than once or twice a year). And supporting it is going to make things harder to implement. If you guys want to go ahead and re-write things, go for it. But it's not anywhere near the top of my priorities.
Regarding your model of categories and subcategories, I ended up merging everything into just a top level category (with some helper methods to find children and such). The reason was that there are several other per-category things I need to keep track of and it gets really hard to match up the appropriate category/sub/subsubcat object. But I may re-evaluate that in the future.
On Mon, Jan 19, 2015 at 7:28 AM, Theta91 notifications@github.com wrote:
A very long response:
Some comments:
There needs to be a standard set of top-level categories (TLCs) from which you can select. There should be no manual typing of a category name unless it is not present in the system. This should take the form of a drop-down box, alphabetically sorted.
Agree. UI needs to be made for this.
In retrospect, putting all this in 1 issue was a very bad idea. I'm going to create a few separate issues for beta / rc-1 integration, like this one. I'm partial to UI stuff, so I will work on this down the line.
To start, TLCs present in the system should include Literature, History, Science, Fine Arts, RMP, Geography, Current Events, Trash, and Social Science. One might even separate RMP into individual categories given the differences in how sets have been handling the category.
Sure, we can pre-populate.
Excellent.
A TLC is given a min/max on tossups and bonuses per packet as in the current distribution entry screen. ONLY the TLC. There should be no "Subcategory" box next to it.
I'm treating all types of categories the same. They're going to have min tossups, max tossups, min acf bonuses, max acf bonuses, min vhsl bonuses, max vhsl bonuses, min total questions and max total questions. These are all per-period requirements. You set the total number of questions in a separate field on the distribution screen.
Thoughts for the far future (as in rc-1 future). Ideally, we would transition to having a two-form distribution entry screen. An upper portion defining the name of the distribution, number of questions in each period, and the question types included in the distribution. Submitting this refreshes the lower portion which is our current table (w/ separate submit button) w/ the category entry formset. Conceivably, this could be implemented as a UI thing: min/max tossups/bonuses/vhsl bonuses are simply not shown and set to 0 in the Distribution model (or wherever this is now -- PeriodWideEntry?).
Instead of a "Subcategory" box, there should be a "Manual Entry" link that turns the dropdown box into a text field, so that you can specify a custom category name. (clicking "Manual Entry" is then replaced with "Pre-configured Entry" which brings back the dropdown when clicked.)
It would be great if this was interactive like this in the future. V1 I think is a separate screen where you can add a new category or a new child (i.e. add a sub category or sub category). On that separate screen you choose from existing cat/sub/subsub or can create a new one.
I'll start a new issue for this.
As now, the distribution entry screen should start with 1 line of entry; there should be a plus button between the line of entry and "Submit changes" that performs "add a new category" -- this should allow you to specify a new TLC.
I think there should be a + button next to every existing category and subcategory so you can add children to it.
Agree.
All of the standard TLCs should have an associated set of standard subcategories. For example, in Literature, this would be "American", "British", "European", "World".
Yes, we can pre-populate this.
That's great news.
Pressing plus on a standard TLC automatically creates all its standard subcategories, which can be removed if so desired.
I think the way this is solved is by having the "add new category/subcategory/subsubcategory" dialog let you multi-select from existing items.
Yes, that's fantastic.
The subcategory distribution for a TLC is specified by % for tossup and bonus. Whether there should be a max/min % I haven't thought about, but I'd lean yes.
Disagree. It's just too much code and logic to have a separate way of calculating things for categories compared to subcategories. We can have some UI that shows what the percentage ends up being to make it easier for users entering new distributions. And distributions are already not completely generic since you need to give it a total number of questions. We could do everything in percentages but that poses its own problems with rounding and guaranteed numbers of questions.
Perhaps we can have a separate issue w/ some technical discussion of what is feasible here and how to implement it (hopefully including Jerry). I think it's very important that a distribution model take subcategories as percents of categories as this is the best way to scale across packets and eliminate some of the issues w/ category entry that were present in QEMS1 and that are frustrating about QEMS2. As an application example, VHSL requires 30 tossups while ACF requires 20. However, HSAPQ's distribution, for instance, changes across levels of competition, not by set type. So things like changing Science to 5 questions per packet for VHSL shouldn't necessitate recalculating all of the subcategories. I imagine this is also something needed by NHBB. For a universal application example, percentages allow a much finer control of the questions in a set without getting into decimal representations -- they are more human friendly.
Along with the associated trappings needed for Distribution/Period entry, here is how I envision the structure of categories:
class Category(models.Model): name = models.CharField(max_length=100)
class SubCategory(models.Model): name = models.CharField(max_length=100) category = models.ForeignKey(Category)
class SubSubCategory(models.Model): name = models.CharField(max_length=100) category = models.ForeignKey(SubCategory)
There should be no instance where you have extra questions or less questions than you need when constructing packets. This means packet templates with filled-in needs need to be generated whenever the packet number for a set is changed.
Right now I'm checking if you meet the minimum total number of questions for all categories. If you don't, you get a warning. It can optionally fill in the set with placeholder questions. You would either edit these questions with real questions or delete them and manually add new questions in their place (Although I need to think through this latter scenario some more, it might cause problems).
I think my plan for now is that if you change the properties of a distribution, then you will have to rebuild the set requirements and repacketize. If you had filler questions in your set, ideally we'd delete them before the next packetization and prompt you again if you still need them.
This is a good plan.
At some near-future point, support needs to be added for combinations of standard TLCs. For example, combining Geography, Current Events, and Trash to form 1 question per packet. Clicking plus by a combined TLC automatically creates subcategories for all the TLCs combined into 1 TLC. This is an extremely important feature as, near-universally, TLCs should occur in units of 1 question per packet (which is not the case for subcategories, obviously). This then eliminates most of the need for the "Set-wide distribution" page.
So I think the way that I'd like to solve this is by just allowing for fractions in all types of categories. And you can combine this with maximum number of questions per packet to ensure that it doesn't pick 2 geography questions for one round and 2 current events questions for the next round.
This did not work in QEMS1, and I don't think it will work in QEMS2 (at least currently). The problem is that you'd need a separate parameter for "maximum number of questions per packet", not just tossups/bonuses, because you don't want both a G tossup and a G bonus in a packet, but currently the max for both G TU and G B would be 1. To be fair, the "units of 1 question per packet" doesn't solve it out-of-the-box either, but I think it provides a better framework for it? In that (under the above revamp to categories/subcategories/subsubcategories), subcategories and categories have a semantic relationship that would make it easier to set-up exclusionary requirements (comments welcome here, as I wouldn't be surprised to be very wrong...). Moving to a generic question model (#201 https://github.com/grapesmoker/qems2/issues/201) may also make this more feasible..
Also, regarding the comments about top-level categories being integer based, that isn't going to work for certain formats. For instance, an ACF packet might have 1 tiebreaker question evenly distributed from the rest of the set. So you'd need to have something like 0.1 literature tossups per tiebreaker period.
Tiebreakers do present a problem in their total unlikeness to the rest of the packet. To be honest, I think tiebreakers should basically be implemented analogously to Categories (this is going to sound weird). I.e. a Tiebreaker Period has x number of questions per packet (an integer). And the Categories in it are specific as percentages (as that's how people normally think about Tiebreakers). This may also be a discussion for a separate issue since it would complicate things, but having Tiebreakers be totally different from other periods would really help w/ flexibility (and allow things like changing the number of tiebreakers w/o having to change the entire distribution for that period).
— Reply to this email directly or view it on GitHub https://github.com/grapesmoker/qems2/issues/118#issuecomment-70511652.
The way distributions work now is incredibly broken. Here is how distributions, distribution entry, etc. must work going forward:
Distribution Entry:
Set Distribution:
Future:
I will add to this list as I get a chance to think more about this, or other people add in comments.