Generate word problems

[JackMostow]

See ARITHMETIC WORD PROBLEMS tab of Swahili translations

See Words Brainstorm

@ealanhill - I included you as an assignee only to keep you informed because you and @huiluo9527 will need to agree on how to represent the items that he generates and you input.

[JackMostow]

@judithodili, @kevindeland, @eyarzebinski, and @amyogan:

Good news! I just added Swahili and English versions of the same 6 example EGMA word problems to the bottom of the ARITHMETIC WORD PROBLEMS tab of Swahili translations. So at minimum you can include them as stories for QA, using READ in HIDE mode.

(If you want to add the answer as feedback, you could put it in REVEAL mode by editing storydata.json.)

Leonora and I respectively narrated these 6 word problems in:

• CONTENT > SWAHILI CONTENT > WORD PROBLEMS IN SWAHILI

• CONTENT > ENGLISH CONTENT > [ENGLISH WORD PROBLEMS] (https://drive.google.com/drive/u/1/folders/10XTAeDLGEaDMIj91tu2OQaFO6kmirK9h)

@judithodili - If your ~200 book-based Swahili word problems are ready in time, Leonora can narrate them at noon tomorrow (Friday 9/14/2018).

Monday 9/17/2018 is the last day I can record Leonora before 10/3/2018.

If @ealanhill finishes the word problem tutor in time to include in code drop 2 -- which it looks like he actually may! -- problems generated from templates will be much more feasible to scaffold.

So this weekend @huiluo9527 will hand-generate some problems from the templates in the ARITHMETIC WORD PROBLEMS tab, using the words in Words Brainstorm.
He'll try to generate them in Swahili as well as in English, in part by analogy based on the 6 we recorded. Even if not, we can translate his English problems into Swahili (using Google Translate + Leonora's fixes). Either way Leonora can record more Swahili word problems this Monday, 9/17/2018.

@huiluo9527 - When you instantiate the templates to generate word problems, can you also use the same parameters to instantiate the parameterized prompts in 2018-07-18 Word Problem Types and Instruction, e.g. "Tap here [line 2] Delta times to give PERSON Delta more NOUNS." Then on Monday we can also record the instantiated prompts to use in Alan's word problem tutor.

Thanks! - Jack

[JackMostow]

@judithodili, @huiluo9527, @ealanhill, @eyarzebinski, @amyogan:

Narrations for the first 22 of the word problems adapted by Judith are now in ENGLISH WORD PROBLEMS and WORD PROBLEMS IN SWAHILI. The (corrected) text is at the bottom of the ARITHMETIC WORD PROBLEMS tab of `Swahili translations.

Beware that the first line of story.txt for each of these 44 "stories" is its "title" (e.g. Word problem 1 from Judith), which should be deleted.

[JackMostow]

@judithodili , @eyarzebinski , @ealanhill, and @msalim -

The 12 problems and their scaffolds generated by Hui from our templates are in the last part of the ARITHMETIC WORD PROBLEMS tab of Swahili narrations. Their Swahili narrations are in WORD PROBLEMS IN SWAHILI.

The story for, e.g., Hui's 1st problem is in RT11-POSEIDON 1st Hui prob 142108.413 09172018 and the corresponding scaffolds are in RT11-POSEIDON 1st Hui sca 142159.573 09172018.

I already deleted the story "title" from story.txt for these 24 folders but not for the ones uploaded earlier.

[JackMostow]

From: Hui Luo huiluo@andrew.cmu.edu Sent: Thursday, September 20, 2018 12:18 AM To: Jack Mostow mostow@cs.cmu.edu Subject: Word problems

Hi professor,

I have been thinking about the question generation for a few days, and my first solution is to generate a question from a given text, which is a general method in the QG field. For example, the given text is "I had three apples. I bought two more apples. Now I have five apples". Then with the trained model we can generate a question like, "I had three apples. I bought two more apples. How many apples do I have?" However, the problem is we do not have such kind of source texts to generate our questions, and to train our model we also need a large corpus. So I tried another method, we can simply head-generate some templates and insert some parameter to generate a word problem. For example, I used a template: %(person_name1)s has %(start_num)s %(nouns)s. %(person_name2)s has %(delta_num)s %(nouns)s. How many %(nouns)s do they have altogether? And we can generate a problem like this: Daniel has one cookie. Dylan has three cookies. How many cookies do they have altogether? With random selections of person names, nouns and different numbers, we can generate many problems with a single template. I have tried this method with several templates, and the automatically generated problems are OK. The weakness of this method is that it need pre-fixed templates and parameters. Is there any problem with my solution? I think we can continue to generate more templates to generate a large varieties of word problems. By the way, when is our next meeting? Do we need to meet weekly?

Thanks! Hui

[JackMostow]

Hui - We use GitHub to keep our communication organized, so I posted your email above and reply here. Yes let's meet weekly once I’m back on Wed. 9/26. For word problems and their scaffolding, it helps to understand their underlying structure in terms of the underlying types of operation (CHANGE, COMPARE, COMBINE) and the position of the unknown.

Generating word problems and instruction in Swahili requires understanding that which number word to use depends on the semantic class of noun it modifies, often (but not always) indicated by its prefix. E.g. 2 may be "mbili", "wawili", "viwili", etc. depending on the type of thing it is counting. For a quick introduction, see Noun Classes in http://www.swahilicheatsheet.com. Look at the Swahili translations of the English problems you generated, and think what it would take to generate the appropriate Swahili number words for them, in order to implement Swahili versions of the templates for problems and scaffolds. Thanks. - Jack

[JackMostow]

@judithodili, @amyogan, @huiluo9527, and @msalim - If we're to have only a testing activity for word problems, with no tutor, let's at least give feedback in the form of supplying the correct answer at the end. Doing it verbally would require using the correct number word for the problem-specific noun class. Instead, we can give the kid [10 seconds] to answer, then display the numerical answer as an illustration.

The story.txt would look something as follows:

bla bla bla [story problem] {question_mark_icon.jpg} bla bla? [ask the answer] wait_for_answer [a [10 second] recorded silence]

"Here's the answer." {7.jpg} [a large numeral for the correct answer; the blank lines put it on a new page} show_answer [a [3 second] recorded silence to keep the answer showing]

bla bla bla [next story problem starts on new screen] {question_mark_icon.jpg}

and so on.

1. @judithodili - Is this less work than narrating problem-specific answers?

2. @amyogan - Is it ok from a usability perspective?

3. @huiluo9527 - I put the numeric answers in a separate column of ARITHMETIC WORD PROBLEMS.

Thanks. - Jack

[amyogan]

Hmm... I would think this would present some usability issues, unless there is something on the screen (a countdown?) to indicate that the answer is coming. 10 seconds is a very long time to wait if everything else in the system is an immediately contingent response.

[JackMostow]

The same issue of how long to wait arose in the context of generic spoken-response comprehension questions. If so, how about 5 seconds? I want to leave kids enough time to solve the problem. I also don't want the wait to be so brief that kids will just wait for the answer without trying to figure it out themselves.

I'm loathe to tamper with the UI/UX at this point by adding a timer. (BTW, a numerical countdown seems confusing in the context of an arithmetic problem!)

But how about a prompt after the question? E.g: "See if you can figure out the answer before RoboTutor shows you what it is!"

[judithodili]

Currently there is a 10 second of silence file played after each question... There is a ‘____’ string displayed after each question which plays a 10 second silence ....

As far as giving them the answers after each question... I’m slightly nervous that giving them answers without corrective might be demotivating if the student never gets the answer correctly? We can try to add it to one story to see how it looks and feels. --

Regards, Judith Odili Uchidiuno www.judithu.com

[JackMostow]

How about the following graphical countdown implemented with no change to the runtime code? HIDE mode speaks the text on each page but displays only its illustration (if any). When RoboTutor asks what the answer is, show a giant question mark that shrinks down to nothing in 5 steps, each 1 second long, at which point show the numerical answer. In story.txt this solution looks like:

bla bla bla [story problem] bla bla? [ask what the answer is]

{50_point_question_mark.jpg} [new page] See if you can figure out the answer before RoboTutor shows you what it is! wait_1_second [1-second long silence]

{40_point_question_mark.jpg} wait_1_second [1-second long silence]

{30_point_question_mark.jpg} wait_1_second [1-second long silence]

{20_point_question_mark.jpg} wait_1_second [1-second long silence]

{10_point_question_mark.jpg} wait_1_second [1-second long silence]

{50_point_7.jpg} [a large numeral for the correct answer; the blank lines put it on a new page} "Here's the answer." show_answer [a [3 second] recorded silence to keep the answer showing]

... [next problem] ...

[amyogan]

As long as there is something changing on the screen, that's the main concern - without that, it's really confusing to understand why it's no longer interactive. So I'm ok with the question mark changing.

[JackMostow]

@judithodili and @amyogan - @huiluo9527 and I decided after extensive discussion to modify our templates for interactive word problem instruction to generate non-interactive worked problems as follows to present as HIDE (or HEAR) stories:

i. Present the unscaffolded problem. ii. Ask the student to say the answer out loud. iii. Pause [5] seconds, showing the same question mark icon as for generic comprehension questions. iv. Show how to solve the problem step by step, illustrating each step with its own dot bag figure. v. Show the numeric answer and say it in the Swahili form appropriate for the noun class, e.g. "maembe mawili" (if I got the Swahili right) or even "Anansi started with 2 mangoes."

Does this plan look OK?

1. By this Sunday, 9/30/18, @huiluo9527 will modify his templates and already automated English generator to generate worked problems in this form, including the prompts and specifications of the dot figures needed.

2. The problems will be in English to translate into Swahili and narrate, but first we need to make sure they are culturally appropriate for Tanzania, i.e. use words familiar to Tanzanian kids. Hui's generator uses lists of person names (name###he/she), various types (foods, breakables, wearables, and containers) of objects (plural###singular), location phrases, and colors public.conf.

a. Please point us to the list of Swahili names I think you mentioned, along with their associated genders. Alternatively, check (and augment if you like) this list and add their genders: Adie, Akai, Akitela, Akoro, Anansi, Apiyo, Byantaka, Coco, Ekai, Ismail, Juliana, Kalabushe, Keke, Lerato, Lodipo, Maguru, Mbali, Mondli, Mulongo, Nambuya, Naomi, Nelima, Odongo, Sindiwe, Sizwe, Thabo.

b. Are any of the objects or locations in public.conf unfamiliar to Tanzanian kids?

3. "Poor man's dot bags" aren't quite right for the specs so let us know what form you'd like them in, or we can give you example sketches so you can tell us what form you'd like the specs. Fake examples of the information we need to specify:

   a. "Line 1 in gray: 3 solid dots and 2 ghost dots. Line 2 in black: 2 solid dots."

   b. "Line 1 in gray: a 10-frame containing 10 solid dots. Lines 2&3 in black: an empty 10-frame."

4. As before, we will translate the generated problems into Swahili by running Google Translate and fixing its errorful output before narrating it.

There are two reasons to prefer HIDE over HEAR mode:

1. The modality learning principle says to keep text from contending with illustrations for visual attention.

2. HEAR mode requires me to record narrations using Project LISTEN's Reading Tutor in order to get the time alignments for each word. HIDE model doesn't -- it just needs each sentence recording, provided the filename matches the narrated text so that RoboTutor can find it.

Thanks! - Jack

[amyogan]

Hi Jack, A few notes:

• this sounds like a lot - will it be ready to be beta-tested on Oct 8?
• You can't have a static question mark for 5 seconds. So something on the screen will move or change?
• The question mark I saw currently in the software is blurry/pixelated, needs to get replaced

[JackMostow]

• I hope so! @judithodili - Can you do 2 and 3 by tomorrow (Friday 9/28) so Hui can use them? For 3 we just need to know what syntax to generate the picture specs in. As long as it's clear to humans, we just need to put the spec in {...} and have whoever creates the pictures (you?) use it as the name.

On 10/3 I plan to work with Leonora to fix and narrate the Swahili translations.

Judith objected to the shrinking question mark as distracting because each page flies in from the side. @kevindeland - Did Ulani disable this behavior for cloze questions as I requested? Can you disable it in general (presumably simplest) or provide an easy way to disable it in story.txt?

@kevindeland - Do you know which file the icon comes from? Would using .png instead of .jpg make it clearer?

[judithodili]

I don’t know exactly what you want me to do. From yesterday’s conversation ... the plan was

1. Present unscaffoled question and wait 3 secs.

2. Show 2 more pages breaking down each step of the problem along with the dot bag representation for images

3. Show the answer page with the numeral (or numeral and dot bag) as a picture.

The plan was for me to generate the broken down steps and create the dot bags for the existing problems and send them to you for recording...

When did the plan change? What is Hui doing? Why is Hui automatically generating anything? --

Regards, Judith Odili Uchidiuno www.judithu.com

[JackMostow]

@judithodili - The plan changed when I met with @huiluo9527 and realized a problem with converting the interactive instruction into a worked example. Specifically, the graphical scaffolding for the interactive instruction did not translate smoothly into concrete representations for a worked example in cases where the unknown is not the result of the arithmetic operation. We didn't find an intuitive dot bag representation of an unspecified first operand (initial quantity) or second operand (delta).

Also, Hui had already implemented a template-driven program that generates English prompts for interactive instruction, as in 2018-09-27 Generated instruction (a Google doc version of the plaintext file problems.

Steps i-v above essentially just break down your steps 1-3 into more detail.

• The major difference is that Hui will generate the English content automatically, including descriptions of the dot bag figures to create.
• Minor differences to resolve are:

1. How long to wait: a. 3 seconds: too little time for the kid to solve the problem? b. 5 seconds: too long for a static display, according to @amyogan

2. What to display during the wait: a. Static question mark b. Shrinking question mark: too distracting if each page flies in from the left?

3. How to present the answer: a. Silently display the numeral, e.g. 2 +: finesses having to generate the proper Swahili b. Display the numeral and speak the proper phrase including the noun, e.g. "maembe mawili" c. Display the numeral and speak the complete sentence, e.g. "Anansi ana maembe mawili" d. Display the numeric operation to the left of the dots, e.g. 5-3 = 2 (in vertical format) -- NO +: preview of arithmetic -: may confuse kids -: we don't teach multiplication, let alone division -: extra work

[judithodili]

For the unknown operand problem, why don’t we represent this as a subtraction problem?

Since we have all the problems we need recorded, and Hui’s automatic thing seems to pull out the broken down steps and generate the graphics, seems like you and Hui have it covered and need nothing from me.

--

Regards, Judith Odili Uchidiuno www.judithu.com

[JackMostow]

Not quite.

The approach I described doesn't parse existing problems like the ones you adapted from books. It will generate new (worked) problems using the templates (including for subtraction problems) in templates.conf and the lists of person names (with gender), objects (with singular and plural forms), adjectives, and locations in public.conf.

So here's what else we need:

1. Before we can run the generator, we need you -- TODAY if possible -- to vet their cultural appropriateness and to provide: a. A list of Swahili names, including gender, to use instead of the English names, and/or b. The gender for each of these names: Adie, Akai, Akitela, Akoro, Anansi, Apiyo, Byantaka, Coco, Ekai, Ismail, Juliana, Kalabushe, Keke, Lerato, Lodipo, Maguru, Mbali, Mondli, Mulongo, Nambuya, Naomi, Nelima, Odongo, Sindiwe, Sizwe, Thabo.

2. Hui's generator will generate human-understandable descriptions of graphics, not the graphics themselves, which someone else (you?) will need to create.

3. We will need someone (probably Leonora) to a. fix the Google Translate translations of the generated problems b. narrate them for me to record on Wednesday 10/3.

OK? Thanks! - Jack

[judithodili]

Unfortunately I can't do that... simply because I have done it before with the other problems, and it would be a complete waste of my time to repeat the process. What you are asking is that rather than generate the breakdown to the original questions which have 1. been vetted for cultural appropriateness 2. been translated and vetted for language, 3. are already recorded and integrated into RT 4. have been debugged and json files generated .. 5 Are word problems structured in a form that a child will see in an actual test..... we should repeat the entire process all because "Hui's generator will generate human-understandable descriptions of graphics"?, that I now have to take and generate all new graphics for the questions? I have spent too much time (and money) on the existing word problems, that repeating the process again is really not practical. There are about 50 questions in total, and it is a lot faster to go with the original plan to get it done.

Jack this is the second time that we have agreed on a strategy as a team, and you have given Hui separate instructions barely an hour later. I dont mind if you want something completely different from what the team agrees but please it's not fair to our time to have these meetings and brainstorm different strategies if you are going to plan to do something different as soon as we leave. For example, yesterday, I was over 30 minutes late for my class... because you and I and Kevin were going back and forth on a strategy to make word problems better.... A few hours later, you meet with Hui, give him separate instructions on a plan that requires the most effort on my end without even asking me if I have the capacity to absorb it time-wise. Again, I have no problems if you don't like the plans the team comes up with and you'd rather go with something you prefer. I just ask that you let me know early, and plan for the resources to execute those plans on schedule so it doesnt affect our end goal.

[eyarzebinski]

@JackMostow this change of plans gives me some concerns about the gantt chart. This is a lot of proposed work to fit in before beta testing on Oct 8.

[JackMostow]

@judithodili - My understanding from our meeting was that you were waiting for me to tell you what diagrams we needed. Did you already generate dot bag diagrams for the narrated problems? If so, where? I can take a look at them to see if it's feasible to retrofit them into worked problems, but I'm skeptical.

Only the problems themselves are recorded so far (other than the interactive instruction that Hui already generated for 12 problems). Converting the problems to properly narrated worked problems would double the amount of narration required.

If worked problems are not feasible for whatever reasons, the lowest-cost fallback plan is to:

1. Present the problem as at present.
2. Prompt the kid to "See if you can tell RoboTutor the answer before it shows it to you."
3. Wait for the kid to answer -- not 10 seconds (too long!), just 3 or 5.
4. Then give feedback by saying "Here's the answer" and displaying the numerical answer for [3] seconds.

Step 1 is already implemented. Step 2 is new but requires only a single prompt to use with all problems. Step 3 requires replacing the noisy 10-second recording of "__" with a 3- or 5-second true silence. Step 4 is new but requires only one new prompt and adding a page with a picture of the numeric answer.

Let's prepare step 4 in any event, to use in either case (worked problem vs. fallback).

1. Do you want me to generate the images?
2. How do you want to incorporate them into the data sources? a. Modify story.txt by appending the following to each problem:

{7.png} Here's the answer.

b. Like a., but modify storydata.json instead. Messier and more error-prone to do by hand.

Let us know. Thanks, and sorry for any inconvenience or frustration. - Jack

[judithodili]

No I didnt want you to tell me the diagrams we needed - I wanted you to give the worked example breakdown for a few questions and I can generate the images. Take this example problem..

1.

Anna has 3 mangoes and Fatima has 2 mangoes. How many mangoes do they have in all?

The worked example break down of the problem is

1. Anna has 3 mangoes {show 3 dots}
2. Fatima has 2 mangoes {show 2 dots}
3. Together they have 5 mangoes - {show 5 dots and the numeral 5}

There are 5 types of problems the EGMA asks

1. Simple addition
2. Simple addition with unknown addends
3. Simple subtraction
4. Simple division
5. Simple multiplication.

We already have word problems for all 5 types - with most already culturally appropriate. Here is what we really need.

1. Finish making the rest of the problems culturally appropriate.
2. Create the worked example breakdown for each problem. (These are fast to do... for the most part, it's a copy/paste rather than writing code for each problem type... You or Hui can help with this).
3. You can work with Leonora to translate these and record them as soon as possible.
4. I will generate the images to go with each breakdown.
5. Matthew generates the stories.

Here is a list of all the questions in English... If you and Hui like to help speed up the process, please start editing the doc with the worked example breakdown of the problems colored green. This doc already has gendered names and familiar objects... I don't know the genders of the names you posted - and have already listed all the familiar objects and places I know the kids are familiar with.

https://docs.google.com/document/d/1zUzgROWIQGvqYcsrRKdQEAdII1IOe7IEMXCQuO0dF0s/edit?ts=5b9bc580

[JackMostow]

@judithodili -

1. What to do about errors in the English? It looks like Evelyn is fixing them as I type.
2. I think it's risky to assume that the only problem types will be clones of the examples in EGMA.
3. I think it will be tricky to create diagrams for problems where the unknown is not the result -- but you're taking responsibility for that task. @huiluo9527 - Can you add the breakdowns Judith requested? I added a couple previously in Swahili translations. Thanks. - Jack

[JackMostow]

@judithodili and @huiluo9527 - The Swahili word problems narrated so far are in the ARITHMETIC WORD PROBLEMS tab of Swahili translations, in the following groups.

rows 137-156: the EGMA example problems.

rows 160-186: problems you adapted; your Google doc includes additional problems

rows 180-298: the 12 problems that Hui hand-generated from his templates, before he automated it. They are already split up and they include instruction.

[judithodili]

Is this an acceptable way to represent a question where the unknown is not the result?

1.

There were 6 trees in front of the school. Some more trees are in the back of the school. There are 9 trees in all. How many trees are in the back of the school? 1.

  There were 6 trees in front of the school.
  2.

  There are 9 trees in all.
  3.

  So there are 3 trees in the back of the school.

[JackMostow]

That example looks ok, assuming that you first present the problem without figures, then the split-up problem with figures.

It manages to avoid showing an undetermined number of "some" dots.

I'd need to see how you plan to illustrate all 11 problem types in the tables in 2018-07-18 Word Problem Types and Instruction:

1. Put together: Combine
2. Take Away: Decrease
3. Compare
4. Split Each of these 4 categories can have the unknown be the operand1, operand2, or the result.

[judithodili]

Please review this doc and the breakdown of all the problems. Feel free to make edits to the breakdowns where you see fit. They are ready for you to translate and record them. Let me know if you need me to find a Swahili speaker prior to Wednesday in exchange for Amazon gift cards (we should do this to save time).

I am currently working on the dot bag illustrations for each question - they will be ready by the time you are done recording. See the attached for samples of the answers to questions 1-17 on the list. I'll illustrate the EGMa questions and Hui's questions right after.

https://docs.google.com/document/d/1zUzgROWIQGvqYcsrRKdQEAdII1IOe7IEMXCQuO0dF0s/edit?ts=5b9bc580

[judithodili]

When recording - please record them in groups of 10 to save processing time on our end. We already have Q1-20 already recorded, please record the breakdown only, and the full question starting from Q21.

[eyarzebinski]

@JackMostow this change of plans gives me some concerns about the gantt chart. This is a lot of proposed work to fit in before beta testing on Oct 8.

@JackMostow @judithodili @amyogan @kevindeland - pinging everyone on the admin team. This is a large amount of proposed work that has a high chance of delaying beta testing given that the content needs to be created, reviewed, recorded, implemented, merged, QA'd, and fixed in time to send an apk to beta sites by Oct 8. Do you want to discuss pushing back the beta testing date?

[JackMostow]

(@judithodili - I'm in a meeting with Amogh to debug the sentence writing activities so can't reply immediately to each post. Also I won't have time to look at the breakdown document till I'm at home. Due to holidays I'll be out for part or all of Monday and Tuesday. BTW, I forgot to mention that I'm no longer allergic to showing the problem text, at least not violently allergic, simply because it gives something to look at.)

Let's not push back beta testing on account of word problems. But let's make sure they at least say and show the numerical answer before going on. If only some of them follow the problem with how to solve it before giving the answer, that's OK.

Thanks. - Jack

[JackMostow]

@judithodili - I edited the English problems in your document. I changed some awkward phrasing. A kid who gets something edible every day is likely to eat it, so I changed it to money in one case.

[JackMostow]

@judithodili -

1. Other than narrations, do you have everything you need from @huiluo9527 and me for questions 1-50 of your Word Problems?

2. We had already recorded questions 51-52 of your Word Problems, the 6 sample EGMA questions, the 12 questions that Hui generated manually. Should we just add the feedback sentences and group them into 2 sequences of 10 problems?

3. Will you wait 3 seconds or 5 seconds for the kid to figure out and say the answer, with or without the countdown graphic requested by Amy?

4. Will your dot bag illustrations use 10-frames?

5. Will the final illustration shown while announcing the solution display the numerical answer?

6. Please point us to a sample problem showing your illustrations.

Thanks. - Jack

[judithodili]

1. I think so.

2. Sure

3. 3 seconds

4. No

5. Yes

6. I can work on it tomorrow.

--

Regards, Judith Odili Uchidiuno www.judithu.com

[JackMostow]

@judithodili - Nice and clear, thanks!

4. Why not use 10-frames for consistency with concrete representations used everywhere (?) else? a. Too much work for the number of diagrams needed b. Numbers over 10 require more than one 10-frame c. Other reason, namely?

5. Please see 2018-07-18 Word Problem Types and Instruction for concrete representation of Split (multiplication and division) problems -- one operand is the number of 10-frames (or other containers), and the other operand is the number of dots in each container.

[judithodili]

4. No ten frames because the kids don’t have that structure on the final tests - solitary dots look closer to counters - and group better to show addition/subtraction over dots in 10 frames.

--

Regards, Judith Odili Uchidiuno www.judithu.com

[JackMostow]

4. Makes sense!

5. (changed to 5 by GitHub) Can use grouping rather than containers for Split (mult/div) problems.

E.g. "4 kids share 12 cookies": OOO OOO OOO OOO or OOO OOO OOO OOO "5 seats with 2 kids each": O O O O O O O O O O or OO OO OO OO OO

[amyogan]

I will just continue to note my concern that interfaces that don't react for > 1 second cause confusion/frustration and there should be something (sound, visual) that changes during that wait time.

[JackMostow]

@amyogan - To what extent would either of the following alleviate your concern?

1. Prefacing the 3-second wait with "See if you can figure out the answer before I show it to you."

2. Incorporating the 3-second wait after each item in a sequence of 10 questions.

[amyogan]

1%?

[JackMostow]

From: Hui Luo huiluo@andrew.cmu.edu Sent: Monday, October 01, 2018 12:42 AM To: Jack Mostow mostow@cs.cmu.edu Subject: Re: FW: when record on 10/3/2018?

Confirmed. Here is the new problem generator with instructions about how to get the answer. The person names are extracted from this file, but still not enough. And this is the automatically generated problems and instruction. Sorry for finishing it a little late, if there is any problem, please let me know.

Thanks! Hui.

[JackMostow]

@huiluo9527 - Thanks!

1. Since we already recorded the narrations of the 6 EGMA problems in rows 137..156 of the ARITHMETIC WORD PROBLEMS tab of Swahili translations and your 12 problems in rows 188..298, can you easily generate the instructions to add to them (after replacing any objects and locations that @judithodili didn't use)? If not, we can record the new narrations instead.

2. The newly generated problems would require some fixes, e.g. begging -> beginning, pinapple -> pineapple, but only if we use them.

Thanks. - Jack

[JackMostow]

@amyogan -

1. In lieu of a shrinking "?", would it suffice to show change by appending one period per second, i.e.: . .. ...

2. The page-sliding animation in story activities would be visually distracting here. Is it ok to eliminate in all story activities? What if any value do you see in it, e.g. in calling visual attention to the page change?

@kevindeland or @ealanhill -

3. How quickly can you disable the page-sliding animation in all story activities?

4. How quickly could you disable the page-sliding animation in story activities selectively so that we can suppress it for word problems? It would require adding some way to specify whether to slide pages.

Thanks. - Jack

[kevindeland]

2. Eliminating the page-turning aesthetic seems a radical change for a small gain.

3-4. This would require not just removing the page-sliding animation, but also making the page image go on the same side (otherwise you would be switching L to R).

[JackMostow]

@kevindeland - (Why) should I care which page goes on which side(s), except when I want it to stay on the same side to avoid a disruptive transition?

[amyogan]

@amyogan https://github.com/amyogan -

1.

In lieu of a shrinking "?", would it suffice to show change by appending one period per second

Yes! (Although I recommend every half second based on some basic properties of perception, etc that are commonly seen in HCI studies)

1.

The page-sliding animation in story activities would be visually distracting here. Is it ok to eliminate in all story activities? What if any value do you see in it, e.g. in calling visual attention to the page change?

I don't think you need it although this sounds more complex from what Kevin says?

[huiluo9527]

@huiluo9527 - Thanks!

1. Since we already recorded the narrations of the 6 EGMA problems in rows 137..156 of the ARITHMETIC WORD PROBLEMS tab of Swahili translations and your 12 problems in rows 188..298, can you easily generate the instructions to add to them (after replacing any objects and locations that @judithodili didn't use)? If not, we can record the new narrations instead.
2. The newly generated problems would require some fixes, e.g. begging -> beginning, pinapple -> pineapple, but only if we use them.

Thanks. - Jack

I am sorry, these problems are hand-generated without template, so it can be hard to put some problems' parameters into the new instruction templates, since they have fixed parameters.

[judithodili]

I think the work we are putting into these worked examples for the word problems are not feasible for our time constraints. Is this a good compromise?

Instead of

Page 1: Anna has 3 mangoes and Fatima has 2 mangoes. How many mangoes do they have in all? Page 2: Anna has 3 mangoes Page 3: Fatima has 2 mangoes Page 4: Together, they have 5 mangoes Can we just show the answer page like so (otherwise, I can't generate all the images needed in time)

Page 1: Anna has 3 mangoes and Fatima has 2 mangoes. How many mangoes do they have in all? Page 2: Together they have 5 mangoes (with a 3+2 dot bag) illustration.

OR

Page 1: Anna has 3 mangoes and Fatima has 2 mangoes. How many mangoes do they have in all? Page 2: If you add Anna's 3 mangoes to Fatima's 2 mangoes, together they have 5 mangoes (with a 3+2 dot bag) illustration.

Please respond ASAP.

[JackMostow]

Yes, time is short. The question is what tradeoff makes sense under the circumstances. The proposed solution, at least as described, covers only addition (not subtraction or multiplication), and only when the unknown is the result (not operand 1 or 2).

Here's a minimal solution as a fallback -- no dot bag figures, no changes to READ, minimal new narrations:

1. After the question, prompt "See if you can figure out the answer before RoboTutor shows it...." (The static "...." is instead of an animation.)

2. Then show the numeric answer for [3] seconds -- silently, to avoid having to narrate the number form appropriate for the particular noun class.

What more should we include, if anything?

3. For the problems where you've already created dot bag figures, we may as well use them. Where? a. For the first few problems? b. For the first problem in each data source?

[judithodili]

Of course we will adapt to the different question types. I don’t think the numerical answer is enough - and I have done most of the answer dot bags already (but it’ll take too much time to do dot bags for the steps before the answer). As long as you are okay with 60 more solution narrations, I can get them ready for you by tomorrow.

--

Regards, Judith Odili Uchidiuno www.judithu.com

[JackMostow]

@judithodili - To answer informedly, I need to see concrete examples, including sequences of dot bags, of what you propose for:

1. Subtraction problems
2. Problems with unknown operand 1 or operand 2
3. Multiplication/division problems Thanks. - Jack