Draft the Survey

[marcdotson]

Hey, @paulstat6. I have three RAs that would be happy to help draft the survey, @embeukes, @morganbale, and @meghop. I'm not sure where you are in the process, but if you have an outline, we could put it together in Qualtrics or just in Word?

[marcdotson]

Hey @embeukes, @morganbale, and @meghop -- @paulstat6 is going to be on campus tomorrow (Thursday). We're going to talk at 3 pm in my office about the survey. If you're available, please join us!

[marcdotson]

We had a good meeting with @paulstat6 yesterday. @embeukes, if you'll take the lead, with @morganbale and @meghop helping, let's build out the draft of the survey (attached). This includes fleshing out the attributes and levels grid (don't worry about pictures just yet). If you have any questions, drop on by or comment here.

Auto Intenders Survey Draft 1.docx

[embeukes]

@marcdotson will do! Are we planning on doing this in Qualtrics?

[marcdotson]

Nope, @embeukes. Just draft it out in Word (including associated logic in all caps and bold, like what you see already). It will be custom-coded by Paul's team.

[embeukes]

@marcdotson @meghop @morganbale here's an outline of the attributes and levels grid: https://docs.google.com/spreadsheets/d/1Yhe8pD3jvkcue_k3yPXTVRVd4LTlJut3MhtA0Cn4myw/edit?usp=sharing

[embeukes]

@marcdotson are these survey participants from across the US or a certain location?

[paulstat6]

Across the US.

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Emily Beukers notifications@github.com Sent: Wednesday, February 27, 2019 12:41 PM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: paulstat6 Edward.Johnson@dynata.com; Mention mention@noreply.github.com Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

@marcdotsonhttps://github.com/marcdotson are these survey participants from across the US or a certain location?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-468001593, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AtZqUIuKaB_ZjkZ9vb2sc65IhCNo3aZgks5vRt8ugaJpZM4bGBNf.

[marcdotson]

Thanks, Paul!

[embeukes]

@marcdotson @meghop @morganbale here's a link to the survey we are editing as well: https://docs.google.com/document/d/1np-hWovOozXM6BRmaLdVLIUiRuKni241ptp54Ph0EmE/edit?usp=sharing

[embeukes]

@marcdotson I added demographic questions and built out the attribute & levels grid based on our discussion. https://docs.google.com/document/d/1np-hWovOozXM6BRmaLdVLIUiRuKni241ptp54Ph0EmE/edit?usp=sharing

[marcdotson]

@embeukes is investigating possible ways to deal with Make and Model/Body Style in the conjoint attributes and levels grid. @marcdotson is going to talk with Jeff and Elea about how this has been deal with in the past. Complete draft of the survey soon!

[embeukes]

@marcdotson here are some of the options I found for exploring make/model.

1. Sawtooth actually recommends using a menu-based conjoint for things like cars. Have you worked with those before and would it still measure utility the same way? https://www.sawtoothsoftware.com/download/mbcbooklet.pdf
2. Most other surveys I found were either only testing a handful of brands (which we could do by filtering out anyone not interested in the ~10 that we include) or they used country of origin in place of brand (e.g. German, Japanese, American) Personally I don't think that second option measures what we actually want to measure...
3. The only way I see to include more of the market would be the ACBC model. And the question with that is does it really work?

Were you able to find anything else from your contacts?

[paulstat6]

Here is my feedback:

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Emily Beukers notifications@github.com Sent: Saturday, March 16, 2019 11:09 AM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: paulstat6 Edward.Johnson@dynata.com; Mention mention@noreply.github.com Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

@marcdotsonhttps://github.com/marcdotson here are some of the options I found for exploring make/model.

1. Sawtooth actually recommends using a menu-based conjoint for things like cars. Have you worked with those before and would it still measure utility the same way? https://www.sawtoothsoftware.com/download/mbcbooklet.pdf I am very familiar with MBC. If someone was ordering a car online and customizing it then I would agree 100% to do an MBC. However, most car purchases are not custom made or done online. Instead they are done at dealerships were you don’t really have the option to customize but you have to pick amidst existing inventory. Furthermore, we aren’t really exploring add-ons or doing multiple purchases where the MBC could potentially apply.

2. Most other surveys I found were either only testing a handful of brands (which we could do by filtering out anyone not interested in the ~10 that we include) or they used country of origin in place of brand (e.g. German, Japanese, American) Personally I don't think that second option measures what we actually want to measure... I honestly don’t like the second measurement at all. I am okay testing the handful of brands if it covers a good percent of the market (say 80-90%). I like the idea of grouping brands together more.

3. The only way I see to include more of the market would be the ACBC model. And the question with that is does it really work?

It has worked well for us in the past. We use it quite frequently in practice. However most of the time we are only gut checking our results or testing against internal hit rates rather than external validation or predictions. Chris Chapman has published at least one example where ACBC was able to accurately predict market take rate. I think Marc’s hesitation comes from a theoretical angle rather than a practical usage angle though.

Were you able to find anything else from your contacts?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-473566175, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AtZqUCjRnt9DAAJ4Bg6qweif74RBngD0ks5vXSVFgaJpZM4bGBNf.

[embeukes]

@marcdotson @meghop

Here's what Megan found as far as Market Shares (the following make up over 80%). More details are found in the google sheet.

1. Ford (13.84%)
2. Toyota (12.90%)
3. Chevrolet (11.70%)
4. Honda (8.39)
5. Nissan (7.80%)
6. Jeep (5.65%)
7. Subaru (3.95%)
8. Hyundai (3.94%)
9. Ram (3.46%)
10. Kia (3.42%)
11. GMC (3.23%)
12. Dodge (2.66%)

[embeukes]

@marcdotson As far as grouping brands go, its far more common to group by body style than brand, due to the variety of styles that one brand can produce (e.g. Ford Mustang vs. Ford F150 vs. Ford Focus). If we wanted to group brands the only way I can think to do it would be high-end (luxury), mid, and low end segments based off of average model price.

[marcdotson]

@paulstat6 @adam-n-smith @embeukes so I think we may be overcomplicating this. I discussed this with Jeff Dotson, who has done a number car surveys with Elea Feit, and he recommended screening based on type of car rather than brand.

So we ask whether or not the respondent is interesting in purchasing -- let's say -- a crossover/SUV in the next period of time -- say six months. We can use that specificity to simplify not only the brand but the other attributes and levels we include.

We're not producing this for an actual client, so the real focus of the survey revolves around having a straightforward conjoint so we can clearly demonstrate the impact of various covariates and model specifications in the model of heterogeneity.

Does that sound fair? If so, what type of vehicle should we screen based on?

[paulstat6]

High incidence is my preference.

Sent from my iPhone

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. On Mar 25, 2019, at 3:55 PM, Marc Dotson notifications@github.com<mailto:notifications@github.com> wrote:

@paulstat6https://github.com/paulstat6 @adam-n-smithhttps://github.com/adam-n-smith @embeukeshttps://github.com/embeukes so I think we may be overcomplicating this. I discussed this with Jeff Dotson, who has done a number car surveys with Elea Feit, and he recommended screening based on type of car rather than brand.

So we ask whether or not the respondent is interesting in purchasing -- let's say -- a crossover/SUV in the next period of time -- say six months. We can use that specificity to simplify not only the brand but the other attributes and levels we include.

We're not producing this for an actual client, so the real focus of the survey revolves around having a straightforward conjoint so we can clearly demonstrate the impact of various covariates and model specifications in the model of heterogeneity.

Does that sound fair? What type of vehicle should we screen based on?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-476392890, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AtZqUBsV001JU9QWyXVtPGWkTMMa_bItks5vaUXWgaJpZM4bGBNf.

[adam-n-smith]

Sedan or crossover/SUV?

[embeukes]

Crossover/SUV's combined make up close to 50% of the market and shares for Sedans/Cars are on a downward trend - so let's do Crossover/SUV's. :) https://www.statista.com/statistics/276506/change-in-us-car-demand-by-vehicle-type/

More details about market share of top Crossover/SUV makes and models are found here: https://docs.google.com/spreadsheets/d/1_I1_JG5LJ6Q91T0UG6L0PkkYpT8ebhaS0YZ9wr6PLYM/edit?usp=sharing

[marcdotson]

@paulstat6 @adam-n-smith @embeukes I think screening based on consideration of purchasing crossovers/SUVs would work best. The price range is broad enough, I don't think we need to worry about building up specific prices based on previous attribute levels -- let's just make sure we cover the expected range with sufficient granularity.

Since we're using GitHub, let's use GitHub. I've moved the survey with a screening based on crossovers/SUVs from Google Docs to R Markdown, knit as a GitHub document here.

If you'd like to make changes, you'll need to create a branch and make changes to the R Markdown, knit, then submit a push request, as detailed here. Or if you don't want to deal with that, post requested changes here and the RAs can make the changes.

[paulstat6]

I looked through it and I’ll make it work. Do you’ll have the design so we can go into programming? Just want to make sure I have all the materials before I submit it to programming.

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Marc Dotson notifications@github.com Sent: Tuesday, March 26, 2019 7:56 PM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: paulstat6 Edward.Johnson@dynata.com; Mention mention@noreply.github.com Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

@paulstat6https://github.com/paulstat6 @adam-n-smithhttps://github.com/adam-n-smith @embeukeshttps://github.com/embeukes I think screening based on consideration of purchasing crossovers/SUVs would work best. The price range is broad enough, I don't think we need to worry about building up specific prices based on previous attribute levels -- let's just make sure we cover the expected range with sufficient granularity.

Since we're using GitHub, let's use GitHub. I've moved the survey with a screening based on crossovers/SUVs from Google Docs to R Markdown, knit as a GitHub document herehttps://github.com/marcdotson/modeling-heterogeneity/blob/master/Data/Survey.md.

If you'd like to make changes, you'll need to create a branch and make changes to the R Markdown, knit, then submit a push request, as detailed herehttps://github.com/marcdotson/ra-training. Or if you don't want to deal with that, post requested changes here and the RAs can make the changes.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-476931850, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AtZqULqiT2eKXYCwdFGB5u3bKVRKwT7bks5vas-mgaJpZM4bGBNf.

[marcdotson]

@paulstat6 you don't sound excited about this. Am I missing an objective? It's for an internal test about the use of this kind of data and you won't be presenting this to an actual client, right?

Also, to clarify, it sounds like you'll want us to provide the experimental design? How many respondents are we hoping to field?

[paulstat6]

Just catching me on a bad day where I am feeling overwhelmed after coming back from vacation. I can deal with this even if I wish we were looking at all cars. It is a reasonable solution that should meet our objectives.

We are hoping to field to 1,000 respondents, but I would make the design work for only 500 if needs be. I can do the design myself in Sawtooth, but I want to make sure you’ll sign off on it. If you would prefer to do the design I am all for that as well. I won’t be utilizing our AAG team who can do the designs themselves, just the programming team who knows how to implement the design.

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Marc Dotson notifications@github.com Sent: Thursday, March 28, 2019 6:03 PM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: Edward Johnson Edward.Johnson@Dynata.com; Mention mention@noreply.github.com Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

@paulstat6https://github.com/paulstat6 you don't sound excited about this. Am I missing an objective? It's for an internal test about the use of this kind of data and you won't be presenting this to an actual client, right?

Also, to clarify, it sounds like you'll want us to provide the experimental design? How many respondents are we hoping to field?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-477814755, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AtZqUJ0zD5XFcOLnO7mH6g1Mgii54QlNks5vbVhKgaJpZM4bGBNf.

[marcdotson]

@paulstat6 I get that. I'll confirm with @adam-n-smith and create the design with my RAs.

[adam-n-smith]

@marcdotson let me know what you need from me or if you want to discuss anything.

[marcdotson]

If it looks good to you, @adam-n-smith, I'll create the design and have you take a final review before we give @paulstat6 the go-ahead to launch. It's the kids' spring break this week, so I'll try and get it done with a number of other things before we leave Wednesday.

[marcdotson]

I've merged @adam-n-smith's changes and created the design matrix using the design script in Code. Balance across attribute levels is good:

I think we're ready to start coding. I'll email you the design matrix, @paulstat6, since it's a bit too large to share here. Note that I haven't coded an outside option, although I'm assuming we'll want one.

[marcdotson]

@embeukes @morganbale @meghop let's get images for the brand and safety rating attributes.

• For brand, we need logos as PNG files with white backgrounds, all the same size.
• For safety rating, we need to create five images, each with five starts with one or more filled.

Make sense?

[paulstat6]

Please email them to me when done and I’ll send on to the programming team.

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Marc Dotson notifications@github.com Sent: Wednesday, April 10, 2019 12:59 PM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: Edward Johnson Edward'Paul'.Johnson@Dynata.com; Mention mention@noreply.github.com Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

@embeukeshttps://github.com/embeukes @morganbalehttps://github.com/morganbale @meghophttps://github.com/meghop let's get images for the brand and safety rating attributes.

• For brand, we need logos as PNG files with white backgrounds, all the same size.
• For safety rating, we need to create five images, each with five starts with one or more filled.

Make sense?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-481820253, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AtZqUAgR8J1dDhwlC6DDW0rDwSK-z96qks5vfjSOgaJpZM4bGBNf.

[embeukes]

@paulstat6 @marcdotson I have emailed those images to you and pending a pull request they will be in the Figures folder in Github as well.

[marcdotson]

@embeukes @morganbale @meghop we have the survey! Let's get testing.

Here's the survey.

Here's the survey with logic.

Post any issues here. We'd like to be ready to go to field by Monday.

[morganbale]

@marcdotson @embeukes @meghop

I didn't find anything wrong. The only thing I noticed is that if you say you rent, it asks the value of your current residence, wasn't sure if that is on purpose. I tested all the number limits (year born, income, etc) as well. I tested to make sure all brand options show up on open end and different combos for the first two questions.

[paulstat6]

That is a skip logic that should be fixed. Confirming the issue and getting it submitted for correction now. We do not want to ask renters the value of their current residence.

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Morgan Bale notifications@github.com Sent: Friday, April 19, 2019 3:20 PM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: Edward Johnson Edward'Paul'.Johnson@Dynata.com; Mention mention@noreply.github.com Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

@marcdotsonhttps://github.com/marcdotson @embeukeshttps://github.com/embeukes @meghophttps://github.com/meghop

I didn't find anything wrong. The only thing I noticed is that if you say you rent, it asks the value of your current residence, wasn't sure if that is on purpose. I tested all the number limits (year born, income, etc) as well. I tested to make sure all brand options show up on open end and different combos for the first two questions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-485016905, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALLGUUGWNDRW43ZKHY4NQL3PRIZOVANCNFSM4GYYCNPQ.

[paulstat6]

Confirmed the issue and that the skip logic was not in the Word document we sent to the programmers. Correcting and resending. Good catch Morgan! Thank you.

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Morgan Bale notifications@github.com Sent: Friday, April 19, 2019 3:20 PM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: Edward Johnson Edward'Paul'.Johnson@Dynata.com; Mention mention@noreply.github.com Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

@marcdotsonhttps://github.com/marcdotson @embeukeshttps://github.com/embeukes @meghophttps://github.com/meghop

I didn't find anything wrong. The only thing I noticed is that if you say you rent, it asks the value of your current residence, wasn't sure if that is on purpose. I tested all the number limits (year born, income, etc) as well. I tested to make sure all brand options show up on open end and different combos for the first two questions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-485016905, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALLGUUGWNDRW43ZKHY4NQL3PRIZOVANCNFSM4GYYCNPQ.

[paulstat6]

Logic is fixed. Feel free to go in and retest as desired.

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Edward Johnson Sent: Friday, April 19, 2019 3:25 PM To: marcdotson/modeling-heterogeneity reply@reply.github.com; marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: Mention mention@noreply.github.com Subject: RE: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

Confirmed the issue and that the skip logic was not in the Word document we sent to the programmers. Correcting and resending. Good catch Morgan! Thank you.

From: Morgan Bale notifications@github.com<mailto:notifications@github.com> Sent: Friday, April 19, 2019 3:20 PM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com<mailto:modeling-heterogeneity@noreply.github.com> Cc: Edward Johnson Edward'Paul'.Johnson@Dynata.com; Mention mention@noreply.github.com<mailto:mention@noreply.github.com> Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

@marcdotsonhttps://github.com/marcdotson @embeukeshttps://github.com/embeukes @meghophttps://github.com/meghop

I didn't find anything wrong. The only thing I noticed is that if you say you rent, it asks the value of your current residence, wasn't sure if that is on purpose. I tested all the number limits (year born, income, etc) as well. I tested to make sure all brand options show up on open end and different combos for the first two questions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-485016905, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALLGUUGWNDRW43ZKHY4NQL3PRIZOVANCNFSM4GYYCNPQ.

[marcdotson]

@paulstat6 @adam-n-smith is the "Back" option only for testing purposes? We don't want them going back to modify answers based on information they get exposed to later on in the survey.

Also, I'm questioning my wording on the conjoint. Should be "Which of the following vehicles do you prefer most?" or "Which of the following vehicles would you be most likely to purchase?"

Finally, what’s the expected time for appending the cleaned geolocation data once we finish fielding the survey?

[morganbale]

@marcdotson I think most likely to purchase is better

[paulstat6]

We can make those changes Monday morning.

Sent from my iPhone

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. On Apr 19, 2019, at 4:56 PM, Morgan Bale notifications@github.com<mailto:notifications@github.com> wrote:

@marcdotsonhttps://github.com/marcdotson I think most likely to purchase is better

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-485033075, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALLGUUHZGYHDWKETADKANR3PRJEZNANCNFSM4GYYCNPQ.

[embeukes]

@marcdotson @morganbale I went through and tested all the skip logic, number ranges, and anything else I could. Everything seems to work great, there's just a couple of nit-picky things I wanted to check:

1. On Q2.7 (the auto-suggest brands) do we want them to be able to enter text other than what we have auto-suggested? As of right now any text is accepted, which is nice if we forgot a brand in our list but also means that people if people are typing faster than our auto-suggest they can misspell brands (e.g. "Honde") and those would be accepted.
2. When people get "terminated" will they be sent to some kind of "thanks for taking the survey" screen? As of right now it just sends me to an error page.
3. I completely skimmed the instructions on the first conjoint slide about it being a crossover/SUV. Maybe we could bold that? Or include it in the instructions on the following conjoint questions? I feel like knowing that effects how much people are willing to pay.
4. Is 20,000 the lowest price in the conjoint? For new SUV/Crossovers that totally makes sense, but since we are including used cars as well should we go lower? Maybe I just represent the super cheap end of the market haha. :) Thanks! Emily

[marcdotson]

Thanks, @embeukes. @paulstat6 will need to weigh in as well.

1. I think whatever they put is fine -- it's only semi-aided recall.
2. Yes, the will get a termination screen. What you see now is just because we're testing it.
3. That's a good idea: bold that it's a crossover/SUV for the first conjoint question and change the subsequent conjoint questions to: "Which of the following crossover/SUVs would you be most likely to purchase?"
4. 20,000 is the lowest. We already have a ton of price levels. What do you think, @paulstat6 and @adam-n-smith?

[paulstat6]

I would likely be putting summed or conditional pricing on the conjoint to make it realistic, then again I am a big fan of ACBC. 😊

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Marc Dotson notifications@github.com Sent: Monday, April 22, 2019 9:07 AM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: Edward Johnson Edward'Paul'.Johnson@Dynata.com; Mention mention@noreply.github.com Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

Thanks, @embeukeshttps://github.com/embeukes. @paulstat6https://github.com/paulstat6 will need to weigh in as well.

1. I think whatever they put is fine -- it's only semi-aided recall.
2. Yes, the will get a termination screen. What you see now is just because we're testing it.
3. That's a good idea: bold that it's a crossover/SUV for the first conjoint question and change the subsequent conjoint questions to: "Which of the following crossover/SUVs would you be most likely to purchase?"
4. 20,000 is the lowest. We already have a ton of price levels. What do you think, @paulstat6https://github.com/paulstat6 and @adam-n-smithhttps://github.com/adam-n-smith?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-485443932, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALLGUUDWQGXGC4RIWHKRKNTPRXIBJANCNFSM4GYYCNPQ.

[paulstat6]

All,

I do think we should put all the extra changes together so I can just send one email to the project staff. Here is all that I have right now:

1. Remove back button
2. Bold that it's a crossover/SUV for the first conjoint question and change the subsequent conjoint questions to: "Which of the following crossover/SUVs would you be most likely to purchase?"

I agree with Marc on #1 and #2. Let me know if we want to do anything with #4.

Thanks,

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Edward Johnson Sent: Monday, April 22, 2019 9:09 AM To: marcdotson/modeling-heterogeneity reply@reply.github.com; marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: Mention mention@noreply.github.com Subject: RE: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

I would likely be putting summed or conditional pricing on the conjoint to make it realistic, then again I am a big fan of ACBC. 😊

From: Marc Dotson notifications@github.com<mailto:notifications@github.com> Sent: Monday, April 22, 2019 9:07 AM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com<mailto:modeling-heterogeneity@noreply.github.com> Cc: Edward Johnson Edward'Paul'.Johnson@Dynata.com; Mention mention@noreply.github.com<mailto:mention@noreply.github.com> Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

Thanks, @embeukeshttps://github.com/embeukes. @paulstat6https://github.com/paulstat6 will need to weigh in as well.

1. I think whatever they put is fine -- it's only semi-aided recall.
2. Yes, the will get a termination screen. What you see now is just because we're testing it.
3. That's a good idea: bold that it's a crossover/SUV for the first conjoint question and change the subsequent conjoint questions to: "Which of the following crossover/SUVs would you be most likely to purchase?"
4. 20,000 is the lowest. We already have a ton of price levels. What do you think, @paulstat6https://github.com/paulstat6 and @adam-n-smithhttps://github.com/adam-n-smith?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-485443932, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALLGUUDWQGXGC4RIWHKRKNTPRXIBJANCNFSM4GYYCNPQ.

[paulstat6]

All,

Back button was just because it was test mode. Final survey will not have back button. The conjoint question changes will be made soon. Let me know if we are good to launch or if we should hold out for more input.

Thanks,

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Edward Johnson Sent: Monday, April 22, 2019 10:18 AM To: marcdotson/modeling-heterogeneity reply@reply.github.com; marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: Mention mention@noreply.github.com Subject: RE: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

All,

I do think we should put all the extra changes together so I can just send one email to the project staff. Here is all that I have right now:

1. Remove back button
2. Bold that it's a crossover/SUV for the first conjoint question and change the subsequent conjoint questions to: "Which of the following crossover/SUVs would you be most likely to purchase?"

I agree with Marc on #1 and #2. Let me know if we want to do anything with #4.

Thanks,

From: Edward Johnson Sent: Monday, April 22, 2019 9:09 AM To: marcdotson/modeling-heterogeneity reply@reply.github.com<mailto:reply@reply.github.com>; marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com<mailto:modeling-heterogeneity@noreply.github.com> Cc: Mention mention@noreply.github.com<mailto:mention@noreply.github.com> Subject: RE: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

I would likely be putting summed or conditional pricing on the conjoint to make it realistic, then again I am a big fan of ACBC. 😊

From: Marc Dotson notifications@github.com<mailto:notifications@github.com> Sent: Monday, April 22, 2019 9:07 AM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com<mailto:modeling-heterogeneity@noreply.github.com> Cc: Edward Johnson Edward'Paul'.Johnson@Dynata.com; Mention mention@noreply.github.com<mailto:mention@noreply.github.com> Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

Thanks, @embeukeshttps://github.com/embeukes. @paulstat6https://github.com/paulstat6 will need to weigh in as well.

1. I think whatever they put is fine -- it's only semi-aided recall.
2. Yes, the will get a termination screen. What you see now is just because we're testing it.
3. That's a good idea: bold that it's a crossover/SUV for the first conjoint question and change the subsequent conjoint questions to: "Which of the following crossover/SUVs would you be most likely to purchase?"
4. 20,000 is the lowest. We already have a ton of price levels. What do you think, @paulstat6https://github.com/paulstat6 and @adam-n-smithhttps://github.com/adam-n-smith?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-485443932, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALLGUUDWQGXGC4RIWHKRKNTPRXIBJANCNFSM4GYYCNPQ.

[marcdotson]

@paulstat6 while @embeukes's point is a good one, I don't think we want to expand the price any more than we have. If we get into the 1000s of dollars, we're getting into clear dominating alternatives.

I say we're good to proceed with the existing design matrix.

[paulstat6]

Okay all changes are made. Feel free to do a final round of testing and we will soft launch. Maybe I should send soft launch data and we can revisit, especially if we are getting a lot of NONE responses?

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Marc Dotson notifications@github.com Sent: Monday, April 22, 2019 12:20 PM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: Edward Johnson Edward'Paul'.Johnson@Dynata.com; Mention mention@noreply.github.com Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

@paulstat6https://github.com/paulstat6 while @embeukeshttps://github.com/embeukes's point is a good one, I don't think we want to expand the price any more than we have. If we get into the 1000s of dollars, we're getting into clear dominating alternatives.

I say we're good to proceed with the existing design matrix.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-485501500, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALLGUUHR66QIMUTU2AQMHKTPRX6UJANCNFSM4GYYCNPQ.

[marcdotson]

Sounds like a good plan. @embeukes @morganbale @meghop let's do one more round of testing and then we'll soft launch.

[morganbale]

@marcdotson @embeukes @meghop Worked fine for me, the changes were in place.

[embeukes]

@marcdotson @morganbale I just ran through everything as well and everything seems to be working!

[marcdotson]

@paulstat6 sounds like we're good for a soft launch. What are you planning -- 100 respondents then we stop and check?

[paulstat6]

Yep. I’ll give people a heads up when we are there. Excel file or SPSS preferable?

[Dynata]http://www.dynata.com/

Edward 'Paul' Johnson Director, Product Analytics

O:+1.801.379.4017 M:+1.801.380.2642 F:+1.801.379.5073

dynata.comhttp://www.dynata.com

Dynata | 3300 N Ashton Blvd, Suite 350 | Lehi, Utah, 84043, United States of America

The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Marc Dotson notifications@github.com Sent: Monday, April 22, 2019 3:37 PM To: marcdotson/modeling-heterogeneity modeling-heterogeneity@noreply.github.com Cc: Edward Johnson Edward'Paul'.Johnson@Dynata.com; Mention mention@noreply.github.com Subject: Re: [marcdotson/modeling-heterogeneity] Draft the Survey (#1)

@paulstat6https://github.com/paulstat6 sounds like we're good for a soft launch. What are you planning -- 100 respondents then we stop and check?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/marcdotson/modeling-heterogeneity/issues/1#issuecomment-485561474, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALLGUUEI5MTDRKPPGMRXPFTPRYVZTANCNFSM4GYYCNPQ.