What is the problem you are seeing? Please describe.
The TryMyUI audience is insufficiently targeted, even after selecting testers based on which products they have used in the past.
How is this problem misaligned with goals of app mining?
Apps that don't fit well into the selection provided by TryMyUI are punished by lower scores and unfair evaluation. It leads to bias in scoring favoring apps that corresponds to the audience criteria of TryMyUI, punishing apps that target niches not well defined by products used.
Apps emulating popular products may have an undue advantage as they can more easily match their TMUI audience per #1. It may lead to a proliferation of apps in certain already established app categories, contributing to the problem reported in #192.
What is the explicit recommendation you’re looking to propose?
Let app developers provide a custom framing for TryMyUI reviewers.
Currently, before reviewing an app, TryMyUI reviewers are asked to adopt a "frame of mind" in a popup that says "You have heard about this website from a friend and wanted to check it out for yourself". App developers could submit custom text for this popup, asking the reviewer to adopt a mindset and role like if they were in the prospective user/customer base of the app.
What is the dry run period (if any)
It's unclear how to do a dry run for this proposal. Perhaps make it a soft launch by limiting it initially to subsequent TryMyUI reviews, since their past scores will blunt the impact.
Describe your long term considerations in proposing this change. Please include the ways you can predict this recommendation could go wrong and possible ways mitigate.
It could be abused by making the "frame of mind" text positively influence scoring in unintended ways. Making this text public would provide transparency so the community could make adjustments if necessary.
What is the problem you are seeing? Please describe.
The TryMyUI audience is insufficiently targeted, even after selecting testers based on which products they have used in the past.
How is this problem misaligned with goals of app mining?
Apps that don't fit well into the selection provided by TryMyUI are punished by lower scores and unfair evaluation. It leads to bias in scoring favoring apps that corresponds to the audience criteria of TryMyUI, punishing apps that target niches not well defined by products used.
Apps emulating popular products may have an undue advantage as they can more easily match their TMUI audience per #1. It may lead to a proliferation of apps in certain already established app categories, contributing to the problem reported in #192.
What is the explicit recommendation you’re looking to propose?
Let app developers provide a custom framing for TryMyUI reviewers.
Currently, before reviewing an app, TryMyUI reviewers are asked to adopt a "frame of mind" in a popup that says "You have heard about this website from a friend and wanted to check it out for yourself". App developers could submit custom text for this popup, asking the reviewer to adopt a mindset and role like if they were in the prospective user/customer base of the app.
What is the dry run period (if any)
It's unclear how to do a dry run for this proposal. Perhaps make it a soft launch by limiting it initially to subsequent TryMyUI reviews, since their past scores will blunt the impact.
Describe your long term considerations in proposing this change. Please include the ways you can predict this recommendation could go wrong and possible ways mitigate.
It could be abused by making the "frame of mind" text positively influence scoring in unintended ways. Making this text public would provide transparency so the community could make adjustments if necessary.
Additional context
1 on TryMyUI differentiated audiences.