stacks-archive / app-mining

For App Mining landing page development and App Mining operations.
https://app.co/mining
MIT License
48 stars 16 forks source link

Paid marketing as related to Awario scoring #117

Closed cuevasm closed 5 years ago

cuevasm commented 5 years ago

What is the problem you are seeing? Please describe. It is unclear whether paying for marketing activities (or which type of activities) is currently allowed. Furthermore, it's unclear whether we want to stop certain paid efforts (such as paying for tweets) or if we see this is a positive activity when building up Reach and competing with centralized services.

How is this problem misaligned with goals of app mining? Everyone should have clarity on what is allowed and not allowed and have the chance to make their case for why they should be able to do something. The current mystery is unfair as some may not be trying things they assumed wouldn't be allowed.

What is the explicit recommendation you’re looking to propose? I propose that the community of active Miners determine if there will be restrictions on the types of marketing that can be done or not, starting with a list of the items people have currently raised as an issue (i.e. paying for someone to tweet about their project).

Describe your long term considerations in proposing this change. Please include the ways you can predict this recommendation could go wrong and possible ways mitigate. I worry that trying to regulate individual marketing strategies will be really tough at scale and never fair to everyone. I worry about us determining something valid for one project is something we don't want in the competition and it hamstrings their marketing efforts. I worry about policing this effectively given no one can prove who bought what tweets for who, etc.

Additional context Ultimately I think it's usually pretty clear if something is shady or not. I think in these cases we should just ask the Miner what they are thinking and then determine if we think that should be able to continue based on their explanation. If not, we ask them to stop and ideally they stop. I want to believe in the best behavior of everyone in this program to respect the request of the other Miners or decide the program isn't the right fit for them. I realize this will not satisfy everyone.

cuevasm commented 5 years ago

My original issue doesn't state that social media influencers shoudn't be counted - it doesn't take a position as this isn't my decision to make. My opinion, however, is actually that there's nothing wrong with influencer marketing whatsoever. It's a proven and effective marketing engine. I'm not sure how we could get into deciding for other projects what audience is relevant to them or what is 'fake' (beyond the algorithms and systems Awario already has in place to exclude the obvious). I don't think projects should be held to only marketing in audiences that are relevant to Blockstack, there are plenty of people outside that audience that are interested in their dapps.

Second, I disagree with you that influencer marketing is somehow not inline with the objectives. The objectives are to grow the visibility of your project and ultimately increase the surface area by which you can acquire users. Influencer marketing most certainly accomplishes that on numerous levels - almost every company in the world is doing some form of this type of marketing. Any level of social marketing will have some bots or fakeness to it, I think it's the nature of the beast, but there's a lot of good that ultimately comes with being successful there.

cuevasm commented 5 years ago

Also keep in mind that scores here are logarithmic, so if someone wants to spend all their money buying tweets, they're not getting way way out ahead of other projects just based on that. And that's to say nothing of all the other reviewers--it would be pretty hard for them put all this effort into 'cheating' this one and be successful with all the other app reviewers. i.e. no one is going to win App Mining only because they simply won at Awario.

dantrevino commented 5 years ago

Apparently forms.id when from #11 to #2 without any major app changes. May to June -> PH Upvotes +3, ~Awario~ TMUI Desirability down from 60 to 48???? The difference looks like awario. Am I reading that wrong?

I totally agree this is not a black and white issue. The question in my mind is, is this the behavior that we want to reward?

hstove commented 5 years ago

Apparently forms.id when from #11 to #2 without any major app changes. May to June -> PH Upvotes +3, Awario Desirability down from 60 to 48???? The difference looks like awario. Am I reading that wrong?

Assuming you mean TMUI desirability. You would be right that Awario contributed a large amount to their score increase.

andresousa commented 5 years ago

It's very clear by now that forms.id went from #11 to #2 solely due to the increase in the Awario score. It's also very likely that the increase in the Awario score resulted from bought tweets of 2/3 twitter accounts at least. Rules could also be imposed but like @cuevasm was saying, trying to regulate marketing strategies will be really tough at scale. If rules can't be applied at this moment, we need to decide if it's acceptable or not in general. If the community decides it's acceptable, I can this happening frequently on the following weeks/months with great implications on the app mining results like we've seen with forms.id this month.

jyudkin1 commented 5 years ago

@andresousa for clarity Forms.id went from #3 - #2, https://app.co/mining/may-2019 (these are the non audited numbers)

hstove commented 5 years ago

Hey, I'd like to address something for forms.id.

There 4 scores are:

So, when you average the 4, Awario only counts for 42% of their total score. It's not like it had a massively outsized impact. The reason they scored high is a great PH score, a great Awario score, and a great score from last round. It's a combination of multiple things that caused them to rank 2nd. It is definitely inaccurate to say they increased "solely from Awario".

dantrevino commented 5 years ago

Mea culpa. I mean #11 in April to #2 in June.

@hstove is 42% appropriate? I guess that is the question.

And again, I 100% agree with @cuevasm that there is no easy black and white answer to paid marketing. I did not spend a whole lot of time digging into scores. I just looked up the spreadsheet to find out what they're doing, and the first awario link i see is some really scammy twitter account that had the same forms.id tweet a number of times.

I apologize for not bringing a proper solution... at this time I can only offer my opinion something that something about this doesn't smell right.

hstove commented 5 years ago

is 42% appropriate? I guess that is the question.

Dmail has 63% of their final score from PH Bitpatron has 47% of their score from PH

For many apps, you will likely find that a big part of their score came from 1 reviewer. If you do think that apps shouldn't be rewarded for doing really well with 1 reviewer, I'm not really sure what a solution for that would be.

I'd also note that, with Awario, we chose log10(reach). Every other metrics-based reviewer just uses the raw data. If we didn't have log10, Forms.id would have had a z-score of... a lot. In addition, the theta function brought their Awario score from 3.1 to 1.75. If they want to do well next month, they'll have to increase their reach by at least 100%, or they'll likely get a negative "growth" score. I think these factors point to Awario's ranking mechanism being pretty decently balanced, considering that Form's reach is 10x what the second highest reach is.

Note that what I'm saying here has nothing to do with paid marketing vs. organic.

dantrevino commented 5 years ago

That's good context. Thanks @hstove. Note, my appreciations of Hank's comments have nothing to do with paid marketing vs organic.

andresousa commented 5 years ago

Thanks for the clarification and context Hank. I apologize for not having confirmed the previous # of forms.id. It's clear now the increase in the score of forms.id and others.

friedger commented 5 years ago

:+1: for providing guidelines!

As I am a developer, not a marketing director, I will pay for marketing.

stackatron commented 5 years ago

Duplicate with https://github.com/blockstack/app-mining/issues/120 closing this one, please continue over there.