Closed friedger closed 5 years ago
friedger
commented
5 years ago Example:
blockcred
commented
5 years ago We are afraid that at the end of the day it will be converted to a social media effort rather than development effort.
On March result, there is an App from the top Five doesn't use Blockcstack infrastructure ( check the dray-run sheet to see that if this review applied, this app will go 10 positions down).
But what moved him to top 5 is only the total upvotes numbers from PH... That's it
So, it is in somehow could be considered as a hidden message to App miners to focus on marketing issues rather than the development part
friedger
commented
5 years ago @blockcred That is why the new app reviewer was added. Zinc will slowly move downwards
blockcred
commented
5 years ago @friedger In a mathematical way, the 4 reviewers should be weighted, the higher weight should go to the development effort
jackveiga
commented
5 years ago The same app creator is part of another app in the top 10 that has no development, no twitter activity, no nothing since January that has now climbed to 7th place (Jan - 15th, Feb - 20th, Mar - 7th). All based on marketing with a good ph campaign when they launched, decent trymyui ratings that they attained recently and no negative votes from de.
Also, besides the app that @blockcred mentioned there is another app in the top 10 that is nothing more than a landing page at this stage. Is not their first month within the top 10 either.
hstove
commented
5 years ago Hey, would you mind if I asked to use the names of the apps here? I understand not wanting to throw apps under the bus here, and I appreciate that. But using vague descriptions makes it harder to know exactly what you mean.
Also, thanks @jackveiga @friedger @blockcred for getting involved on Github, we definitely appreciate that.
My only problem with judging based on 'product development' is that it is subjective. We don't have open source as a requirement, and even if we did, is lines of code the best metric for product development? I don't think it is. We could require all apps to submit their 'product development' over the month, but who judges that? It would end up being subjective if someone from Blockstack PBC used their judgement to entirely withhold an app from an app mining cohort. It could be a new app reviewer, which could punish apps, but it wouldn't guarantee that they don't still place well.
I also feel that product development, in terms of simply adding new features, is not always a sign of good progress. If it was, you'd have to just tweak your app for the mere fact of tweaking it, which could actually be a bad thing. What if you feel that the best thing to do is increase your audience through marketing, because you've found that your product is working fine right now? I think that's valid, personally.
blockcred
commented
5 years ago Thanks @hstove
According to the app I talked about, it is Gladys ( No. 4 March 2019)
I think that the philosophy behind the App mining program is to reward the Apps according to the development efforts and progress, I don't believe that any App after launching and getting customers is good to be listed in the program, simply, because it will be a not fair competition, the rich will be richer. But what I believe is, after being launched, the App should be moved to another program that supports the running Apps. putting all Apps in one pool will always cause a problem.
The released Apps is ready for marketing activities rather than Apps still in MVP or Alfa version.
Also, in the mathematical formulas that are using to calculate the final scores, the previous scores are playing a great factor, this will upset the newcomers. and I suggest to RESET the results every new cycle. when you try this methodology, you will be surprised by the results.
What we have now ... Number one will be always number one even if he affected a little bit negatively in one month by any reviewer because the historical record will play as a guard for the final results.
Finally, what everyone looking for are transparency and equality. If I am a newcomer, I need to be sure that by my efforts, I can be from the top 3 or even the first one, why not.
Thanks a lot
jackveiga
commented
5 years ago @hstove the apps that I was referring to were 1) Encrypt My Photos ranked 7th in March, and 2) BlockVault ranked 8th in March.
Understand your points, plus you can also easily argue that one can game the development by creating a lot of fake activity but it doesn't mean that its actually doing anything to add value to the app. I think that a peer system review #63 could potentially work better and help solve this problem of inactivity and gaming the system. Peers are working in the same environment, facing the same challenges and can spot bullshit from miles away. This should be factored in.
I don't feel that we have yet reached a point where we have solid apps that should focus on marketing more than their development since the app mining program barely started. I think at this stage since the program is quite recent, growing your audience vs product development is still equally important, and my reasoning will be explained in the next paragraph. This will change when the program matures and we reach a stage that apps are more of a finish mature product and usage metrics can be factored in the rankings, letting users vote with their time/usage.
A lot of these apps that were mentioned promised more than they offered at this current time (like most of the apps, and not saying this is an issue is just to explain my rationale). Apps get voted on channels like PH and DE based on these promises and their ideal potential (no one has yet launched a fully finished app, which might change as the program matures). Still, some apps (like the ones mentioned) end up idling for +60 days both in regards to development and engagement with their audiences (look at their twitters) and this is not reflected in their rankings. They did a fantastic job promoting themselves at first by promising X and Y offering on Z app which allowed them to get a good kick start after putting in a couple of hours work launching a website and/or basic MPV app with the ID and/or Gaia working (no problem until here), and then they stop but continue to get rewarded in the following months.
My perception as an outsider looking at the ranking is that both Encrypt My Photos and Recall (since we both compete in the same category) have the same level of offering, audience, installations, and engagement when that is far from the truth, however, we are now both equally being rewarded closely the same (we are in 6th place).
Thank you for the quick feedback.
Pierre-Gilles
commented
5 years ago Hey! Founder of Gladys here.
I wanted to answer the few critics wrote here about Gladys rank in app mining challenge.
To give you a little bit more context about Gladys, I started this open-source project in 2013 (almost 6 years ago!).
It went slowly from being a side-project (It started in my student room) to my job, as last year I decided to quit my engineering job to focus on Gladys and work more on this open-source software. So I’m now a solo-dev working on this open-source software as my day job :)
In November 2018, I was contacted by @GinaAbrams, had her on the phone and she suggested that it could be great to allow Gladys users to save their home automation setup in Gaia storage, and therefore to integrate Blockstack login in Gladys. The integration made sense, because Gladys has always been a privacy-first software, and allowing my users to save their home automation setup securely was a great feature.
So I implemented the Blockstack login + Gaia storage and released it before December app mining month.
I think it's a win-win for both of us, as we both have different tech communities. Most of my users didn't know about Blockstack before.
Currently I’m working at full speed on Gladys 4, the next major version of Gladys. This is a huge release, and I’m working on it on a separate repo because it’s a whole different project structure ( https://github.com/GladysAssistant/gladys-4-playground ) this is why there are less commits on Gladys main repo.
You can go to my Twitter account, I’m almost live tweeting the development everyday…. ^^ So no, Gladys is definitely not an inactive project! See my Github contributions on Gladys on the past 2 months =>
As you may have noticed, I’m unfortunately never on Blockstack monthly developer call as I’m living in Asia and calls always happens during the night for me :/
Still, I’m discussing with Blockstack team regularly and try to be active on this GitHub repo when I can.
@blockcred The dry run sheet may have a problem about Gladys, as I said I implemented login + Gaia. I contacted @GinaAbrams to have more informations on that.
To give my opinion on the points listed here:
Trying to guess the activity of a project just by looking at commits on one repo might not be a good idea. People will game the system by committing more and more on master (bad dev practice), will avoid making branch + squash git flow, and we’ll end up with bloated repositories. It doesn’t promote long term work and clean dev flow. And I’m even not talking about committing automatically with bots.
Then, has @hstove noted, judging product development by feature added or public development activity is maybe not a good way of judging a product.
Developing a software is not just about code.
Lots of time is spent talking to users, gathering feedbacks, writing content, sending newsletters, and I’m not even talking about the business side: accounting stuff, taxes, banking, legal and many other boring stuff :p
You can add on top of that user management + handling employees/contractors.
I think Blockstack wants apps who stays in the long run, generate revenues and create larger users communities.
But I completely agree that it’s not easy to find objective metrics to evaluate this kind of projects.
friedger
commented
5 years ago @Pierre-Gilles Thanks for your inside. Please do not take it personally. I am trying to discuss the algorithms. I am happy if we come to a conclusion that the algorithm should not care about progress of any apps. Do you think the module will evolve? For OI ConvertCSV I am more or less done with features, there is little that can be added if it is about backup and restore.
What do you think about monthly reporting about status updates? Maybe just posting two lines about what was done last month and what is planned in the clone of vote.blockstack.org?
stackatron
commented
5 years ago Devils advocate here is that I can make zero changes to my app and still improve by getting better at emailing new users/customers and helping them onboard. This could increase Awario, PH, and potentially investor score. I don't think we should get into the business of demanding app developers perform certain behaviors—only that they create certain results.
However, I wonder if:
provide a monthly report about the product development. Would be helpful to the app mining process in general?
Proposing we close this since I believe we shouldn't police app developers behavior.
jackveiga
commented
5 years ago I agree with @jeffdomke that we should be rewarding results and not behaviours so maybe this is not the best solution. I still think that a measure like #63 is important at this stage because as shown by some examples posted in this thread it is possible to cheat your way to the top with good clever marketing. If there is nothing that dissuades this behaviour, by having a mechanism in the vote that can judge apps that are not actually delivering month after month on what they are promising plus selling on their marketing campaigns I'm afraid that this will just encourage more developers, especially newcomers, to shift almost entirely to marketing to maximise their odds of reaching the top 10.
friedger
commented
5 years ago This issue is about product development in general , not only code.
I understand that the app mining criteria do not specify that the app should improve. IMHO, this needs to be fixed.
Once an app does not improve anymore, no rewards should be given. The best apps should be apps that still improve (update libraries, security patches, raises awareness,..)
On Fri, 22 Mar 2019, 03:12 Jack Veiga, notifications@github.com wrote:
I agree with @jeffdomke https://github.com/jeffdomke that we should be rewarding results and not behaviours so maybe this is not the best solution. I still think that a measure like #63 https://github.com/blockstack/app-mining/issues/63 is important at this stage because as shown by some examples posted in this thread it is possible to cheat your way to the top with good clever marketing. If there is nothing that dissuades this behaviour, by having a mechanism in the vote that can judge apps that are not actually delivering month after month on what they are promising plus selling on their marketing campaigns I'm afraid that this will just encourage more developers, especially newcomers, to shift almost entirely to marketing to maximise their odds of reaching the top 10.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/blockstack/app-mining/issues/64#issuecomment-475469333, or mute the thread https://github.com/notifications/unsubscribe-auth/ABYcWS5r9TICCI_Vh2E29CwEXMHIlpCsks5vZDvvgaJpZM4b-vVQ .
stackatron
commented
5 years ago Once an app does not improve anymore, no rewards should be given. The best apps should be apps that still improve
@friedger I agree in principle, but I can't see a practical and fair way to measure this?
friedger
commented
5 years ago @jeffdomke At least require
If that is not given then no payout is made. This would also handle the case when an app publisher died, lost interest, etc. It also
stackatron
commented
5 years ago IMO, not a great idea for a multitude of reasons. But, I think the idea of allowing apps to post a status of how they improved is useful in many ways. Starting new issue #86. Closing this one.
What is the problem you are seeing? Please describe. The system rewards apps that do not show active product development.
How is this problem misaligned with goals of app mining? This discourage other app publishers to produce high quality apps
What is the explicit recommendation you’re looking to propose? Add a reviewer that rate apps by its progress in the product development
App publisher taking part in the app mining program need to provide a monthly report about the product development. The incorrect reports can be flagged by the community.
Failure to produce a the report results in disqualification.
Additional context see also #63