Proposal: New Internet Labs's score should be given a heavier weight

[jcnelson]

What is the problem you are seeing? Please describe. The system does not penalize apps enough for storing user data in places other than Gaia.

How is this problem misaligned with goals of app mining? With app mining gaining popularity, it seems that people will (understandably) port as many existing applications as they can to Blockstack, and do the bare minimum to become eligible for app mining. This includes undesirable strategies like taking centralized but high-quality open-source SaaS applications, slapping Blockstack Auth onto them, submitting them to app mining, and reaping high app mining rankings from good UI/UX (which they may not have even developed themselves). Users and reviewers not explicitly looking for this behavior will be none the wiser that their data is still being stored unencrypted on a centralized app server. I consider this to be a subversion of the app mining program's goals. Unless we make sure this behavior does not get rewarded, this behavior will lead to the proliferation and funding of many existing apps that don't respect the user's digital rights crowding out newly-written apps that do. Moreover, it rewards people who don't even build apps, but just port Blockstack Auth to existing apps (which isn't that hard to do, compared to making the app store data in Gaia).

What is the explicit recommendation you’re looking to propose? Increase the weight of the review from New Internet Labs, so apps that do not store data in a way that respects the user's digital rights do not receive nearly as high a ranking as apps that do.

Describe your long term considerations in proposing this change. Please include the ways you can predict this recommendation could go wrong and possible ways mitigate. This recommendation helps ensure that developers who put the time and effort into ensuring that users' data is stored independently of the application are sufficiently rewarded for doing so. This recommendation also prevents opportunistic developers who don't care about Blockstack's mission or their users' digital rights from receiving a lot of money from app mining.

Additional context Not naming names, but there have been a few very-polished-looking apps announced on the forum recently that on first impression look fantastic, but on subsequent examination of their network traffic, don't actually encrypt data or store it in Gaia. These apps should not receive a high ranking, even though they would have great TryMyUI scores and possibly great Awario scores.

ADDENDUM: some additional, additional context. I have noticed this problem in several applications over this program's lifetime -- it's not specific to recently-added applications. Moreover, I have noticed that some applications will store data to Gaia, but will also send it to application servers for subsequent processing. I don't think app mining should reward this kind of behavior, since in this case the app client is effectively leaking the user's data without their consent. While the problem as stated pertains to storing data in Gaia, I want to emphasize that I think applications need to (1) treat the user's Gaia hub as the primary replica of the data, and (2) keep all business logic client-side in order to ensure the user's data remains confidential.

The reasoning behind it is that a "cant-be-evil" application should never be in a position to leak user data, even by accident. This means that private user data needs to be end-to-end encrypted, and all business logic that operates on private data needs to run exclusively within the Web browser. The user should have the final say over who gets to see their unencrypted data.

That said, if enacted, I don't think this proposal should penalize applications that deploy Radiks indexers or other aggregation infrastructure, provided that (1) any user can deploy their own infrastructure servers, (2) the application lets users pick which infrastructure servers to use (so they can't get locked in), and (3) the infrastructure servers are secondary replicas -- they only store soft state, and they do not see confidential data. If all of these are true, then infrastructure servers fall into the same category of application design as things like CDNs -- they only serve to enhance the availability of data, and do not implement any irreplaceable business logic.

[stackatron]

Would like more input from the App Miners please.

[Walterion01]

Sure @jeffdomke, @jcnelson Thanks for investigating our work and thanks for considering them very polished, as I think you are mentioning Arcane Docs. They are the result of months of works and be sure we try to provide the best experience for our users as we are not here to make simple apps and get funding. We want to give users secure and respectable options, and we are very serious about it. As you can see we did some projects with Blockstack Auth, began with BlackHole that is completely serverless and I talked about empowering NIL months ago here #7 to promote such apps, as I saw some apps don't even encrypt users data. For providing users great user experience, there is a need to give them high-quality tools, and we did that with BlackHole and Arcane family (Web and Desktop). I assure you that it will not be easy as "slapping Blockstack Auth" for anyone to make such apps as if it was we wouldn't see simple list apps being on the top of mining ranking. I will be glad to work on any concern regarding this issue to clearing things up and surly improve them. I sent you the sample of stored data in Gaia as an example in Slack.

[friedger]

I was hoping that we will find a new technical reviewer but until then I am happy to see doubling NIL as a reviewer.

Most of my apps use open source and if there is an opportunity to create new blockstack apps based on centralized apps then I think that is a good thing. Users should have the choice to use the apps also with their blockstack ID. If the rewards are used to improve the existing open source app or not might be difficult to judge.

Sending pull request towards the "original" project might be a long process with low priority (see opencollective.com) or be rejected by core developers (see mastodon).

We could add an eligibility check for apps whether they comply with open source licenses.

[friedger]

Maybe specifying an anti-pattern like: do not sent data to a server that was not stored on gaia before. If this anti-pattern is used the final score will be reduced by 0.5 points.

See #98

[friedger]

The problem with NIL is that top apps currently can't get more than 0.6 points, while they can get more than 1.+ on AW and PH.

[dantrevino]

I, probably expectedly, 100% agree with this proposal. I believe this would enhance the type of applications that more broadly align with the mission of blockstack to put users in control of their data and identity.

[jcnelson]

@Walterion1 This isn't specific to Arcane Office or your applications in general. Specifically, when I wrote "slap Blockstack Auth on," I had a few different apps in mind, and none of them were yours ;)

But since you asked, I have updated the proposal to provide some more context to explain where I'm coming from here.

[Walterion01]

Thank you very much, @jcnelson. That is very kind of you to explain more. I think this issue is a lot like #7 and as I stated there, pushing to be only client-side (and even not indexers) is very hard and almost impractical in some cases, but we should push the limits to go there and that behavior is an excellent idea.

[jcnelson]

I think this issue is a lot like #7 and as I stated there, pushing to be only client-side (and even not indexers) is very hard and almost impractical in some cases, but we should push the limits to go there and that behavior is an excellent idea.

I agree that some applications effectively need indexers to work at scale. For example, the sample app banter.pub needs an indexer in order to make it efficient to query everyone's posts. Without an indexer, each client would need to query each other users' Gaia hubs to find their latest posts, which would be both time-consuming and bandwidth-intense.

As mentioned in the proposal text, I don't think apps that need to use indexers should be ranked lower for doing so, provided that they do so in a way that doesn't violate user privacy or lock the user into using a particular instance. In the service of making this an easier goal for app miners to meet, I know @hstove is working on making Radiks servers peer-to-peer, so if you ran your own indexer, it would automatically stay in sync with other indexers crawling data from the same app.

While I will concede that there are a few real-world SaaS applications where it is "very hard and almost impractical in some cases" keep all business logic client-side, I will also point out that (1) this is not true for most SaaS applications, and (2) for the small exceptional set of applications where this is truly impractical (example: this would be hard to do for 23andMe), the app mining program should only reward such applications if they allow users to choose who runs the server-side logic, and where it is run.

[sdsantos]

@jcnelson I agree with the proposal.

And as a side note, it would be nice to start evaluating the blockstack importance on the product. Example: I can clone GIMP, add a simple option to store images on Blockstack, and release it under a new name and brand. But is the blockstack function relevant? Do most people use the app and not even realize it?

[friedger]

Here is a dry run for doubling NIL score: https://docs.google.com/spreadsheets/d/1Psf7xV3dir1JvFsSI2zEdy7MGv9NifYAkgqa0ZkikDg/edit?usp=sharing

SocialVault drops from 41 to 62 Satoshi Games from 49 to 83 dapps.id from 58 to 92 others go up or down one or two places...

[hstove]

I think NIL is already doing it's job pretty well. Here is a different dry run sheet (sorry Friedger, I didn't see any actual different weighting in your sheets).

https://docs.google.com/spreadsheets/d/13PXIJhEhTusjVT9elYS3LnGqSj6DBjTUDCzB_R6Inkw/edit#gid=746720458

In the original data, the highest app to not get the full NIL score is ranked 40th. After the weight change, the highest app is >60th. So, it's already pretty clear that you need to get a full NIL score to get ranked well in App Mining.

I would suggest to close this. It's a good idea, I just think NIL is already doing its job.

[stackatron]

PBC discussed, agree with Hank. One solution to explore is adding more dimensions to NIL to produce more variance in the scores. I will open a ticket for that: https://github.com/blockstack/app-mining/issues/143

[friedger]

@Hank I disagree with the assumption that rank 40 is not a well ranked position.

[hank]

This is a bit out of my field so please refrain from tagging me in the future.

[dantrevino]

Just to echo the comments that I made in slack. While doubling NIL's score alone may not be especially helpful, I do believe that this approach, along with #143 may provide some relief from the growth of Can Be Evil apps within app mining.

If there is little impact by doubling the weight of NIL score, can we do that now with the expectation that we'll also be adding more items in #143?

[hstove]

@dantrevino, I really just don't think changing the weight will make a difference. Besides, aren't there apps, whose patterns you don't like, who already get a full NIL score?

[friedger]

Apps with full score are listed here: https://app-center.openintents.org/more-blockstacky-apps

[dantrevino]

@dantrevino, I really just don't think changing the weight will make a difference. Besides, aren't there apps, whose patterns you don't like, who already get a full NIL score?

Yes, that is why, in conjunction with increasing the weight of NIL score, we need to increase the number of dimensions that NIL covers. And for the record, its not about disliking patterns, IMHO, NIL is our best filter for truly Cant Be Evil apps. TMUI doesnt do that. Awario doesnt do that.

Unfortunately, "Cant Be Evil" hasn't been part of the discussion at all up to this point, and it is painfully obvious. We all know that slapping Blockstack auth and even adding Gaia are not sufficient for creating a Cant Be Evil app. That is the only thing that NIL measures at this time.

It's past time to make Cant Be Evil a priority, and that means we need to extend the measures (#143) AND increase the weight of NIL scores so that everyone understands what the priority is.

[hstove]

Yeah, @dantrevino, that's a good point. I'm OK with moving forward with a heavier weight, given we'll increase the criteria.

Proposal: Your NIL score is counted "twice" in your final score. So, the score is the average of: NIL, NIL, Awario, and TMUI.

[stackatron]

NIL is about to announce a bunch of new criteria. Think we should reassess this post accepting those changes. Icebox for now, can come back to it in a couple weeks.

[njordhov]

NIL is 50% of the score for debuting apps, but only 25% of the score for incumbents (or more accurately, dropping towards the limit of 33% if assuming the current model and expanding the series of past scores). This creates a dynamics that should be taken into account when changing the weight of NIL.