search

gridcoin-community / Gridcoin-Tasks

Gridcoin community tasks repository
https://gridcoin.us
MIT License
24 stars 5 forks source link

Poll - Process: Project Listing Proposal v1.0 #194

Closed G-UK closed 6 years ago

G-UK commented 6 years ago

Project Listing Process

Title: Project Whitelist and Greylist Process proposal Question: [Process] Do you accept this proposed process to manage listing of BOINC projects? Link: https://github.com/gridcoin-community/Gridcoin-Tasks/issues/194 Answer Options: Yes, No, Abstain Duration: 4 Weeks Type: Magnitude & Balance

Management of listed BOINC projects (Greylisting)

The Greylist enables temporary removal and relisting of BOINC projects from the Gridcoin Whitelist based upon some simple rules.

A project will be automatically Greylisted if ANY ONE of the following requirements are met:

A project will be automatically re-whitelisted without requiring a new vote within the Gridcoin wallet if ALL of the following requirements are met:

Whitelisting a new BOINC project

A project is eligible to be whitelisted only if it meets ALL of the following criteria:

If all of these criteria are met, the following process must be followed.

1) Discussion thread opened on https://cryptocurrencytalk.com/forum/2436-projects/ 2) Poll is opened within the Gridcoin client in the form:

 * [Whitelist] Should the project "Project Name" be Whitelisted?
 * Answer Options: Yes, No, Abstain
 * Link to Discussion Thread provided
 * 2 Week duration
 * Magnitude & Balance

No other opinion must be given in the wallet poll to ensure neutrality.

Poll is successful if "Yes" gains greater than 50% of vote share and vote participation is above a weight of 7.5%

3) Agree go-live date on the TeamGridcoin Slack # boinc_projects channel with the admins with whitelist admin privileges. 4) Inform the project of Gridcoins intention to whitelist the project on the agreed date and ask if there is any objection to this.

References

Work Availability Score

WAS = Green If (Mean daily credit for 7 Days > (0.1 * Mean daily credit for 40 Days)) Else Red

Example Spreadsheet https://teamgridcoin.slack.com/files/U7DMXPRPB/F8VC3UCG0/whitelist_20-01-18.ods

Notes

1) Project De-listing process will follow separately after further discussion. #201 2) If TCD (Total Credit Delta) is implemented then Work Availability Score (WAS) would still work in terms of whitelisting/greylisting however replacement by a TCD solution should be considered. 3) All data used for calculations can be scraped from the existing available project XML files.

Change Log

v1.0: 20th Jan 2018

v0.5: 17th Jan 2018

v0.4: 4th Jan 2018

v0.3: 19th Dec 2017

v0.2: 16th Dec 2017

v0.1: 13th Dec 2017

jring-o commented 6 years ago

Love where this is going.

Regarding the greylisting process:

When a project is moved from white->grey or grey->white, what is the notification process? Is it immediate or is there a day or 2 transition period?

I think 24 hours both ways sounds right, but I have no real reasoning for this so... thoughts?

I think in client notification should be used, particularly if we improve the GUI in the future.

Should we look into seeing if project admins will e-mail project participants regarding whitelist, greylist, delisting?

Maybe set up a new CCT page for greylist/whitelist, or make a post in the projects section when a project is grelisted/delisted. I think the projects thread might one day become cluttered with conversations about projects. Having a page dedicated to whitelist/greylist/delist news might prove beneficial.

When a project is moved back to the whitelist from the greylist, if the conversation is in private messages, what sort of communication forwarding is to be expected?

I think any private message conversation that does not compromise the security of a project or of Gridcoin and explains the reasons for greylisting along with the solutions put in place should be considered public and made available upon request. I don't think this will be a problem, but we might as well have it in writing.

G-UK commented 6 years ago

When a project is moved from white->grey or grey->white, what is the notification process? Is it immediate or is there a day or 2 transition period?

I think 24 hours both ways sounds right, but I have no real reasoning for this so... thoughts?

I think in client notification should be used, particularly if we improve the GUI in the future.

I would think it would be alongside the Superblock. When the SB is created a quick calc can show if the projects 7 day credit average falls below the required level, if it does a flag is set that could be then be seen in the wallet or Gridcoinstats etc.

The calculation could be done either in the SB or offline wherever the list is published. (I have no idea what the coding challenge is to include it in the SB but that is how I imagined it)

Re-whitelisting would require manual removal of this flag due to the admin explanation safety net.

Should we look into seeing if project admins will e-mail project participants regarding whitelist, greylist, delisting?

I've tried to minimise relying on project admins to do anything. I don't think it should be a requirement but should be a nice to have.

Maybe set up a new CCT page for greylist/whitelist, or make a post in the projects section when a project is grelisted/delisted. I think the projects thread might one day become cluttered with conversations about projects. Having a page dedicated to whitelist/greylist/delist news might prove beneficial.

I have only put CCT down for discussion on projects as it is the current location, I think a seperate discussion is needed for where these and similar discussions take place.

When a project is moved back to the whitelist from the greylist, if the conversation is in private messages, what sort of communication forwarding is to be expected?

I think any private message conversation that does not compromise the security of a project or of Gridcoin and explains the reasons for greylisting along with the solutions put in place should be considered public and made available upon request. I don't think this will be a problem, but we might as well have it in writing.

I would expect everything to be open, so if someone PM's a project admin and an explanation is provided in a PM'd reply this must be published (with permission) to satisfy re-whitelisting. If the admin doesn't want to publically announce anything then they shouldn't be whitelisted in my opinion as we should aim to be as open as possible.

Like you say though, I don't see it being a problem

G-UK commented 6 years ago

Just asked startail on slack about how difficult it would be to add a flag to the gridcoinstats projects page to say if a whitelisted project meets the work availability requirement. This would show at a glance what the work availability is like, this should also be done on Gridcoin.us on the existing Whitelist page alongside creating the Greylist page.

Manual Greylist Process I imagine to be: 1) Each SB, gridcoinstats and gridcoin.us automatically shows each whitelisted project as Green or Red. 2) Any projects marked as Red are flagged by a volunteer on slack for a whitelist admin to remove in next SB. 3) Project copy & pasted from Whitelist to Greylist on Gricoin.us with date of Greylisting.

Re-Whitelisting: 1) When satisfactory explanation is posted/linked onto the CCT discussion thread. Green/Red status on gridcoin.us can be checked. 2) If Green the project can be flagged for a whitelist admin to add in next SB. 3) Project copy & pasted from Greylist to Whitelist on Gricoin.us

This requires volunteers to check and post to slack as well as enough people who can make the whitelist and website changes in a reasonable timescale. But then that is expected as a manual process and is alot better than what we have now.

Edit: @barton2526 just pointed out the issues with www.gridcoin.us so back to the drawing board.

jring-o commented 6 years ago

I would think it would be alongside the Superblock. When the SB is created a quick calc can show if the projects 7 day credit average falls below the required level, if it does a flag is set that could be then be seen in the wallet or Gridcoinstats etc.

Just to make sure I understand: The change would take place immediately after a superblock is added to the blockchain.

Just asked startail on slack about how difficult it would be to add a flag to the gridcoinstats projects page to say if a whitelisted project meets the work availability requirement.

I think this will be beneficial regardless of whatever greylist system we come up with. At a glance statistics regarding WU availability for projects on gridcoinstats! @startailcoon

so back to the drawing board.

hmm? Which part?

G-UK commented 6 years ago

In effect I would imagine any project stats update would be updated with the Superblock as that is I believe when we currently update the user and project statistics. It doesn't have to be like this but I thought that this would be the most logical to keep everything aligned. Basically the superbock is just the trigger to check the projects against the quantitive requirements and say yes or no.

On a similar vein of trying to keep things as simple and easy to implement as possible I was thinking the project status calc could be done on an existing website without needing to change anything in either the wallet or blockchain. My thinking was that as long as the calculation and source data is published it doesn't matter where the calculation is done and could even be done seperatly on different sites as anyone could check the figures from the source data.

Unfortunately @barton2526 pointed out that such modifications couldn't be made to gridcoin.us as Rob had locked certain permissions on the site down. This doesn't mean it couldn't be done on another site like gridcoinstats or gridcoin.research but it would be best if it was available on the official website.

Now the Holy Grail would be that the stats would be calculated and the resultant white/greylist result would be updated all in the blockchain without any need for manual intervention but this is more a proposal to get the basics in place first.

tomasbrod commented 6 years ago

When TCD and the planned credit / reward changes become reality, this ruleset should be revised. TCD, if it works out, will be able to bridge over limited (months?) periods of no work.

Notify me when done and I will create the poll.

tomasbrod commented 6 years ago

Releated: https://steemit.com/gridcoin/@nateonthenet/why-gridcoin-whitelisting-must-go

G-UK commented 6 years ago

Responses to Hangout#47 discussion.

Thank you for discussing and giving feedback, I am not comfortable discussing in voice however I am happy to answer questions in chat/text.

I had thought of the issue raised if a project has no work for 40 days then just before delisting adds some new work. What stops this from happening is: 1) A second test is applied to see if the project has had no more than 7 days with no credit in the last 40 before it can be added back to the Whitelist. 2) A project admin needs to have posted/responded with an explanation of the work outage and confirmation that work levels have returned to nomal.

The first point is quantitive so should help remove ambiguity. The second point however could cause some issues but in my opinion is still needed as a safety net. It could be removed if wanted, this would have the positive effect that the greylist is automatic based upon the two scores.

There is also a second safety net in that a poll can be started (with public explanation) on fully delisting a misbehaving project. (See Delisting)

Feel free to simulate various permutations using the provided spreadsheet, I may have missed something.

tomasbrod commented 6 years ago

Where is the spreadsheet? I seem to be missing it's location.

I think the project admin interaction should be turned down. The admins have enough work with technical and scientific issues. Notify them: yes, but only require response/agreement in more serious situation. Sometimes projects run out of work because there is simply none left on the current problem.

I think the one month period to transition from grey-list to black-list is too short.

Is the greylist<->whitelist decision an stateless function? ... it can be evaluated without knowing whether the project was on greylist the day before.

I have not created the poll yet, because I am not sure on what we will be voting on.

G-UK commented 6 years ago

Latest spreadsheet version is here: https://teamgridcoin.slack.com/files/U7DMXPRPB/F8SKCRL81/whitelist_15-01-18.ods

I can remove this requirement from the greylist -> whitetelist: "The Project Administrator has posted upon the project forum or responded to Private Messaging regarding resumption of the project at previous work levels"

I can also increase the greylist -> unlisted to 60 days (2 months)

I planned it as being stateful and the spreadsheet uses the current state in the calc, however I think it could be reworked as stateless.

G-UK commented 6 years ago

Updated to v0.5 Hopefully the modified poll requirement will add a bit more security(?) to the voting process. It could still be abused but this makes it a little bit harder, now you would require a whale to set-up and vote using multiple wallets.

Under normal circumstances votes would require both popular support and whale support to pass and couldn't be hijacked by one whale (unless as previously mentioned they went out of there way). Increasing vote participation would still be a better way of securing the polls however.

Listing Scenarios for Zero Credit Days requirement: A project is out of work or does not export stats for 7 consecutive days 1) Project is greylisted 2) Project will only be whitelisted again once they have 31 days of normal work.

Project is intermittantly out of work and triggers greylisting after having 7 zero credit days in the last 40 1) Project is greylisted 2) Project gets whitelisted again once the oldest no credit day instance falls outside 40 day limit*

*Assuming project has no additional zero credit days.

Mercosity commented 6 years ago

Great work .. Finally Greylisting is a thing of the present and no longer a thing of the future .. Well done ..

NeuralMiner commented 6 years ago

This is a great proposal. Very well written and thought out. I have one thing to note:

Poll is successful if "Yes" gains BOTH greater than 50% of vote share AND greater than 50% the total number of votes cast.

Someone can use that second half to push/stop votes; it's pretty trivial to spool up a new wallet. There's no way to verify one vote = one person. For example, some people have two wallets; a crunch wallet and an investor wallet. If they vote with both, that would count as two votes, which could trigger that second "and greater than 50% of the total number of votes cast".

jring-o commented 6 years ago

Amazing work on this proposal. Thank you. I have 2 questions:

Adding a BOINC Project to Gridcoin (Whitelisting)

Should we consider first contacting the project admin to insure that they are willing to have their project whitelisted and able to provide an increased WU load?

Removing a BOINC Project from Gridcoin (De-listing)

From: https://www.reddit.com/r/gridcoin/comments/7r23b8/a_few_brief_thoughts_on_whitelisted_projects/

There are a few good comments on the thread -- I'd recommend taking a look.

What if, as we begin to look more in depth at what projects are whitelisted, we consider forcing every whitelisted project to be voted on (re-approved) every X number of months -- 6 months, 12 months, something like that.

This would ensure that the whitelist keeps up with the evolving intent of the growing GRC network. Additionally, every project will be discussed after a set time of being whitelisted. This ensures that an individual (one of wealth of at least 100k GRC) does not decide on a whim that it's time to vote to de-whitelist something, which often is viewed as an attack and leads to aggressive conversations that can be less productive.

Voting to re-approve something via a protocol vs voting to remove something via individual wealth.

tomasbrod commented 6 years ago

Now I think this proposal is of very good quality. I will create the poll as soon as @G-UK writes the poll details in the first post. I suggest following:

title: rule:_manual_greylist_whitelist_process question: ... expiration, duration (42 days) answer: Yes, No, Request_change, Abstain

On the other hand, the time to go back from Grey to White is IMO, too long.

skcin commented 6 years ago

I also think this proposal is great. But I share the concern raised by @NeuralMiner about the 50% the total number of votes. I also think @tomasbrod is right about the time to go from greylist to whitelist. Maybe the counter for Zero Credit Days could be reset if you go from greylist to whitelist.

whitelist --> greylist: if (Days on Whitelist >= 40): Number of Zero Credit Days > 7 in 40 if (Days on Whitelist < 40): Number of Zero Credit Days since Whitelisting > 7

grelist --> whitelist: Number of Zero Credit Days ≤ 2 in 14 or Number of Zero Credit Days ≤ 7 in 40

... something like that. That way a greylisted project (if reliable enough) could make it back after 14 days, or whatever number you want to pick.

G-UK commented 6 years ago

Would these be acceptable/possible changes for a final version?

Polls: Change From "Poll is successful if "Yes" gains BOTH greater than 50% of vote share AND greater than 50% the total number of votes cast." Change To "Poll is successful if ALL of the following met:

If on a Mag & Balance poll we can see the split of vote weight between Mag & Balance then I think this would work. If not I will just change it back to vote weight but with the added vote share requirement.

Zero Credit Days (ZCD's) (all instances): Change From "Number of Zero Credit Days ≤ 7 in 40" Change To "Number of Zero Credit Days ≤ 7 in 20"

Logic here is to reduce the time to re-whitelist. At worst (7 consecutive ZCD's) would mean 12 days before re-whitelisting. The side affect is that we are less sensitive to projects having the intermitant ZCD's as we are only looking at 20 days history not 40.

@jring-o I did originally have contacting a project admin in the proposal but I have doubts that we can rely on needing a project admin to do anything for us. If possible I would like to minimise relying on people to do things.

Also on the re-voting, I don't think a full vote on each project is the way to go but having a regular (6 month?) multi-choice opinion poll may be a good idea to start discussion on any projects that people have a concern about.

jring-o commented 6 years ago

@G-UK how might this regular opinion poll look?

It is important that we create a schedule to revisit project viability so we avoid the ego/profit oriented conversations that revolve around removing a project from the whitelist. It is not a good look for us = )

Regarding contacting admins -- All that would be needed is a project admin's permission to be added to the whitelist. What if a project doesn't want to have to deal with Gridcoin? From a marketing perspective, it might be detrimental to be seen as being able to force all BOINC projects into our network without their permission. Some admins might not want to have GRC rewarded for their projects. A simple affirmative or negative is all that would be needed from an admin. Bring them in, don't force us on them.

NeuralMiner commented 6 years ago

@jring-o, we've been respecting project's wishes on whether or not to be included in the WL. I don't think we should stray from that.

Each poll I've created for a WL, I've contacted the admin first, introduced us and what we do, and then got their approval. We need to be on good terms with the projects if something ever comes up.

tomasbrod commented 6 years ago

@G-UK The ZCD is only a second line? The WAS should catch no-work condition fast. In that case, I do not see the reason behind such emphasis on ZCD.

Regarding Whitelist Polls: I do not think poll matter should be included in this proposal. It may prevent this from being accepted, and we want something to be done already.

When you are ready for a poll, write the poll details in the first post (preferably top) and notify me.

tomasbrod commented 6 years ago

Can I change the poll title to: Project Whitelist and Graylist Process proposal?

G-UK commented 6 years ago

Yes that would be ok.

tcblack commented 6 years ago

I think status.gridcoin.us would be a great place to put up a page that displays all of this information on a daily/hourly basis if it can be scripted into a web page.

dopeshitnetworks-irc-dopeshit-net commented 6 years ago

Since this was originally my idea , its already documented how to resolve some of the concerns and issues since G_UK ignored or must not have read those posts or just ignored the extensive explanations... This model is not 100% as I had written and was working on before ( 3 forum posts in past 2 months explaining and outlining ).

So this is how it should work , a project is contacted ( its so pro to use @gmail @hotmail and @yahoo addresses so a team should get @gridcoin.us email addresses to initially contact along with future communication and they should be made public readable ). I am modeling this after how IRC networks are setup and work. Application is received and when accepted ( they use a vote system too ) the project has to follow a set of guidelines. The Project Gray List System ( originally project jail ) be 100% automated.. After a project was accepted and added to the WL it would be on a 6 week trial. That is the Grey List itself , could be different but by 6-8 weeks should prove stability of work unit flow and project network reliability and reachability. If that project has followed those ( yes resets if the trigger such as 72hr 0 WU available ) requirements it would automatically be placed on the WL ( its already in the super block ) but if not it would stay until it meets it. Those requirements would be such as ##### minimum of WU at all times while on the Gray list. Server/Network uptime of 98% allowing possible planned maintenance to be worked in along with if its a brand new project they can work out bugs. This all works out in reverse too , after a project goes officially on the white list they have requirements to stay such as ##### of WU available and the same no more than 72hr 0 WU available allowing to process the WU data batches. Since its an automated system if a project drops below the thresholds set in order to be a project on the whitelist it would automatically trigger putting it on the Gray list. Forget this Red and Green BS talk and the way you know a project was on the Whitelist vs Gray list is project status pages and the need for a revamp of the www.gridcoin.us whitelist project webpage that would need to include whitelist and gray list and data placed , length of time on Whitelist and Gray List. Maybe read the last 6+ months that I have been working with #gridcoin freenode ( till shunned by our fearless leader ) and the mumble sessions. Thank you for taking my work and project and stealing it I hope you enjoy the credit. Again shows the integrity of the Gridcoin team and its people.

G-UK commented 6 years ago

Fill in those blanks, work out how you handle project outages and variable workload, produce a working concept, demonstrate said concept, gather peoples opinions and ideas, build those into an actual written proposal, work to get something that stands a chance to get accepted.

Do all that and you may possibly have a point.

As it is you posted the same wishy washy crap a year ago and have demonstrated NOTHING since. The Greylist idea was discussed long before you or I came along. You can't have a vague idea, do nothing and then sit on it forever more. This stuff is important and needs sorting out.

Not to mention that you have accused me of stealing your ideas twice now when this proposal bears no resemblance to what you posted. Hell it bears little resemblance to my original ideas so you better get on accusing everyone who has given input of stealing.

YOU DO NOT OWN EVERYTHING MENTIONING A FECKING GREYLIST.

You are a bitter person with an axe to grind because you got into a spat with someone months ago.

hot-bit-git commented 6 years ago

Non-technical requirements for listing and de-listing.

G-UK and above discussion focused on technical requirements for listing. These are necessary and can be automated, what is another plus.

As recent discussion (Moo!Wrapper de-listing problem) put into the light, there are also non-technical criteria to consider.

Having such a policy in place should improve quality of future discussions on project listing eligibility.

tomasbrod commented 6 years ago

I am in favour of weighted white-list. Same as @hot-bit-git 's last bullet point. Instead of a all-in, equal splitting of magnitude across all white-listed projects, project can be "half-in". Normal projects can have weight of 1, split projects, like ODLK&ODLK1, would have both 0.5, etc. Perhaps "less productive" projects (moo?) can be assigned lower weight instead of removal.

gridcoin23 commented 6 years ago

I am in favour of weighted white-list. Same as @hot-bit-git 's last bullet point. Instead of a all-in, equal splitting of magnitude across all white-listed projects, project can be "half-in". Normal projects can have weight of 1, split projects, like ODLK&ODLK1, would have both 0.5, etc. Perhaps "less productive" projects (moo?) can be assigned lower weight instead of removal.

Related discussion here: https://steemit.com/gridcoin/@donkeykong9000/gridcoin-multi-tiered-white-list

jring-o commented 6 years ago

As this conversation progresses, I think we need to keep focused on one issue at a time.

Let's keep magnitude the same and make a process for getting new projects onto the whitelist.

Next we'll need a processes for reviewing or removing projects from the whitelist.

After that, let's look at magnitude and how to rank projects. Ranking projects/magnitude distribution is probably going to be a very in depth discussion.

gridcoin23 commented 6 years ago

As this conversation progresses, I think we need to keep focused on one issue at a time.

Yes. That is a good plan. What practical things can any of us do to help with the current implementation? I don't just want to pitch ideas and never help out. One question though -- are the project admins aware of all of this? Their feedback should be part of our assessment of the proposal's success.

Regarding future work, I have in mind to do a more in-depth analysis comparing RAC and TCD. I thought about putting this on Steemit (to get paid something..); on the other hand, I would be discussing publicly one or two potential "exploits". Nothing is that serious, but anyways I thought I'd ask first if people rather I post the analysis somewhere else.

jring-o commented 6 years ago

What practical things can any of us do to help with the current implementation?

For now, the most practical thing to do is encourage people to join the conversation and vote on the proposal in the client. The more conversation we have on the topic the more ideas we can pull out. Additionally, the more conversation we have the more visibility this will receive and more visibility will mean a greater likelihood that new developers will see what we're doing and join in the fun.

Also, thoughts and visibility on issue #201 -- delisting -- would be great.

Regarding RAC vs TCD, if you think of a potential exploit before development and implementation, don't worry about making it public (use steemit -- get paid for your work!). That's a good thing. It lets us think of solutions before things get built, and if we build things without knowing the exploits... that's just a waste of time. Moving forward, it should not be part of this thread.

tomasbrod commented 6 years ago

Current iteration of "neural" "network" creates superblocks every day. To keep the reward distribution accurate, statistics files must be updated at similar, or higher frequency, but no more than hourly. I suggest adding this requirement to greylist rules.

A project is out of work or does not export stats for 7 consecutive days

Whole week is too long time to maintain fair distribution. Once a two days would be acceptable.

G-UK commented 6 years ago

Please See #213 for updated proposal.

denravonska commented 6 years ago

@grcjamezz I removed the comment. If you have concrete improvements write them here. If you have another proposal on how to do it write it in a new issue or somewhere else permanent, like in your Steemit post but without the personal attacks. If you still want to lash at other members do it privately.