Write leaderboard points upload script

[Schmavery]

Goal

Write a script that can be periodically run that will update a google sheet by:

• update point totals for challenges based on technical metrics from the testnet
• not overwriting manually-entered challenges

Proposal:

We use one sheet per phase with several tabs/pages. First tab is the aggregate of all points for the phase. This can probably be done with sheets formulas and may not need to be edited from the upload script at all.

Every subphase in a testnet phase (i.e. 3.1, 3.2) will have a sheet containing the points per challenge per user for that testnet.

user	challenge1	challenge2	...
user1	1000	100
user2	100	100
user3	70	0

When running a points update, we:

• Download existing points from the points page for the current testnet in the sheet
• Compute all technical metrics based on the testnet
• Compute technical challenge points based on metrics
  • This requires that we have information about which challenges are active for the current testnet
  • For each challenge that is active, we need code that will calculate it based on the metrics
• Replace technical challenge points in the downloaded sheet with the new point values for the technical challenges.
• Upload new points data

Existing work: https://github.com/CodaProtocol/coda-automation/pull/205 We wrote a library to help SRE team upload points to the sheet, but for various reasons that didn't work out. It roughly implements some of the above plan. We will reimplement any relevant/useful parts of this in Reasonml as necessary.

Q&A:

How to handle user nickname/public key mapping?

Pull this data from another google sheet temporarily. Longterm, someone in the community spends a weekend hacking together a name service..

How to handle displaying the current challenges descriptions?

The other sheet will have a tab with a mapping from challenge identifier to description. The order of the challenge descriptions will be taken from the order of the columns in the points sheet.

Do we use https://github.com/CodaProtocol/coda/pull/4650 or the archive node?

We should abstract metrics gathering into a function that can be swapped between #4650 and archive node. We will start with #4650 and then add support for archive node.

Why not use sheets formulas to calculate points?

We get potentially more flexibility with how we calculate points based on the metrics if we do it in the sheet, but for some metrics this will result in complex formulas that will be hard to maintain and potentially brittle

What to do about things that are easily to compute using the account ledger but hard to compute using blocks?

For now, use the graphql public proxy

Where are the blobs?

https://github.com/CodaProtocol/coda/blob/develop/frontend/points-hack-april20/poc-gather-data.sh

Relevant info from leaderboard requirements:

Kinds of challenges:

• Threshold challenges (1 e.g.: if you create X blocks, you receive points)
• Continuous challenges (ex: every block you create gives you X points)
• Manual challenges (ex. node operator survey)
• Meta challenges (e.g. bonus points if you completed at least n challenges in this release)
• Release-meta Challenges (e.g. bonus points if you participated in the last 3 releases)
  • This will be a manual metric

One challenge modifiers that can be applied (More than one can be applied to the same challenge):

• Ranking challenge (ex: top N users get a +Y bonus)

Technical Metrics:

Note: Metrics can optionally be "windowed" (ex: only counts the blocks within a time range) in order to support windowed challenges.

Computable using blocks data

• Blocks created
• Transactions sent
• Snark work created (? Is this exposed in the block??)
• Snark fees collected
• Highest snark fee awarded
• Number of transactions sent to a given address

  Requires account ledger:

• Number of delegations to user
• Number of tokens in the account

Examples of challenges

• Get 500pts if you send txn to the echo service
• Create and sell zk-SNARKs on Coda
  • Earn 3 fees by producing and selling zk-SNARKs on the snarketplace: 1000 pts*
  • Anyone who earned 50 fees will be rewarded with an additional 1000 pts*.
  • 1st place, 2nd place, 3rd place will earn points, respectively 5500, 4000, 3000 bonus pts*.
  • Other users in the top 10 position will get 1500 bonus pts*.
  • Other users in the top 20 position will get 1000 bonus pts*.
• Produce 1 block and get them accepted in the main chain for 1000 pts
  • Producing at least 3 blocks will earn an additional 1000 pts*.
  • 1st place, 2nd place, 3rd place will earn points, respectively 5500, 4000, 3000 pts.*
  • Other users in the top 10 position will get 1500 pts*.
  • Other users in the top 20 position will get 1000 pts*.
• Get 1000 pts if you produce at least one block where the coinbase gets sent elsewhere

[michellewong793]

• top n function on metric **- at least n function on metric
• echo bot challenge**

[o1christine]

Looks great! A few questions/comments.

Write a script that can be periodically run that will update a google sheet

Just for my information, what period are we thinking of? The reason I'm asking is because we're planning to add a requirement for testnet users to complete a challenge within X hours after testnet launch (probably before Friday testnet standup). That way we can easily see which users are actively participating and whose stake needs to be re-delegated.

How to handle displaying the current challenges descriptions?

The other sheet will have a tab with a mapping from challenge identifier to description. The order of the challenge descriptions will be taken from the order of the columns in the points sheet.

What "other sheet" is this? In the last releases, we linked to a forum post that includes descriptions of all challenges, in the spreadsheet with points for the relevant release. Example: https://forums.codaprotocol.com/t/testnet-beta-release-3-2a-challenges/378

Why not use sheets formulas to calculate points?

We get potentially more flexibility with how we calculate points based on the metrics if we do it in the sheet, but for some metrics this will result in complex formulas that will be hard to maintain and potentially brittle

A small comment for consideration -- if the points are calculated in the sheet, it's easier to investigate and troubleshoot when users think that there might have been an error in the calculation. We should be prepared to maybe allocate some time to look into questions that users might have when the metrics are not visible in sheets.

Relevant info from leaderboard requirements:

I think you've already seen the planned challenges, but just to make sure that all challenge types are covered -- here are the planned challenges for the next release: https://www.notion.so/codaprotocol/Testnet-Beta-Release-3-2b-Challenges-140ad0dd65854ee5b59ea18a29f9afe5 The only thing that I still need to do is to check some numbers with Evan or Brandon (what are realistic numbers for block production and snark work to decide on the 2nd level threshold). Will confirm this asap and let you know.

Add tracking to the blob for who the coinbase was actually sent to

Can we still do the challenge to send the coinbase elsewhere? Many users tried this out in the Coda sandbox, it would be cool if they can apply what they've learned on the public testnet. However, I heard from Bijan that it might be challenging to include it during release 3.2b. Let us know, and we'll adjust the comms accordingly!