Closed Schmavery closed 4 years ago
Looks great! A few questions/comments.
Write a script that can be periodically run that will update a google sheet
Just for my information, what period are we thinking of? The reason I'm asking is because we're planning to add a requirement for testnet users to complete a challenge within X hours after testnet launch (probably before Friday testnet standup). That way we can easily see which users are actively participating and whose stake needs to be re-delegated.
How to handle displaying the current challenges descriptions?
The other sheet will have a tab with a mapping from challenge identifier to description. The order of the challenge descriptions will be taken from the order of the columns in the points sheet.
What "other sheet" is this? In the last releases, we linked to a forum post that includes descriptions of all challenges, in the spreadsheet with points for the relevant release. Example: https://forums.codaprotocol.com/t/testnet-beta-release-3-2a-challenges/378
Why not use sheets formulas to calculate points?
We get potentially more flexibility with how we calculate points based on the metrics if we do it in the sheet, but for some metrics this will result in complex formulas that will be hard to maintain and potentially brittle
A small comment for consideration -- if the points are calculated in the sheet, it's easier to investigate and troubleshoot when users think that there might have been an error in the calculation. We should be prepared to maybe allocate some time to look into questions that users might have when the metrics are not visible in sheets.
Relevant info from leaderboard requirements:
I think you've already seen the planned challenges, but just to make sure that all challenge types are covered -- here are the planned challenges for the next release: https://www.notion.so/codaprotocol/Testnet-Beta-Release-3-2b-Challenges-140ad0dd65854ee5b59ea18a29f9afe5 The only thing that I still need to do is to check some numbers with Evan or Brandon (what are realistic numbers for block production and snark work to decide on the 2nd level threshold). Will confirm this asap and let you know.
Add tracking to the blob for who the coinbase was actually sent to
Can we still do the challenge to send the coinbase elsewhere? Many users tried this out in the Coda sandbox, it would be cool if they can apply what they've learned on the public testnet. However, I heard from Bijan that it might be challenging to include it during release 3.2b. Let us know, and we'll adjust the comms accordingly!
Goal
Write a script that can be periodically run that will update a google sheet by:
Proposal:
We use one sheet per phase with several tabs/pages. First tab is the aggregate of all points for the phase. This can probably be done with sheets formulas and may not need to be edited from the upload script at all.
Every subphase in a testnet phase (i.e. 3.1, 3.2) will have a sheet containing the points per challenge per user for that testnet.
When running a points update, we:
Existing work: https://github.com/CodaProtocol/coda-automation/pull/205 We wrote a library to help SRE team upload points to the sheet, but for various reasons that didn't work out. It roughly implements some of the above plan. We will reimplement any relevant/useful parts of this in Reasonml as necessary.
Q&A:
How to handle user nickname/public key mapping?
How to handle displaying the current challenges descriptions?
Do we use https://github.com/CodaProtocol/coda/pull/4650 or the archive node?
Why not use sheets formulas to calculate points?
What to do about things that are easily to compute using the account ledger but hard to compute using blocks?
Where are the blobs?
Relevant info from leaderboard requirements:
Kinds of challenges:
Release-meta Challenges (e.g. bonus points if you participated in the last 3 releases)One challenge modifiers that can be applied (More than one can be applied to the same challenge):
Technical Metrics:
Computable using blocks data
Requires account ledger:
Examples of challenges