search

IndexerDAO / IndexerDAO

3 stars 0 forks source link

Indexing 101 Planning Document #85

Open alex-pakalniskis opened 1 year ago

alex-pakalniskis commented 1 year ago

Overview

Free and open-source lessons to build the skills required to get started as an indexer.

Release schedule

Every other Indexer Office Hours throughout Q2 and Q3 of 2023.

Syllabus

Unit Lesson Title Topics
1 1 Introduction to Linux Command line basics, git, user management, systemd, firewalls
1 2 Getting started with Ethereum clients Nethermind and Erigon on systemd, Hardware factors to consider
1 3 Lab 1: Sync a blockchain node Sync an RPC node with systemd: Gnosis, Nethermind on a Hetzner server
2 1 Intro to indexer operations Wallets (hot vs cold, indexer vs operator), Multisigs, Faucets, Network Registration
2 2 Getting started with containers Docker, docker-compose, Kubernetes (briefly), Helm (briefly)
2 3 Lab 2: Configure your indexing stack Deploy The Graph indexing stack and connect to your Gnosis RPC node
3 1 Intro to indexing strategy Basic overview of subgraphs and market conditions
3 2 Getting started with allocation management todo
3 3 Lab 3: Index and allocate on a subgraph Index and allocate on a Gnosis subgraph
4 1 What's next? Mainnet, Multiple subgraphs, Kubernetes, Multi-chain

Cost

$10000 for 10 lessons, each with:

Payment for time spent:

Previous developer education grant proposals

Relevant experience

alex-pakalniskis commented 1 year ago

@Web3Slimchance and @reinasun

does anything look massively wrong in the prototype syllabus?

thanks :smile:

reinasun commented 1 year ago

This looks amazing! and very helpful to new indexing-community members. I wonder if, perhaps, we can do an intro video outlining the following information:

• Our mission and vision. What information to expect and when. Where to go with questions or if unable to make a specific training. quick reference sheets to related links • Indexing and how it relates to The Graph community (subgraph dev, curating, delegating) • Use cases and how indexing expands the web3 ecosystem • The importance of collaboration and connection for long-term success • Perhaps an ending video with ways to stay connected, work through issues, and contribute

Also, do you think it would be useful to overview our use case scenario? To help people understand the choices that were made and why, monetary considerations/planning, and how to deal with the unexpected.

One more idea: Prizes, NFTs, badges, GRT, mentions to the most engaged to be added to social media may help with continued engagement

alex-pakalniskis commented 1 year ago

Great ideas about the intro/outro videos to share our mission/vision, emphasize importance of collaboration, and tips for getting connected and contributing.

I think the reward idea is great too. Would be cool if we could do something like Kudos: https://mintkudos.xyz/

reinasun commented 1 year ago

mintkudos! love that idea

alex-pakalniskis commented 1 year ago

Project Name*

Indexing 101

Please provide a description of your project*

Ten free and open-source lessons, videos, and live workshops to foster a small learning community around indexer fundamentals.

Participants will be rewarded with a POAP NFT upon demonstrating their mastery of indexing fundamentals through session participation, quizzes, and assignments.

Tutorials will be released every other Indexer Office Hours throughout Q2 and Q3 of 2023.

Unit Lesson Title Topics
1 1 Introduction to Linux Command line basics, git, user management, systemd, firewalls
1 2 Getting started with Ethereum clients Nethermind and Erigon on systemd, Hardware factors to consider
1 3 Lab 1: Sync a blockchain node Sync an RPC node with systemd: Gnosis, Nethermind on a Hetzner server
2 1 Intro to indexer operations Wallets (hot vs cold, indexer vs operator), Multisigs, Faucets, Network Registration
2 2 Getting started with containers Docker, docker-compose, Kubernetes (briefly), Helm (briefly)
2 3 Lab 2: Configure your indexing stack Deploy The Graph indexing stack and connect to your Gnosis RPC node
3 1 Intro to indexing strategy Basic overview of subgraphs and market conditions
3 2 Getting started with allocation management Opening and closing allocations, Troubeshooting
3 3 Lab 3: Index and allocate on a subgraph Index and allocate on a Gnosis subgraph
4 1 What's next? Mainnet, Multiple subgraphs, Kubernetes, Multi-chain

Are you applying as a team or individual?*

Team

What type of grant are you pursuing?*

Project (one-off)

Please select the category your project best fits into*

Education

Please select who you think will benefit from your project?*

Indexers

How much funding are you looking for? (USD)*

$10000

Have you applied for (or already received) funding from somewhere else?*

No

What's your vision for how your project will impact The Graph and/or web3?*

Our long-term vision is to become a DevOps talent incubator for The Graph and broader web3 ecosystem. We hope to support aspiring and active indexers with on-going education and project-based learning opportunities to ensure best practices are shared throughout the community.

What can you share about any past work that is relevant to this grant?*

IndexerDAO has been supporting the indexing community during MIPs with server setup guides (https://indexerdao.github.io/server-setup/) and QA testing (Launchpad, Graphcast, RPC node setup guides). We've held two community support workshops earlier in MIPs and are planning more.

Why are you interested in contributing to The Graph and/or web3?*

Our DAO is comprised of The Graph protocol contributors from a variety of backgrounds. We are passionate about decentralization, building a more diverse indexing talent pool, and fostering a community of collaboration and knowledge sharing. In applying for this grant we hope to accelerate our contributions to the protocol (and web3) by enabling our team to dedicate more time to developing educational content for the growing multi-chain indexing community.

Please provide the ETH Address which would receive the grant funding, if approved*

0x223260f3Ca014AdC40C98d50D5C4178657D53E26

reinasun commented 1 year ago

Helpful info perhaps to add to meeting invites

Free New Indexer 10 Session Workshop

About this event: This virtual workshop will run over 10 sessions to help you start your path to indexing with the graph. All content is in English and it is totally free.

How long are the sessions? Each session is 2 hours with 1.5 of content and 0.5 hours of Q&A

Why be an Indexer with The Graph? An Indexer provides the Web3 community with fast, reliable, and secure data. In return, active Indexer can provide services and earn income from the network, proportional to the amount of work they perform and their GRT stake.

What is a GRT stake? To ensure economic security of The Graph Network and the integrity of data being queried, participants stake and use Graph Tokens (GRT). GRT is a work token that is an ERC-20 on the Ethereum blockchain, used to allocate resources in the network.

Who can participate in this workshop? This Workshop is designed for anyone who wants to learn about The Graph and Indexing.

What will you learn? You will learn about the Graph, it’s ecosystem, fundamentals, and the components to become a successful indexer.

How does Indexing fit into The Graph ecosystem? The Graph Network consists of Indexers, Curators and Delegators that provide services to the network, and serve data to Web3 applications. Consumers use the applications and consume the data.

What are the main indexing components? The Indexer Software Stack Hardware & Storage Blockchain nodes Orchestration Operation Security Tools

What are fundamentals skills to becoming an Indexer? Community Connection: Discord Server, Github, Indexer OH Marketing: Indexing Strategy, subgraphs, market conditions, delegators, curators Linux: command line basics, system Tool Management:
Software: Rust, docker, kubernetes, helm Managing Blockchain Nodes: RPC, The Graph Node, ETH Client Hardware:/Storage: Small, Standard, Medium, Large Operations: Allocation management, Monitoring, Alerting, IR, Backups Security: Operator wallet, firewall, ports Cloud: Heztner (side note: google cloud, aws, ms azure, Alibaba)

alex-pakalniskis commented 1 year ago

Hey doxe01.eth and Lorena,

I'll answer both of your questions in this response as there is some overlap.

We, IndexerDAO, are planning on releasing this educational content over quarter 2 and 3 of 2023 (April 1 through September 30). This is a team effort (as indicated in the grant application) and the project will be worked on by 2-4 IndexerDAO members. I submitted this application as a member of IndexerDAO.

All learning materials will be freely available to the community on GitHub and YouTube, with lessons released as they are completed so the community does not need to wait until we've finished everything. Anyone will be able to watch and follow along with tutorials as soon as they are published. We plan on holding synchronous workshops for interested community members but also leave open the option for asynchronous participation (especially since content may be translated later on and new community members will likely join).

Currently we do not have plans to localize our content but are very open to collaborations with Advocate Text Translators once our English content is produced. We can certainly add subtitles/captions to our video recordings to help facilitate localization by other Advocates. We are investigating GitBook as the platform to share our educational content: https://docs.gitbook.com/. This is the same platform utilized by the Graph Advocates (https://docs.graphadvocates.com/) and StreamingFast (https://substreams.streamingfast.io/). See GitBook Internationalization for more: https://docs.gitbook.com/tour/internationalization, https://docs.gitbook.com/getting-started/publishing/collection-publishing

$10000 USD for 10 lessons, with each requiring:

11-16 hours of work per guide

110-160 hours for 10 guides @ 10000 USD = $62.5 to $90/hr

reinasun commented 1 year ago

Also, what are thoughts on polling the community for input on their indexing wants/needs?

reinasun commented 1 year ago

Highly technical environment for the last 10+years, ranging from system admin, cloud security engineer, system auditor, and emerging technology project management/integration. Focused on web3 for the last 2+years. Experience is here: www.linkedin.com/in/shannonreinasol

alex-pakalniskis commented 1 year ago

Also, what are thoughts on polling the community for input on their indexing wants/needs?

@reinasun I think this is a great idea but I'd rather wait until we get this proof-of-concept tutorial series up and running before polling for feedback.

I think that getting this content out is important and we can always take feedback to create additional content, i.e. different chains and Ethereum clients, different server provider, etc.

Also it might be easier for the community to identify/recognize their wants when they can compare against our tutorials.

Thoughts?

reinasun commented 1 year ago

I agree and It's a great idea to create content as a foundation, then to use surveys or feedback to adjust if needed. One of the best things to do is to understand the audience

alex-pakalniskis commented 1 year ago

17th morning (Pacific time) co-working session

alex-pakalniskis commented 1 year ago

@Cuervodelmar could you walk through the three pages under this section? https://docs.indexerdao.com/indexing-101/linux-and-ethereum-clients

here are some leading questions/comments for your knowledge:

alex-pakalniskis commented 1 year ago

@idalithb are you interested in supporting this project by contributing to a lesson?

Here's the lesson I had in mind (its related to content you've been reading on your own).

Title: Intro to indexer operations

Contents: Wallets (hot vs cold, indexer vs operator), Multisigs, Faucets, Network Registration

Let me know if you're game and we can set up a sesh.

CuervoDelMar commented 1 year ago

@alex-pakalniskis

alex-pakalniskis commented 1 year ago

@CuervoDelMar here's another link to a different CLI tutorial I'm creating: https://gisc605.alexpakalniskis.com/lab-exercises/1-introduction-to-the-command-line Let me know if this sort of flow is preferable or not for Indexing 101 Lesson 1 (Intro to Linux).

For Lesson 2 please read through the whole thing. I just wanted to add a disclaimer in case it doesn't yet make total sense (sorry that wasn't more clear).

CuervoDelMar commented 1 year ago

@alex-pakalniskis for the intro page I think it would benefit from breaking them into more concise lessons. Just from a context of coming in cold to the world of blockchain.

Reading through the lesson it's pretty straight forward but I'm hitting a bit of a snag with the last part for installing Ethereum brew install ethereum --devel and receive the following: Error: invalid option: --devel

Other than that maybe a disclaimer before the installation steps mentions that some of these require homebrew installed via linux to help facilitate the CL and EL clients. Like "Some of these component require installation of Homebrew " just for like the total newbies .

alex-pakalniskis commented 1 year ago

TODO

reinasun commented 1 year ago

Hi all,

I'm having a difficult time connecting on Discord. Is it possible to find me on Discord and add me back on server?

I've attached my discord info.

Thank you, Shannon [image: Screenshot 2023-07-13 at 11.46.33 AM.png]

On Sun, Jun 25, 2023 at 10:55 AM Alex Pakalniskis @.***> wrote:

TODO

  • Get server access for Irving to test Lab 1
  • Update Lesson 1 to be more concise and migrate current content to appendix
  • Update Lesson 2 from macOS- to Ubuntu-based instructions

— Reply to this email directly, view it on GitHub https://github.com/IndexerDAO/IndexerDAO/issues/85#issuecomment-1606197119, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU2M7BKHJ4UTVMF7PAEQ5P3XNB3S5ANCNFSM6AAAAAAT5DJACY . You are receiving this because you were mentioned.Message ID: @.***>

CuervoDelMar commented 1 year ago

@alex-pakalniskis got to Step 8 of the SSH part (unsure where I'll be pulling the SSH key from to paste. Also I might've done it wrong since I tried to use VIM to modify th efile). Also wanted to share two screen shots from the other steps in Termius. 1) Install build-essential, ran it as is but just a head's up that this comes up.

1

2) After creating dev user. Should we leave these blank? Or do we want our learners to set something specific up. I was able to get through by leaving these blank

2
alex-pakalniskis commented 1 year ago

hey @reinasun

Your previous comment is formatted weird on my end: https://github.com/IndexerDAO/IndexerDAO/issues/85#issuecomment-1634730384

Could you reshare your Discord username and we can re-add you.

reinasun commented 1 year ago

Hi,

This was lost in my email. Sorry for the delay. my Discord user name is channon9821.

Thank you, Shannon

On Thu, Jul 20, 2023 at 2:16 PM Alex Pakalniskis @.***> wrote:

hey @reinasun https://github.com/reinasun

Your previous comment is formatted weird on my end: #85 (comment) https://github.com/IndexerDAO/IndexerDAO/issues/85#issuecomment-1634730384

Could you reshare your Discord username and we can re-add you.

— Reply to this email directly, view it on GitHub https://github.com/IndexerDAO/IndexerDAO/issues/85#issuecomment-1644619476, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU2M7BL2573HURQFBKMYNHLXRGNZFANCNFSM6AAAAAAT5DJACY . You are receiving this because you were mentioned.Message ID: @.***>

alex-pakalniskis commented 1 year ago

@reinasun I can't directly add you since it doesn't look like we're "friends" on Discord. Can you join via this link? (good for 7 days): https://discord.gg/yednt2T7

reinasun commented 1 year ago

[image: Screenshot 2023-08-03 at 12.09.30 PM.png]Hi Alex. unfortunately the invite expired. it's only been 6 hours

On Thu, Aug 3, 2023 at 6:08 AM Alex Pakalniskis @.***> wrote:

@reinasun https://github.com/reinasun I can't directly add you since it doesn't look like we're "friends" on Discord. Can you join via this link? (good for 7 days): https://discord.gg/yednt2T7

— Reply to this email directly, view it on GitHub https://github.com/IndexerDAO/IndexerDAO/issues/85#issuecomment-1663955648, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU2M7BPESI5SOQXSH6233PTXTOPGHANCNFSM6AAAAAAT5DJACY . You are receiving this because you were mentioned.Message ID: @.***>

alex-pakalniskis commented 1 year ago

@reinasun https://discord.gg/yednt2T7