0xPolygonZero / zk_evm

Apache License 2.0
86 stars 38 forks source link

RFC: clean up repos #741

Open 0xaatif opened 1 month ago

0xaatif commented 1 month ago

Here's a table of all the Rust repos we have in our org. I think this surface area is too big. If we have active code, let's keep it in our active repos, where we can keep it current, tested, audited etc. Otherwise let's archive it. e.g smt_utils, eth_trie_tools should actually just be (crates or modules) in the zk_evm repo, where there's already trie diffing stuff.

Here's a strawman proposal

proposal repo stale (months) description
archive or move to plonky plonky2-u32 6 u32 arithmetic gadgets for plonky2
archive or move to plonky2 CircleStark 🔒 9 Plonky3 fork for circle stark development
archive or move to plonky2 plonky-edge-block-trace-parser 12 Logic for parsing a block trace for edge into txn generation payloads for plonky (2 & 3).
archive or move to plonky2 plonky2-ecdsa 13 ECDSA gadgets for plonky2
archive or move to zk_evm eth_trie_utils 9 Types and utility functions for building/working with partial Ethereum tries
archive or move to zk_evm eth-trie-tools 5 Various debugging tools for working with tries for eth_trie_utils.
archive or move to zk_evm eth-tx-proof 3 Proving historical Ethereum transactions using the Plonky2 zkEVM
archive or move to zk_evm proof-protocol-decoder 9 Common protocol for generating ZK proofs for blocks on different blockchains.
archive or move to zk_evm smt_utils 12 Utilities related to the Sparse Merkle Tree used for type2 provers
archive? who uses this? evm-tests 1 Parser and test runner for testing compatable common Ethereum full node tests against Polygon Zero's EVM.
archive? who uses this? fast-mds-check 🔒 8 Code to check if a matrix has the MDS property.
archive? who uses this? nums 5 Number theoretic functions, such as primality testing and factorization, for BigUints
archive plonky2-insertion 20 list-insertion gadget for plonky2
archive plonky2-waksman 20 permutation gadgets using the Waksman network for plonky2
archive system-zero 19 An obsolete zkVM from Polygon Zero
archive zero-provers 🔒 13 Prover clients for Polygon Zero
keep paladin 1 Bringing divine order to remote task execution.
keep plonky2 1
keep zk_evm 1
keep archived hyperstark 📥 18 Early prototype.
keep archived plonky-block-proof-gen 📥 10 Generates block proofs from proof IR
keep archived zero-bin 📥 5 plonky2 proving over paladin
Bootstrapping script ```python #!/usr/bin/env python import json import subprocess import sys from datetime import datetime from dateutil.rrule import rrule, MONTHLY from dateutil.parser import parse as parse_date from typing import TypedDict, Literal def eprintln(s: str): print(s, file = sys.stderr) class Repo(TypedDict): description: str isArchived: bool isFork: bool isMirror: bool name: str pushedAt: str url: str visibility: Literal["PUBLIC"] | Literal["PRIVATE"] | Literal["INTERNAL"] completed = subprocess.run( [ "gh", "repo", "list", "0xPolygonZero", "--json", "description,isArchived,isFork,isMirror,name,primaryLanguage,pushedAt,url,visibility", "--jq", '[ .[] | select(.primaryLanguage.name == "Rust") | del(.primaryLanguage) ]' ], check=True, capture_output=True, ) now = datetime.now() repos: list[Repo] = json.loads(completed.stdout) eprintln(f"{len(repos)=}") print("| repo | stale (months) | description |") print("|------|----------------|-------------|") for repo in repos: name = " ".join([ f"[{repo['name']}]({repo['url']})", "".join([ "📥" if repo["isArchived"] else "", "🍴" if repo["isFork"] else "", "🪞" if repo["isMirror"] else "", "🔒" if repo["visibility"] in ("PRIVATE, INTERNAL") else "", ]) ]) months = sum(1 for _ in rrule(MONTHLY, dtstart=parse_date(repo["pushedAt"]).date(), until=now.date())) description = repo["description"] print(f"| {name}| {months} | {description} |") ```
Clone all repos ```bash #/usr/bin/env bash set -euxo pipefail pushd 0xPolygonZero gh repo list 0xPolygonZero \ --limit 1000 \ --json 'sshUrl,primaryLanguage' \ --jq '.[] | select(.primaryLanguage.name == "Rust") | .sshUrl' \ | while read -r ssh_url _; do git clone "$ssh_url" & done wait popd ```
Nashtare commented 3 weeks ago

A few notes:

BGluth commented 2 weeks ago

Just to add a few other notes: