[Top Builder 2024]: Bitcoineando_R2

[esixce]

🏗️ Top Builder 2024 Application Form to track progress through Round 1 - 3 ~ Currently in Round 1

📝 Instructions

1. Only complete this form if you have been chosen for Top Builder, by PlebLab.
2. Provide detailed information to assist the judges in evaluating your project.

🧑‍🤝‍🧑 The Team

• Team Name: Bitcoineando

• Number of Founders: 1

• Team Background: Share the professional backgrounds of each founder. I come from a family of filmmakers and only started my engineering career six years ago. While honing my computer skills, I did film gigs for Bitcoiners, such as editing for Tone Vays and coordinating for ‘Searching for Satoshi’ by Paul Kemp. I graduated with Honors from the Johns Hopkins master's program, if that still means anything. I’m here to learn.

• Bitcoin & Lightning Involvement: Describe your involvement and experience. I’m in the Chaincode Bitcoin Foss seminar and Programming Bitcoin in February. I also make Bitcoin art under the btc_cpr pseudonym. I just got hired to host a Bitcoin technical podcast; the first episode is on mnemonic seeds.

• Origin Story: What inspired your idea? I’m concerned that the people who would benefit the most from embracing Bitcoin will not because of the lack of access to adequate resources. A monthly subscription of $30 USD may seem trivial to a person earning in dollars, but it amounts to one-fifth of the minimum monthly wage in Colombia.

💡 The Idea

• Project Overview: What does your project do? The objective is to leverage automation to build the highest quality web application with the smallest financial footprint to provide educational access to Bitcoin metrics and statistics in Spanish and English.

This is not another price action tracker. The idea is to step away from traditional interpretations of money and look at Bitcoin for what it is. I’m interested in charts that allow the client to pick the variables. I want to expose the complexity that mainstream Bitcoin seeks to hide.

• Website: (Optional) (purchased but not configured) bitcoineando.com

• GitHub: Prototype:
 https://github.com/esixce/bitcoinDash https://github.com/esixce/back-dash-btc https://github.com/esixce/btcRpc https://github.com/esixce/bitcoinRPC

• Project Stage: Describe the current progress. I built a Prototype about a year ago but had to finish school, so currently, components are not integrated. Working on MVP.

• Supplementary Materials: Provide any proof of work, workshops, or demo videos. Provide any proof of work, workshops, or demo videos.

📚 Additional Information

• Additional Links/Info:

📌 Submission Type

• [x] Initial Project Creation

🚀 Current Project Progress

• Progress Made: Wrote a project plan with Vision, Requirements, and Design. Queried the node using RPC to create an RDB; however, this is being replaced with Abe. Created a prototype of the frontend in javascript which I’m in the process of replacing with a Typescript Angular Framework. Created an API in Python for the prototype. The final version may employ Python for data analytics or node queries, but I’m currently writing the WebApp backend in Rust. Experimented with different graphing libraries.

• Challenges Faced: Challenges extracting all the data from the node related to its sheer volume. Challenges related to analyzing the data due to “outliers”, the sheer volume, and the vast number of ways the data can be analyzed. Challenges providing large datasets to the frontend in an efficient way Challenges providing fast response to interactive components

• Future Roadmap of what's next: Complete the extraction of blockchain data and set up automation for future extractions Complete the analysis of data and create additional intermediary tables Complete the API to feed the selected datasets to the Client Complete the graphing for the provided datasets. Translate everything.

⏳ Round 1 Milestone

• Keep in mind that completion will need to be by February 10th, 2024.
• First-round milestone the team has set for themselves: Keep in mind that completion will need to be by February 10th, 2024. First-round milestone the team has set for themselves: Complete the extraction of blockchain data and set up automation for future extractions Have the basic component and integration in play:
• Hosting Frontend.
• Hosting backend.
• Hosting node.
• Hosting RDB
• Establishing communication between frontend and backend via HTTP.
• Establishing communication between frontend and backend via WebSockets.
• Establishing communication between the backend and the RDS.
• Configure ZeroMQ for real-time blockchain updates. Round 2 Milestone:
 Complete the analysis of data and create additional intermediary tables. Complete the API to feed the selected datasets to the Client. Round 3 Milestone: Complete the graphing for the provided datasets. Dream Milestone: Translate Everything

📝 Submission Complete

• All required information has been provided above.

📅 Top Builder Program Structure

Registration Opens: Monday, December 18, 2023

• Application: Interested teams apply for Top Builder.
• Selection: Teams will be chosen based on their application. See FAQ for details.

Round 1: January 13 - February 10, 2024

• Workshops: PlebLab weekly workshops start, available live and online.
• Chat Access: Selected teams join the PlebLab internal chat.
• Mentorship: Begins for teams.
• Project Milestones: Teams are set and judged on their first-round milestones.

Round 2: February 10 - 24, 2024

• Team Reduction: PlebLab cuts half of the Top Builder teams. (Only half continued in Round 2.)
• Project Milestones: Teams are set and judged on their second-round milestones.
• Workshops: PlebLab weekly workshops continue.
• Mentorship: Continues.

Round 3: Starting February 24, 2024

• Team Reduction: PlebLab narrows down to 6 Top Builder teams. (All will be pitching in Austin.)
• Final Milestones: Teams set final project milestones for the last round.
• Preparation: Teams prepare their pitch in front of judges. Final teams are contacted to come to Austin for Startup Day.

Final Round at Startup Day: March 14, 2024

• Preparation: Teams work on their final pitches at PlebLab.
• Presentations: Final teams present before judges on Startup Day.
• Deliberation: Judges decide during a happy hour.
• Announcement: The winner of Top Builder is announced at Austin BitDevs.
• Celebration: After-party for Top Builder.

[esixce]

Round One Progress

I was able to fix the configurations of the prototype or, rather, the sketch of the site.

bitcoiniando.com

Github repos:

• btc-dash
• btc-dash-back
• btc-dash-sql
• btc-dash-conv

Completed Round 1 Milestones:

• Wrote base Angular frontend with login
• Wrote base Rust backend with auth, OAuth
• Established communication between frontend and backend via HTTP.
• Established communication between the backend and the RDS.
• Node running in EC2 instance
• Hosting mysql RDB in AWS
• MySql scripts for block, transaction, coinbase, vin, vout, user

Pending:

• Configure ZeroMQ for real-time blockchain updates.
• Establish communication between frontend and backend via WebSockets.

Additional Deliverables for Milestone 1:

• In the process of populating rdb using python rpc script (@400thou' blocksATM)
• Creating dataset for Treemap
• Creating API endpoint to feed Treemap data to client
• Creating Treemap visual of Blocks, Txs, Vin/Vout

Treemap D3 Angular integrated Pending: styling and get data from api

Relational database with blocks, txs, vout, addresses, coinbase. Sneak peak:

ER Diagram for relational database.

• pending: cumulative fields. this will be done tomorrow since its required for the sample visual. we are gathering historical data from the node so I don't want to interfere with the db.

[esixce]

Round One - Milestone Deliverable Report

Overview

During Week 1, we successfully completed several milestones as outlined below. The focus was on setting up the basic infrastructure, establishing communication between frontend and backend, and laying the groundwork for real-time updates and data visualization.

Completed Milestones

1. Base Angular Frontend with Login

   • Developed the frontend interface using Angular framework.
   • Implemented user authentication and login functionality.

2. Base Rust Backend with Auth, OAuth

   • Created a backend server using Rust.
   • Implemented authentication mechanisms including OAuth.

3. Communication Setup: Frontend to Backend (HTTP)

   • Established communication channels between the frontend and backend using HTTP protocols.

4. Communication Setup: Backend to RDS

   • Enabled communication between the backend server and the Relational Database Service (RDS).

5. Node Running in EC2 Instance

   • Successfully deployed the backend server on an EC2 instance.

6. Hosting MySQL RDB in AWS

   • Set up and configured a MySQL database on AWS for storing blockchain data.

7. MySQL Scripts for Various Data Entities

   • Developed MySQL scripts for handling block, transaction, coinbase, vin, vout, and user data.

Pending Milestones

1. Configure ZeroMQ for Real-time Blockchain Updates

   • Pending setup of ZeroMQ to enable real-time updates from the blockchain.

2. Establish Communication: Frontend to Backend (WebSockets)

   • Need to establish WebSocket communication between the frontend and backend for enhanced real-time interaction.

Additional Deliverables for Milestone 1

1. Populating RDB Using Python RPC Script

   • Currently in the process of populating the MySQL database using a Python RPC script, focusing on blocks at the 400,000th block.

2. Creating Dataset for Treemap

   • In progress: Creating a dataset suitable for generating a Treemap visualization.

3. Creating API Endpoint for Treemap Data

   • Planning to develop an API endpoint to provide Treemap data to the client application.

4. Creating Treemap Visual of Blocks, Transactions, Vin/Vout

   • Intending to generate a Treemap visualization depicting blocks, transactions, and inputs/outputs.

Conclusion

The completion of these milestones marks significant progress towards our project goals. With pending tasks identified and additional deliverables outlined, we are well-positioned to advance to the next phase of development.

Blocks with data

Login System frontend

Cumulative fields for datasets

Querying backend from browser to get chart data

[esixce]

Round Two - Progress Chart

Overview

The following chart summarizes the progress made during Round Two of our project.	Original Milestone	Status
Hosting Frontend	Completed
Hosting Backend	In Progress
Hosting Node	Completed
Hosting RDB	Completed
Communication between frontend and backend via HTTP	Completed
Communication between frontend and backend via WebSockets	In Progress
Communication between the backend and the RDS	Completed
Extraction of blockchain data	In Progress
Automation of blockchain data	In Progress
Configure ZeroMQ for real-time blockchain updates	In Progress
Complete analysis of data and create additional intermediary tables	In Progress
Complete API to feed selected datasets to the Client	In Progress
Complete graphing for provided datasets	Not Started
Translate Everything	Not Started

Working towards achieving the flow seen in the diagram below:

Obstacles and Achievements

Discarded original EC2 instance and built new one from scratch with clean new node. Configured node to accept RPC calls, ZeroMQ, and mempool size. Moved the python script that is collecting data from the node to the EC2 instance. Configured the script to collect data from the new node on the new EC2 instance and post it to the mysql rds. Currently at 130'000. The script runs much faster in the EC2 instance than on my computer. Currently transferring data 24/7 at an approximate rate of 1 block per 3 seconds. Experimented with scripts to query the mempool and alert on desired behavior.

Hosted frontend and hosted backend. Not ready for general audience access. Pending:

• Enable the legitimate authentication that has been built but not configured.
• Use the auth to build the sockets.
• Serve required data to frontend through sockets
• Configure Backend and Zeromq
• Configure python scripts and Zeromq
• Create frontend tickers for zeromq data
• Create generic line graph
• Create additional visuals
• Create treemap for mempool
• Create table explorer template
• Select method to determine bitcoin price
• Query electricity prices API
• Define tools for multilanguage
• Analyze mining pool block templates
• Analyze mempool anomalies
• Restrict security configurations
• Release

There's four major fronts:

• The administrative and logistical: getting authentication to work properly so we can have socket connections.
• The practical: Deploying the backend and front and getting the differences between local en remote environment correct.
• The creative and technical: Getting the datasets for the visuals to be interesting and accurate.
• The financial: A Bitcoin dashboard is just like a right of passage. Its something that anyone can do but requires a lot of grunt work. The dashboard is to gain the credibility and legitimacy to then do other things because the only innovation you can bring to a bitcoin dashboard is in the visuals since the data has been analyzed by many. The interesting question is "What comes next?"

Been doing research on what analysis to run on the Bitcoin mempool. Its too expensive to collect all the mempool data so the only viable avenue is to run real-time analytics. I'm particularly interested in monitoring anomalies:

• Sudden spikes in Volume
• Sudden spikes in Fees (fee attacks)
• Unusually large transactions (inputs, outputs)
• Almost identical transactions (malleability attacks)
• Stuck transactions (censorship attacks)
• Small amounts to many addresses (dusting attacks)
• Congesting network with small fees (transaction spam)
• Meaningless data to pack blocks (transaction floods)

Then I can perform statistical analysis on the anomaly metrics in addition to the usual suspects like size over time or maybe it's more interesting to write a knapsack and show a sample block.

Conclusion

Ideas don't come fully formed. They take shape over time. You have to walk in shallow water to get to the deeper end of the pool. I feel like the blockhain data part of the project is like cleaning your room and putting all the toys in the correct drawer. The real magic happens when youre done cleaning your room, and you know where all the lego pieces are and you make a masterpiece.

The work im doing is tedious. Its not fancy. Its grunt work. But it holds potential energy. Eventually the scripts will be running in the background and it wont take physical labour for the structure to work. Then the data will work for me instead of me having to work for the data. I already felt that with the python script running in the ec2 instance.

[esixce]

Round Two - Milestone Deliverable Report

Overview

The following chart summarizes the progress made during Round Two of our project.	Original Milestone	Status	Date
Hosting Frontend	Completed	02/15
Hosting Backend	Completed	02/18
Hosting Node	Completed	01/14 & 02/10
Hosting RDB	Completed	01/14
Communication between frontend and backend via HTTP	Completed	01/30
Communication between frontend and backend via WebSockets	In Progress
Communication between the backend and the RDS	Completed	01/30
Extraction of blockchain data	In Progress	since 01/30
Automation of blockchain data	In Progress
Configure ZeroMQ for real-time blockchain updates	In Progress
Complete analysis of data and create additional intermediary tables	In Progress
Complete API to feed selected datasets to the Client	In Progress
Complete graphing for provided datasets	Not Started
Translate Everything	Not Started

Obstacles and achievements

API is persistent