tablelandnetwork / weeknotes

A place for weekly updates from the @tablelelandnetwork team
0 stars 0 forks source link

[NOT-49] Weeknotes individual update: October 23, 2023 #48

Closed dtbuchholz closed 11 months ago

dtbuchholz commented 11 months ago

ETHOnline observations

by Dan Buchholz

We're less than a week away from rounding up ETHOnline. There has been a LOT of activity from developers building on Tableland in both the Tableland Discord as well as ETHGlobal's. It's still a bit early to know what types of applications developers are building, but in terms of how they're using the protocol, it's been all across the board.

The Studio is a hard requirement for any prizes, and the hackathon has been extremely helpful for producing value input, from bugs to new features to just, general, positive feedback. One of the goals for why we built the Studio was to make it easier for developers to get started without worry about all of the setup work. There are a lot of features planned to make that process even easier, but the developer activity has shown a significant uptick and points to the Studio enabling a much lower lift for newcoming devs.

Testing out Basin

by Dan Buchholz

We released the Basin MVP some weeks ago, and I've been testing various Postgres setups for compatibility checks as well as checking out some helpful tooling for Filecoin and Parquet. Recall that Basin let's you set up Postgres publications and replicate that data to Filecoin—once a Filecoin deal is made, that data can be extracted as Parquet for further usage.

There are some initial setup requirements to use Basin, but Supabase had most of these ready out-of-the-box. So, all it really required was running the basin publication create command with database connection information that Supabase displays in their Project settings > Database dashboard, and then the basin publication start command worked with relative ease.

After data is replicated, tools like lassie and car make it easy to extract data from a Filecoin deal's CID, and then the parquet-cli can help further manipulate that data locally. Parquet is a bit newer for me, so I figured I'd share some of this tooling for anyone in the same boat. For example, you can do something like:

> lassie fetch -o - bafybeia7g3c3avf7h5nt7ij27oytzo2jrmabye7taovvx3glo33w4dzbc4 | car extract

To retrieve the replicated data locally, and then parquet CLI lets use interact with the file to show its contents:

> parquet head -n 2 starter/my_table/export178ebbfc78a992150000000000000001-n909083943229489153.0.parquet

{"id": 0, "val": "val1"}
{"id": 1, "val": "val2"}

Lastly, DuckDB has a nice CLI tool where you can open up its shell and interact with parquet files directly:

> duckdb
v0.9.1 401c8061c6
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
SELECT * FROM READ_PARQUET('~/starter/my_table/export178ebbfc78a992150000000000000001-n909083943229489153.0.parquet') LIMIT 2;
┌───────┬─────────┐
│  id   │   val   │
│ int64 │ varchar │
├───────┼─────────┤
│     0 │ val1    │
│     1 │ val2    │
└───────┴─────────┘

From SyncLinear.com | NOT-49