search

hoophq / sequence

Immutable, scalable, and easy to use ledger service.
Apache License 2.0
492 stars 31 forks source link

Docs about data model, add comments to code #20

Open dguisinger opened 3 years ago

dguisinger commented 3 years ago

I am having a really hard time following your code (I admittedly have no Clojure experience which doesn't help), it seems the comments are just related to things that were fixed. I also see your NoSQL Workshop file, but there is nothing that I can just jump into and read how you are storing the data and using the generically named indexes of LSI1 and GSI1.

I would highly recommend adding more comments to your code to describe what functions are doing, and adding documentation of the database schema and row examples of what the raw data looks like.

I did read your blog post regarding Sequence at https://decimals.substack.com/p/things-i-wish-i-knew-before-building

One of the things I was looking for was your merkle tree code to see how you were handling this - but a search of your sources for merkle, tree, or root turns up nothing, so I don't know if you just called it something different or if its not implemented in the current version.

andriosrobert commented 3 years ago

Hey @dguisinger, these are great points!

So, on the data model documentation, we should probably add more docs on that.

But if you use AWS's No-SQL workbench and load the file in the dynamodb_data folder, you will see a few transactions and how they use the indexes. The workbench project also has a few descriptions of the fields.

Generic index names are a good thing for DynamoDB design, they let you overload the indexes, which we do in Sequence.

The docker-compose setup also uses this folder to store transactions, so you can make a few transactions and see what happens in the database.

This discussion had great critiques on the design of the DynamoDB tables: https://news.ycombinator.com/item?id=24432939

Now, on the Merkle Tree side: Sequence doesn't implement all of the design from this article yet. We build a hash chain in Dynamo, but Markle Trees are not yet implemented.