Open nelsonic opened 5 years ago
At present the README.md
is a bit sparse at the start of PR #2:
https://github.com/dwyl/phoenix-ecto-append-only-log-example/pull/2/files#diff-04c6e90faac2675aa89e2176d2eec7d8R4
I realise that I am responsible for making the intro clear and beginner friendly.
I will add to the README.md
as soon as I get a chance.
Database Migrations are Always Backward Compatible. "Tables" are only ever extended with new columns. Columns are never deleted or altered so existing code/queries never "break". This is essential for "Zero Downtime Continuous Deployment". A Database Migration can be applied before the Application Server is Updated and the existing/current version of the Application can continue to run like nothing happened.
@Cleop given your knowledge quest in #5 do you feel that you can write the intro to this tutorial and do a general tidy/tightening so we can "publish" it to the Elixir community? Please let me know if you feel confident that you can approach the first 3 questions with a "beginners mind" and "sell" the idea of an append-only log to people who are "curious" but not yet "convinced":
If you have any questions, as always, I'm happy to answer/clarify. π |> π€ |> π |> π€ |> π LMK! π Thanks! β¨
Append only logs are an attractive approach to database structures because:
Here are some examples of use cases where these characteristics may prove useful:
The Phoenix Append Only Log Example is an immutable database structure.
It is an alternative to using Ecto's standard "CRUD" which allows overwriting and deleting data without "rollback" or "recoverability". In these instances each update over-writes the state of the record so it's impossible to retrieve it without having to go digging through a backup which is often a multi-day process, cost/time prohibitive or simply unavailable.
Instead the Phoenix Append Only Log Example "tables" are only ever extended with new columns. This means that whilst changes can be displayed as you wish to your users, data is never irretrievable or lost to you on the back-end. It is also a time series database meaning that whatever activity occurs, we also know when it occurred, making understanding chronology and how events occurred easy.
All developers who have a basic understanding of database storage in web development and want to "level up" their knowledge/skills. Those who want to improve the reliability of the product they are building. Those who want to understand more ("advanced") "distributed" application architecture including the ability to (optionally/incrementally) use IPFS and Blockchain.
Hi @Cleop thank you for summarising the intro questions/answers.
Please create a Pull Request with the bullet points you have written and an intro paragraph. (Thanks!)
The Why? section should answer the question: "Why should I care about this?" (or "Why should I take the extra steps to use an Append Only Log?")
Seek to answer the question "What's in it for me?" in the first 7 seconds of the readme. This the question going through the reader's head, and we can either engage them with a reason to continue reading, or they will "bounce".
Bullet points are good and have their place, keep them! Consider an intro paragraph (prose) which answers the questions going through the reader's mind and will "flow" better when a human is reading it.
e.g:
If you have ever used the "undo" functionality in a program, you will have experienced the power of an Append-only Log.
When changes always create a new state (without altering history) we can easily return to the previous state. This is the principal in the Elm Architecture (borrowed by Redux so most React apps) and in Elixir/Haskell too; data is always "transformed" never mutated. This makes it much faster to build reliable/predictable apps because debugging apps is considerably easier when state is always known.
If any these terms are unclear to you now, never fear we will be clarifying them below. The main thing to remember is that using an Append-only Log to store your App's data makes it much easier to build the App almost immediately because records are never changed, history is preserved and can easily be referred to i.e. you have built-in debug-ability/traceability.
Once you are overcome the initial learning curve, you will see that your Apps become easy to reason about and you "unlock" many other possibilities for useful features and functionality that will delight the users! You will get your work done much faster and more reliably, users will be happier with the UX and Product Owners/Managers will be able to see how data is transformed in the app; easily visualise the usage data and "flow" on analytics charts/graphs in realtime!
I'm sure you can re-word/work this to read bettererer.
intro questions answered in by @Cleop's PR #9 π
Oops. I closed the issue but still has un-finished items in the checklist...! (derp!) π Re-opening and ensuring that we cover the checklist items. β³
Why?
Having an append-only log is incredibly useful in way more situations than most people realise. Anywhere you would need accountability in data is an excellent candidate for immutability.
When a system/db does not have (field/record level) "version control" each update over-writes the state of the record so it's impossible to retrieve it without having to go digging through a backup which is often a multi-day process, cost/time prohibitive or simply unavailable.
We propose that all apps should be built with an Append Only Log at the core by default. This is not a "new" idea. Several large companies have used the "Lambda" or "Kappa" architecture in production with superb results for reliability and scalability. see: http://milinda.pathirage.org/kappa-architecture.com
What?
Instead of using
Ecto
's standard "CRUD" which allows overwriting and deleting data without "rollback" or "recoverability", we propose writing a thin "proxy" layer between the application code and the PostgreSQL databaseWho?
All developers who have basic understanding of web development where data is stored in a DB and want to "level up" their knowledge/skills and the reliability of the product they are building with a view to understanding more ("advanced") "distributed" application architecture including the ability to (optionally/incrementally) use IPFS and Blockchain.
How?
The purpose of this tutorial is to:
Open Questions:
delete
data (mark an item as deleted) without "destroying" the data.Similar to: (please use these a reference when writing the doc(s))