search

treeverse / lakeFS

lakeFS - Data version control for your data lake | Git for data
https://docs.lakefs.io
Apache License 2.0
4.46k stars 359 forks source link

Update lakeFS "architecture" docs #4246

Closed arielshaqed closed 2 years ago

arielshaqed commented 2 years ago

https://docs.lakefs.io/understand/architecture.html says:

lakeFS stores data in an underlying object store..., with some of its metadata stored in PostgreSQL (see Versioning internals).

That's no longer really true 🥳

splitice commented 2 years ago

Does this mean its now (or soon) going to be possible to go S3 object store only?

This would be great news if true.

arielshaqed commented 2 years ago

Does this mean its now (or soon) going to be possible to go S3 object store only?

This would be great news if true.

Hi @splitice !

That's almost true. As #4247 is about to add, you still need a data store with a bit more atomicity than is possible on S3. But this is now PostgreSQL (used purely as a key-value) or DynamoDB.

It's still really really good news, though! :-)