man-group / ArcticDB

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.
http://arcticdb.io
Other
1.26k stars 80 forks source link

Write docs explaining different abstractions #1431

Open IvoDD opened 4 months ago

IvoDD commented 4 months ago

Is your feature request related to a problem? Please describe. The current higher level code structure is confusing and has potentially too many abstractions:

PythonVersionStore ---has---> Store ---parent-of---> AsyncStore ---has---> Library ---has---> Storage ---parent-of---> Many
                               |
                               v
                         InMemoryStore

Most of the abstractions in this picture don't do more than just directly call methods from the next portion of the hierarchy.

Also I don't think there is a good reason for InMemoryStore to be a Store rather than a Storage.

Describe the solution you'd like

~I suggest a refactor to simplify these abstractions to something like:~

~PythonVersionStore ---> Library ---> Storage ---parent-of---> Many (including InMemory)~

~Here PythonVersionStore will be responsible for the translation between Python and c++.~

~Library will be the main point-of-entry exposing all library operations we expose to users.~

~Storage will be the common interface over all storage types (S3, InMemory, Azure, etc.).~

Write down decent inline documentation explaining the function of each abstraction at the top of the header file.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

IvoDD commented 4 months ago

As discussed with @willdealtry, the various abstractions are meaningful and each has a specific purpose. It is better to have more abstractions with a single purpose than few abstractions with many functions.

Instead we should write up good header docs to explain the function of each abstraction.