search

FabienArcellier / writer-framework

No-code in the front, Python in the back. An open-source framework for creating data apps.
https://streamsync.cloud
Apache License 2.0
0 stars 1 forks source link

store writer framework IDE into project directory instead of ui.json #67

Open FabienArcellier opened 2 months ago

FabienArcellier commented 2 months ago

Storing writer framework IDE files poses several challenges when several developers are working on the same application.

data managed by the IDE is stored in .wf. This folder contains files that are generated and read by the writer framework IDE. These files are not intended to be edited by hand.

/.wf

.
├── components-root.jsonl
├── metadata.json
├── components-page-0-c0f99a9e50044e75a6c636f17490b134.jsonl

a file is dedicated to each high-level concept. This reduces the risk of conflicts when several developers work on the same application.

the json line is used for storage. This format defines a json record on each line. It is simple to parse. Conflict resolution is also simplified because a conflict takes place between 2 lines.

/.wf/components-page-0-c0f99a9e50044e75a6c636f17490b134.jsonl

{ "id": "c0f99a9e-5004-4e75-a6c6-36f17490b134", "type": "page", "content": {"pageMode": "compact", "emptinessColor": "#e9eef1" }, "isCodeManaged": false, "position": 0, "parentId": "root", "handlers": {}, "visible": true }
{ "id": "28d3885b-0fb8-4d41-97c6-978540015431", "type": "section", "content": { "title": "", "containerShadow": "0px 4px 11px -12px #000000" }, "isCodeManaged": false, "position": 0, "parentId": "c0f99a9e-5004-4e75-a6c6-36f17490b134", "handlers": {}, "visible": true }
{ "id": "d4a5e62c-c6fe-49c4-80d4-33862af8727d", "type": "columns", "content": {}, "isCodeManaged": false, "position": 0, "parentId": "28d3885b-0fb8-4d41-97c6-978540015431", "handlers": {}, "visible": true }

/.wf/metadata.json

{ "writer_version": "0.6.2rc3" }

loading and writing the contents of /.wf is atomic. Writer framework loads data all at once in the form of a consistent data structure. This approach keep the complexity of writer framework low.

wf_project_files_write(app_path: str, version: str, components: dict, workflows: dict): # save the project files in ./wf
version, components, workflows = wf_project_files_read(app_path: str): # read the project files from ./wf
FabienArcellier commented 2 months ago

suggestion of @madeinjs :

What prevent us to use one JSON file per component ?

Here a quick example of a Tree structure like /index.json//index.json where index.json contains the component definition.

Ok, it requires to walk in the directory but I see few advantages:

  1. git tracks well the files moves if the page
  2. we don't have to keep the ordering of lines
  3. the tree structure gives a small idea of the APP
  4. we don't have to load/parse the whole APP components to get a component
.
└── root
    ├── 1d195388-35a3-43e1-b825-1d263b100a28
    │   ├── 90fbfa9d-3178-4fc2-b445-c31e1acfa6a7
    │   │   └── index.json
    │   └── index.json
    ├── 771dc336-69b2-400e-9ea3-e881e2332c9d
    │   └── index.json
    ├── 9c77aee4-e2a0-4e8b-9c2b-377f939bb51e
    │   └── index.json
    └── index.json

That may be interesting. We would have to think of .wf as similar architecture of .git directory. The user should not open it.

  1. OK
  2. OK
  3. OK
  4. this one is tricky, the protocol complexity will increase if we load component one by one. I would prefer to keep it as it is today
.wf
└── components
    ├── 1d195388-35a3-43e1-b825-1d263b100a28
    │   ├── 90fbfa9d-3178-4fc2-b445-c31e1acfa6a7
    │   │   └── index.json
    │   └── index.json
    ├── 771dc336-69b2-400e-9ea3-e881e2332c9d
    │   └── index.json
    ├── 9c77aee4-e2a0-4e8b-9c2b-377f939bb51e
    │   └── index.json
    └── index.json
└── workflows
└── metadata.json