diagonalworks / diagonal-b6

b6: Diagonal's geospatial analysis engine.
Apache License 2.0
24 stars 3 forks source link

documenting system #281

Open gacafe opened 3 months ago

gacafe commented 3 months ago

ingest pipeline:

world:

search:

for ingest pipeline / for each data source (binary)

gacafe commented 3 months ago

@xprogramari two separate things - one is a high level architecture diagram + compact and ingest pipeline interaction

xprogramari commented 3 months ago

i will start with general ingest pipeline / diagram !

xprogramari commented 2 months ago

(will send a separate cl for diagrams / readme stuff)

xprogramari commented 2 months ago

for the ingest binaries, i have double checked that each binary documents the following in flags -

xprogramari commented 2 months ago

binaries also document type of data they need (e.g. GTFS / OSM etc) but they don't lead to links where these live online for example, or where they live in our cloud - i suggest we document that part in k8s pipelines, as that seems to make most sense / is how data ingestion will be run; i will add that to pipeline ticket