search

datonic / datadex

📦 Serverless and local-first Open Data Platform
http://datadex.datonic.io
MIT License
220 stars 14 forks source link

Add example importing assets from another Datadex instance #29

Closed davidgasquez closed 3 months ago

davidgasquez commented 9 months ago

Try importing the filecoin-data-portal DAG!

davidgasquez commented 5 months ago

Say someone wants to create another project and reuse some of the assets here. How can we make it so they can reuse the output or rerun the computations from scratch?

I'm imagining a future case where you have some projects dependencies (p3 needs p2 that needs p1). Someone working on p4 should be able to run everything themselves if they want in their project as everything is Dagster and dbt. But they also should be able to run it "trusting" the outputs of p3 CI.

davidgasquez commented 5 months ago

For now, you can import datadex but the dbt models won't be compatible with any database other than DuckDB.