search

ceumicrodata / bead

Interlinked frozen computations (work in progress)
The Unlicense
1 stars 0 forks source link

Design `bead make` command #2

Open korenmiklos opened 2 years ago

korenmiklos commented 2 years ago

Use case:

bead make final-analysis --box=local

will open the bead final-analysis, check its inputs, whether they can be updated. If so, updates them and computes the bead with the entry point (like make), then issuing bead save local.

For an individual bead, this could be called bead update, with bead make going through all upstream inputs to ensure that all dependencies are up to date.

https://en.wikipedia.org/wiki/Topological_sorting https://pypi.org/project/toposort/

korenmiklos commented 2 years ago

Even for a small project where we hold the primary data and we have a chain (with some small branches), we have 12 steps to update the final bead and everything has to be run in the correct order. This is much more difficult for large-scale project using different datasets. Ideally, once we have access to bead boxes, we can

bead make cegjegyzek-LTS-2020

Question: where do intermediate beads get saved into? The same box they were found in?