Open korenmiklos opened 2 years ago
Even for a small project where we hold the primary data and we have a chain (with some small branches), we have 12 steps to update the final bead and everything has to be run in the correct order. This is much more difficult for large-scale project using different datasets. Ideally, once we have access to bead boxes, we can
bead make cegjegyzek-LTS-2020
Question: where do intermediate beads get saved into? The same box they were found in?
Use case:
bead make final-analysis --box=local
will open the bead
final-analysis
, check its inputs, whether they can be updated. If so, updates them and computes the bead with the entry point (likemake
), then issuingbead save local
.For an individual bead, this could be called bead update, with bead make going through all upstream inputs to ensure that all dependencies are up to date.
https://en.wikipedia.org/wiki/Topological_sorting https://pypi.org/project/toposort/