Closed mtuchi closed 1 year ago
@josephjclark is working on a fix, that it will make sure that if there are two downstream jobs that needs the final state of the upstream job, They will both use the final state of that upstream job as their initial state
I am looking at a fix, it needs a bit of thought. But it's not blocking!
a) Lightning doesn't use the CLI's workflows yet (won't for a while) b) It's quite reasonable to return state at the end of each job.
To be fair, as this is a demo, we should also be encouraging better practice of state. We WANT users to write all shared state to state.data
. You shouldn't need to write code to promote it to the top of the state object - just let it sit on data.
So yes, I'm looking into the issues, but a cleaner version of this should be compatible with Lightning, the CLI, and the pattersn we're trying to encourage :)
Notes for the reviewer
This PR address the request in #3 , I have updated the required jobs for the demo and run the jobs on app.openfn.org
Something to note when testing the job locally using the
@openfn/cli
workflow.json
. There is a bug in runtime when passing the final state of the upstream job to two downstream jobs. Eg The final state ofvalidate
, Is supposed to be passed toconvert
anddhis2
but at the momentdhis2
job doesn't get its initial state fromvalidate
and hence it will fail when running it locally.@josephjclark is working on a fix, that it will make sure that if there are two downstream jobs that needs the final state of the upstream job, They will both use the final state of that upstream job as their initial state