Allow jobs to have return values, which get persisted in the DB

[saurabhnanda]

@jappeace @ivb-supercede any thoughts on this functionality / feature? Few notes:

• makes sense only when completed jobs are deleted after a delay. This allows an end-user/UI to trigger a job, and poll for the job's result/output (who's usefulness is ephemeral in nature, i.e. you can afford to lose this result after X days, when the job is deleted)
• enables one to write retryable worfklows of jobs (inspired by Temporal) -- which will be my next PR

[jappeace]

Yes this maybe usefull for quick n' dirty jobs where you don't want to make a dedicated table for the job to writeout in.

Good to see you back :)

[jappeace]

I left supercede, @RiugaBachi may also wish to comment.

[saurabhnanda]

Folks, this has a breaking change in the library's external API. Is everyone largely fine with this?

@jappeace @tfausak @tchoutri

(please tag other active users, as well)

[tchoutri]

No objection on my part

[jappeace]

Is there a way you could implement this without breaking existing code?

[tfausak]

I see why this is useful. My team wouldn't necessarily benefit from it, so I could take or leave this change.

Any change to the schema of the job table is unfortunate. However adding a new optional column is about as low impact as you can get.

I would appreciate a non-breaking release before any breaking change. We've been using a source-repository-package (rather than a Hackage release) for a while now.

[saurabhnanda]

@jappeace @tfausak one way to not break any existing code would be to have a build-time flag along with liberal use of CPP at the following places:

• OddJobs.Types.Job data type to add two extra fields if the CPP flag has been set. Two fields will be added if this flag is set:
  • jobResult
  • jobParentId
• saveJobQuery & saveJobIO & runJob
• defaultDelayedJobDeletion (added in #106 ), to ensure a child job is deleted only along with its parent
• there could be more places where CPP will need to be used.

is there any other way to do this? and if CPP is the only way, when do we get finally get rid of the CPP?

[jappeace]

I was thinking along the lines of: instead of changing this function in place, we add a new one and see if it can be made compatable, however I'm not sure if it's possible.

I'd not go the CPP/flag route as that incurs a rather troublsome maintenance burden and rather do the breakage.

[saurabhnanda]

I was thinking along the lines of: instead of changing this function in place, we add a new one and see if it can be made compatable,

@jappeace I think two versions of functions can still be dealt with, but what do you think about the Job{..} data type?

Does anyone use the Job{..} constructor in their application code? For creating a job if everyone is using the createJob function, then adding optional/nullable fields to Job{..} should not be. a problem...

[saurabhnanda]

I think the two DB columns that all users of the library are going to be forced to create are a greater problem than the Haskell code.... any ideas on how to tackle this?

[saurabhnanda]

@tchoutri @jappeace @tfausak what do you feel about the current attempt on making this backward compatibly and purely opt-in?