Closed jmaassen closed 7 years ago
We decided on the following:
JobHandle
and CopyHandle
with a simple String
identifier. Scheduler
and FileSystem
are created by the user, and all parameters needed to create them can be serialized, we assume the user can store these without our help.Fixed in ee060e6b61c6a0fe1afc828a1f8f25570abb949e
As per request of @nielsdrost and @bpmweel we should think about serializability of the basic Xenon objects (to whatever format, it does not necessarily have to be java.io.Serializable but could also be json, protocol buffers, etc).
For projects building services on top of Xenon, such as xenon-cwl, it is very useful to implement persistence by storing certain objects on disk or in a database.
For some objects this is trivial (
JobDescription
,Path
).Others you cannot serialize directly as they represent an active connection (
Scheduler
,FileSystem
). You can, however, extract the required information to recreate this connection later, typically adaptor + location + username (+ credential, but you do not want to store this?) + properties.Some objects (
JobHandle
) you would like to be able to serialize, but cannot at the moment because it contains a non-serializableScheduler
as a field. In addition, there is no way to create aJobHandle
directly, even if you have all the pieces you need.There are a number of solutions possible here (not mutually exclusive):
Turn
JobHandle
into a bean the user can create directly. It is up to the user to ensure the correct combination ofJobDescription
, job identifier, andScheduler
is used. If is also up to the user to figure out how to serialize aScheduler
andFileSystem
.Simplify
JobHandle
to only contain aJobDescription
andString
job identifier. It would then be up to the user to remember whichJobHandle
goes with whichScheduler
. SerializingScheduler
andFileSystem
are still the users problem.Add
SchedulerDescription
andFileSystemDescription
beans containing all the information on scheduler that you can easily serialize. Also allow the user to create a newScheduler
orFileSystem
using these descriptions. We can optionally add aSchedulerDescription
toJobHandle
.The last options seems closest to the overall solution to me. This may also make the implementation of
xenon-grpc by @sverhoeven simpler ?