populse / capsul

Collaborative Analysis Platform : Simple, Unifying, Lean
Other
7 stars 14 forks source link

New completion API #209

Closed sapetnioc closed 2 years ago

sapetnioc commented 2 years ago

In new API, define a filenames completion feature for processes and pipelines.

sapetnioc commented 2 years ago

Being working on this issue, I need to make a link between configuration and execution context. More precisely, I need to know what are the datasets defined for a given execution context in order to be able to perform path generation (by the way, I propose to use path generation instead of completion it seems more clear to me). Even if the configuration is specific to a computing resource, the path generation must be available on client side. What I propose for this is to split ExecutionContext in two parts:

I am not very found of PureExecutionContext name but I have no better idea. It is inspired by the pathlib module.

denisri commented 2 years ago

"Pure" is not very self-explanatory, indeed. We can imagine other names: "virtual" because it cannot run on the client, "remote" because it represents a remote engine, or "client" because it's the client part (which tells the contrary of the previous proposition, and thus I'm not fond of it). I'm not totally sure we really need to separate/distinguish them anyway ? As we already have an "engine" which tells about the remote status ? But I haven't thought about it.

sapetnioc commented 2 years ago

Another option could be to have only one ExecutionContext class and two functions to build an instance, one remote_execution_context(engine_name, configuration) for client side and one execution_context(json_serialization) for computing side. For the computing side, ExecutionContext would be a context manager (with statement) used to manage execution specific resource (temporary directory creation for instance). This would avoid to have to select another class name and would open the possibility to run a processes locally (in the client side) if ever we need it.