search

EUDAT-GEF / GEF

The GEF
GNU General Public License v2.0
6 stars 9 forks source link

Support running a chain of services #11

Open qiuwei opened 8 years ago

qiuwei commented 8 years ago

Data retrieval and storage can be then implemented as one service in the chain. This also provides great flexibility for other usecases.

The chains can use named docker volumes as the media for data exchange.

qiuwei commented 8 years ago

A chain is defined as an ordered sequence of jobs. A job is defined as a container has two volumes bound, one for input, the other for output.

qiuwei commented 8 years ago

What is a workflow? A graph consists of jobs?(do we have any restrictions on the structure of the graph? linear?)

What is a job? a Job is a container with volumes bound. Only one volume is allowed for output.(Should it be 1-to-1 map? probably yes) But should we allow multiple volumes for input?(probably yes) Are volumes allowed to bind to arbitrary paths inside the container? (probably yes) Are input volumes read-only? (yes!) Should we allow reading and writing to a single volume at the same time? (probably not)

emanueldima commented 8 years ago

I agree that it makes sense to do this, but we also must implement something fast. So supporting a chain should not be a priority for immediate development (until February'17).

qiuwei commented 8 years ago

Yes, you are right.

But workflow is essential to make data retrieval work. Anyway, a workflow is just an ordered sequence of jobs.