bsc-dom / dataclay-packaging

BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

Feature to deploy workers both with and without dataclay #94

Open eudaldsabate opened 3 years ago

dgasull commented 3 years ago

An application is using two COMPSs tasks: one method using dataClay (Task1) objects and another one just for computational reasons (Task2). To run the application, runcompss command is used with --storage-conf parameter to allow dataClay task to be properly executed. Providing storage-conf means that each worker will call init and finish to connect to dataClay. We have two nodes, A and B. dataClay is located in node A, and all COMPSs tasks using dataClay (Task1) are going to node Node B has no access to dataClay at node A and init and finish calls fail even if the tasks at node B will not use dataClay. The requested feature is to avoid initializing dataClay at some workers that are not actually using dataClay.