search

cylondata / twister2

A composable framework for fast and scalable data analytics
https://twister2.org
Apache License 2.0
57 stars 32 forks source link

refactoring Config and JobConfig #728

Open pulasthi opened 4 years ago

pulasthi commented 4 years ago

Currently the users see both the Config and JobConfig objects, which can be confusing.

1) JobConfig: This can be renamed to JobParameters. It can hold job specific parameters supplied by the user. It will we part of Job object. It will be provided to each Worker with IWorker interface. 2) Config: This will have parameters that are specific to Twister2 system. Users should neither see this Config object when they submit the job, nor in IWorker.

A possible solution would be to hide Config from users. When they submit the job, they don't see Config

chathurawidanage commented 4 years ago

edu.iu.dsc.tws.rsched.job.Twister2Submitter#submitJob(edu.iu.dsc.tws.api.Twister2Job) added from 0.4.0