Open clockfly opened 10 years ago
It is going to take me a while to go through the code. I have glanced through it and I am curious why we are calling into the storm command for submit, kill, list, and rebalance? If we want to pass through to storm itself, perhaps it would be preferable to just have a pass through command and by default if storm-yarn cannot find the command requested it just does a passthrough, looking for the appid on the command line, similar to how storm looks for the -c commands on the command line.
Hi Bobby,
I think you means something like this: storm-yarn -appId xxx -c "storm command".
I think both are good approach, yours may be less code. There are several reasons why we choose current approach instead of this.
OK maybe you didn't understand exactly what I wanted. I would like something like
storm-yarn jar ./myCode.jar -c foo=bar -appId xxx
and
storm-yarn kill -appId xxx -w 0 bar
Where storm-yarn would see jar or kill and know that they should be handled by the storm executable. It would then download the config file, and place it in ~/.storm/appid.yaml, afterwards it would run (Through ProcessBuilder not the shell)
storm jar ./myCode.jar -c foo=bar --config appid.yaml
or
storm kill -w 0 bar --config appid.yaml
I believe that this is more maintainable long term. It would also allow for -c arguments to be processed correctly.
Looking at the code for the storm command I can see that it will not work unless we also ensure that a storm.yaml exists in ~/.storm too. Perhaps the route we want to take is to fix the storm command so that it can take a full path to a config file, and it does not have to be on the classpath.
If you still think that your current code is the best I am fine with that so long as we truly mimic storm behavior and
I created a bug for storm about the config path, https://issues.apache.org/jira/browse/STORM-188
I know your point now, basically, we want to parse the configs and MainClass, and then pass everything forward to storm, so that it is more easy for us to maintain the code.
Are you planning on putting up a patch for STORM-188, shortly? If not I'll be happy to.
Yes, I already started working on it. It will be a good practice for me:)
On Thu, Jan 2, 2014 at 11:24 PM, Robert (Bobby) Evans < notifications@github.com> wrote:
shortly? If not I'll be happy to.
Patch submited for https://issues.apache.org/jira/browse/STORM-188
Patch updated.
It will requires STORM to patch STORM-188 to make this works.
Changes: Support more storm commands in storm-yarn shell.