search

yahoo / storm-yarn

Storm-yarn enables Storm clusters to be deployed into machines managed by Hadoop YARN.
Other
417 stars 161 forks source link

Dynamic ports for Nimbus and UI #20

Open anfeng opened 11 years ago

anfeng commented 11 years ago

Currently, Nimbus and UI are binding to a port # given by storm configuration. On a host assigned by YARN, it's potentially the assigned port was already in use.

We should enhance storm core to allow port to be dynamically assigned for Nimbus, UI and etc.

clockfly commented 11 years ago

And supervisor worker ports.

anfeng commented 11 years ago

We have a Storm pull request for port assignment already. We may need to improve the implementation a little more.

anfeng commented 11 years ago

FYI, the port assignment is for Nimbus, Supervisor, UI and DRPC.

clockfly commented 11 years ago

Wa, that is great! This is a must-have feature!

Can you share me the link of the pull request Id?

clockfly commented 10 years ago

Hi Andy,

Have this pull request be merged to storm? If not, can you share us the link?

Sean

clockfly commented 10 years ago

There are STORM-253 to allow storm to use dynamic port for worker.

After this, I think it is more reasonble to config nimbus and ui port directly in Storm AppMaster. The reason is: 1) It will requires much smaller change on storm side, and won't conflict with other storm job(Like the comments made at https://github.com/nathanmarz/storm/pull/622/commits)

2) AppMaster and nimbus, ui are hosted on same machine, it has full knowledge to decide the port for ui and nimbus.

If you think this is OK, I will make a pull request for this.