linearregression / hypertable

Automatically exported from code.google.com/p/hypertable
GNU General Public License v2.0
0 stars 0 forks source link

RangeServerConnection in MML should get updated if changed #811

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
Ran into a situation on Rediff VSNL primary  where the Master failed to come up 
because it hit an exception trying to insert a new RangeServerClient object 
into the ServerList container in the context object.  Here's what happened

1. ht-vsnl-slave7 was inserted with IP address 10.50.60.125
2. At some point, the IP address for ht-vsnl-slave7 changed to 10.50.60.167, 
but the corresponding MML entry was not changed
3. We added two new nodes (ht-vsnl-slave9 and ht-vsnl-slave10) and 
ht-vsnl-slave9 coincidentally had the IP address 10.50.60.125.   When the 
Master tried to add ht-vsnl-slave9, it asserted because it already had an entry 
for that IP.  See Master log output below.

To fix this, whenever a RangeServer connects, we should check the IP it is 
connecting with and if it differs from the RangeServerConnection object that 
what was read from the MML, we should update the IP address in the 
RangeServerConnection object and write out a new MML entry.

----------

1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/OperationSystemUpgrade.cc:85) 
Entering SystemUpgrade-979552
1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/OperationSystemUpgrade.cc:99) 
Leaving SystemUpgrade-979552
1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/Context.cc:47) Tried to insert 
rs9 host=ht-vsnl-slave9 local=0.0.0.0:0 public=10.50.60.125:38060
1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/Context.cc:51) Contains rs1 
host=ht-vsnl-slave1 local=0.0.0.0:0 public=10.50.30.175:38060
1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/Context.cc:51) Contains rs2 
host=ht-vsnl-slave3 local=0.0.0.0:0 public=10.50.30.173:38060
1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/Context.cc:51) Contains rs3 
host=ht-vsnl-slave5 local=0.0.0.0:0 public=10.50.30.171:38060
1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/Context.cc:51) Contains rs4 
host=ht-vsnl-slave2 local=0.0.0.0:0 public=10.50.30.174:38060
1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/Context.cc:51) Contains rs5 
host=ht-vsnl-slave6 local=0.0.0.0:0 public=10.50.30.243:38060
1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/Context.cc:51) Contains rs6 
host=ht-vsnl-slave7 local=0.0.0.0:0 public=10.50.60.125:38060
1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/Context.cc:51) Contains rs7 
host=ht-vsnl-slave8 local=0.0.0.0:0 public=10.50.90.46:38060
1333054356 INFO Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/Context.cc:51) Contains rs8 
host=ht-vsnl-slave10 local=0.0.0.0:0 public=10.50.30.170:38060
1333054356 FATAL Hypertable.Master : 
(/root/src/hypertable/src/cc/Hypertable/Master/Context.cc:53) failed 
expectation: insert_result.second

Original issue reported on code.google.com by nuggetwh...@gmail.com on 29 Mar 2012 at 10:51

GoogleCodeExporter commented 9 years ago

Original comment by nuggetwh...@gmail.com on 29 Mar 2012 at 11:09

GoogleCodeExporter commented 9 years ago

Original comment by ch...@crupp.de on 30 Mar 2012 at 7:57