Open dyrnq opened 1 year ago
2022-11-29 23:59:05,021 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2022-11-29 23:59:05,027 INFO util.GSet: Computing capacity for map NameNodeRetryCache
2022-11-29 23:59:05,027 INFO util.GSet: VM type = 64-bit
2022-11-29 23:59:05,027 INFO util.GSet: 0.029999999329447746% max memory 984 MB = 302.3 KB
2022-11-29 23:59:05,028 INFO util.GSet: capacity = 2^15 = 32768 entries
2022-11-29 23:59:06,408 INFO ipc.Client: Retrying connect to server: vm120/192.168.56.120:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:06,408 INFO ipc.Client: Retrying connect to server: vm121/192.168.56.121:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:06,418 INFO ipc.Client: Retrying connect to server: vm119/192.168.56.119:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:07,427 INFO ipc.Client: Retrying connect to server: vm121/192.168.56.121:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:07,427 INFO ipc.Client: Retrying connect to server: vm120/192.168.56.120:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:07,428 INFO ipc.Client: Retrying connect to server: vm119/192.168.56.119:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:08,432 INFO ipc.Client: Retrying connect to server: vm121/192.168.56.121:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:08,432 INFO ipc.Client: Retrying connect to server: vm120/192.168.56.120:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:08,434 INFO ipc.Client: Retrying connect to server: vm119/192.168.56.119:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:09,435 INFO ipc.Client: Retrying connect to server: vm121/192.168.56.121:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:09,436 INFO ipc.Client: Retrying connect to server: vm120/192.168.56.120:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:09,437 INFO ipc.Client: Retrying connect to server: vm119/192.168.56.119:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:10,438 INFO ipc.Client: Retrying connect to server: vm121/192.168.56.121:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:10,445 INFO ipc.Client: Retrying connect to server: vm120/192.168.56.120:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:10,446 INFO ipc.Client: Retrying connect to server: vm119/192.168.56.119:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:11,450 INFO ipc.Client: Retrying connect to server: vm121/192.168.56.121:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:11,454 INFO ipc.Client: Retrying connect to server: vm119/192.168.56.119:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:11,455 INFO ipc.Client: Retrying connect to server: vm120/192.168.56.120:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:12,455 INFO ipc.Client: Retrying connect to server: vm121/192.168.56.121:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:12,457 INFO ipc.Client: Retrying connect to server: vm119/192.168.56.119:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:12,458 INFO ipc.Client: Retrying connect to server: vm120/192.168.56.120:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:13,461 INFO ipc.Client: Retrying connect to server: vm121/192.168.56.121:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:13,462 INFO ipc.Client: Retrying connect to server: vm119/192.168.56.119:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:13,463 INFO ipc.Client: Retrying connect to server: vm120/192.168.56.120:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:14,466 INFO ipc.Client: Retrying connect to server: vm121/192.168.56.121:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:14,466 INFO ipc.Client: Retrying connect to server: vm120/192.168.56.120:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:14,467 INFO ipc.Client: Retrying connect to server: vm119/192.168.56.119:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:15,468 INFO ipc.Client: Retrying connect to server: vm121/192.168.56.121:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:15,473 INFO ipc.Client: Retrying connect to server: vm120/192.168.56.120:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:15,476 INFO ipc.Client: Retrying connect to server: vm119/192.168.56.119:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2022-11-29 23:59:15,493 WARN namenode.NameNode: Encountered exception during format
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Unable to check if JNs are ready for formatting. 1 exceptions thrown:
192.168.56.121:8485: Call From vm116/192.168.56.116 to vm121:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)
at org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:305)
at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:282)
at org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:1185)
at org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:212)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1274)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1726)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1834)
2022-11-29 23:59:15,561 INFO namenode.FSNamesystem: Stopping services started for active state
2022-11-29 23:59:15,561 INFO namenode.FSNamesystem: Stopping services started for standby state
2022-11-29 23:59:15,561 ERROR namenode.NameNode: Failed to start namenode.
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Unable to check if JNs are ready for formatting. 1 exceptions thrown:
192.168.56.121:8485: Call From vm116/192.168.56.116 to vm121:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)
at org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:305)
at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:282)
at org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:1185)
at org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:212)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1274)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1726)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1834)
2022-11-29 23:59:15,563 INFO util.ExitUtil: Exiting with status 1: org.apache.hadoop.hdfs.qjournal.client.QuorumException: Unable to check if JNs are ready for formatting. 1 exceptions thrown:
192.168.56.121:8485: Call From vm116/192.168.56.116 to vm121:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
2022-11-29 23:59:15,568 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at vm116/192.168.56.116
************************************************************/