search

bio-guoda / guoda-services

Services provided by GUODA, currently a container for tickets and wikis.
MIT License
2 stars 0 forks source link

HDFS is unbalanced #40

Closed mjcollin closed 6 years ago

mjcollin commented 6 years ago

mesos01 has a full disk with the rest of the cluster only being about half full:

http://mesos02.acis.ufl.edu:50070/dfshealth.html#tab-datanode

mjcollin commented 6 years ago

Looks like rebalancing is manual for HDFS:

http://fibrevillage.com/storage/656-how-does-hadoop-hdfs-balance-space-usage-in-the-cluster-and-how-does-hdfs-balancer-work

Trying:

hdfs balancer -threshold 25

to start. Will likely need to make this a cron job on a name node.

jhpoelen commented 6 years ago

Looks like HDFS is happy now. Close this issue?

mjcollin commented 6 years ago

After recent fun w/ name node not syncing to secondary, I'm a bit cautious about cron'ing maintenance. Should not get out of balance too quickly in the future unless we do a lot of writes and deletes.