matthewvon
commented
8 years ago code review issues addressed. "nice" validated manual via top:
top - 16:22:37 up 210 days, 2:22, 1 user, load average: 0.52, 6.57, 9.13 Tasks: 172 total, 0 running, 172 sleeping, 0 stopped, 0 zombie Cpu(s): 0.3%us, 0.1%sy, 0.0%ni, 99.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 32916248k total, 28652888k used, 4263360k free, 62436k buffers Swap: 33521660k total, 17184k used, 33504476k free, 23954508k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
26894 matthewv 22 2 11.8g 3.9g 957m S 0 12.5 0:00.00 beam.smp
26895 matthewv 22 2 11.8g 3.9g 957m S 0 12.5 0:01.16 GeneralCompact
26896 matthewv 22 2 11.8g 3.9g 957m S 0 12.5 0:00.00 GeneralCompact
26897 matthewv 22 2 11.8g 3.9g 957m S 0 12.5 0:00.00 GeneralCompact
26535 matthewv 20 0 11.8g 3.9g 957m S 0 12.5 0:00.04 beam.smp
26716 matthewv 20 0 11.8g 3.9g 957m S 0 12.5 0:00.00 beam.smp
26717 matthewv 20 0 11.8g 3.9g 957m S 0 12.5 0:00.00 beam.smp
26718 matthewv 20 0 11.8g 3.9g 957m S 0 12.5 0:00.04 beam.smp
26719 matthewv 20 0 11.8g 3.9g 957m S 0 12.5 0:00.04 beam.smp
Note PR column is 22 for 3 GeneralCompact threads. Looks like semaphore based QueueThread's name is not registering ... but that is a bug for another day.
I believe this is ready.
paulplace
This is a non-portable use of Linux "nice" to change thread scheduling of general compaction operations. The change is known to increase throughput and reduce latencies in most test loads. There is no test load where it makes throughput or latencies worse.
It also greatly reduces the chances of an unsolved iterator hang condition occurring, i.e. issue goes away in test scenarios. This is likely due to Erlang threads getting priority during heavy compaction and/or data load scenarios.
The changes in version_set.cc and version_set.h were previously committed in a timed grooming PR.