apache / accumulo

Apache Accumulo
https://accumulo.apache.org
Apache License 2.0
1.08k stars 445 forks source link

Investigate if preference is given to splits over compactions #5101

Closed keith-turner closed 1 day ago

keith-turner commented 3 days ago

Is your feature request related to a problem? Please describe.

In Accumulo 2.1 and earlier when a tablet needs to split no compactions will be initiated. This is because if a tablet is really large then its best to compact after split.

Describe the solution you'd like

Determine if the Accumulo 4.0 code will start a compaction when a tablet needs to split and if so possibly avoid doing this.

cshannon commented 2 days ago

I chatted with @keith-turner a bit about this. It looks like 4.0 doesn't have this check, the spot where it would likely be done is here.

Some more investigation is still needed, something else to consider is what to do if a tablet is marked as unsplittable. We'd probably still want to compact in that case

cshannon commented 1 day ago

It looks like the unsplittable tablet case should already be handled as TabletManagementIterator won't return that the tablet needs splitting if it is unsplittable. The check is made by a call to shouldReturnDueToSplit and inside that method the check is done

cshannon commented 1 day ago

I think fixing this is as simple as changing this block of code from two if statements to an if/else if statement so that compaction only gets processed if the tablet doesn't need to split. Ie

if (actions.contains(ManagementAction.NEEDS_SPLITTING)) {
  // split
} else if (actions.contains(ManagementAction.NEEDS_COMPACTING) && compactionGenerator != null) {
  // use else if so we only compact if we don't need to split
}