apache / accumulo

Apache Accumulo
https://accumulo.apache.org
Apache License 2.0
1.07k stars 445 forks source link

Store columns Fate needs to find work in a locality group #4977

Open keith-turner opened 4 days ago

keith-turner commented 4 days ago

Is your feature request related to a problem? Please describe.

Fate continually scans the accumulo.fate table looking for new work. This scan only reads a small amount of data per fate transaction, but needs to filter out all of the other data. If the data needed to find work was stored in its own locality group then this would avoid filtering out data. Using a locality group would also help with cache utilization in the tablet server as only the small amount of data is read when fate does a full table scan looking for work.

Describe the solution you'd like

These two columns are read when fate looks for work. Need to refactor these columns to be in their own column family and then configure the accumulo initialization code to setup the locality group config when creating the table.

ArbaazKhan1 commented 9 hours ago

I can take a look at this