search

apache / accumulo

Apache Accumulo
https://accumulo.apache.org
Apache License 2.0
1.06k stars 445 forks source link

Add more table operations to split million IT #4654

Open keith-turner opened 3 months ago

keith-turner commented 3 months ago

To ensure tablet operations do not consume too much memory it would be useful to add more tablet operations to SplitMillionIT

Could add bulk import, tablet export, and table import.

keith-turner commented 3 months ago

Started experimenting with this here https://github.com/keith-turner/accumulo/tree/accumulo-4645 . Those changes tried to bulk import around 1K files to 1 million tablets where each file covered about 1K tablets. The test timed out after 10 mins stuck in the bulk import Fate code. The manager did not seem to die w/ OOME, but not sure what it was doing. Will probably table this for now and circle back to it later. Thinking it would be good to get bulk import added to the test followed by export/import table.