Open ctubbsii opened 7 months ago
For naming, could have the following property names.
compaction.services.planner.factory
compaction.services.planner.factory.config
And the following SPI with Planner
in the name and changed forName
to forService
.
interface CompactionPlannerFactory {
void init(PluginEnvironment env);
CompactionPlanner forService(String serviceName);
}
interface CompactionPlanner {
Set<GroupConfig> getGroups();
List<Job> plan(tabletPlanningInformation);
}
Thinking about property validation, something that I just added to the Upgrader in 2.1.3, having properties defined in the SPI classes and not in Property.java makes validation incomplete. It might be good for the service provider interfaces to extend a common base interface that has a method which returns a property validator object. That would let the SPI implementer use whatever properties they like, but we still have a way to validate that the configuration is correct.
Had a discussion with @ddanielr about this. Taking into account #3546 I think the default configuration should be simple. As in:
[
{
"cs1": {
"maxOpenFilesPerJob": "30",
"groups": [
{
"name": "default",
}
]
}
}
]
I removed the bug
label as this issue does not reflect a defect in the current implementation. I also moved this from from correctness
iteration in the Elasticity
project to the scaling
iteration. These changes are not required for the correct operation.
Had a discussion with @keith-turner and @ddanielr about some of the compaction configuration and SPI design, and we came up with some of these ideas that could be implemented:
One major unknown is what to call it. We went back and forth a bit on whether it should be
CompactionService*
orCompactionPlanner*
compaction.service.factory
holds the class name (singleton ref in ServerContext?) b.compaction.service.factory.config
holds a single string of all the factory's config// configuration for a bounded named group / queue class GroupConfig { String name; int maxQueueSize; }
interface CompactionService { Set getGroups();
List plan(tabletPlanningInformation);
}
One of the main benefits of this is having simplified configuration for user's configuration file. Also, we'll be able to set a default value in the DefaultConfiguration that actually lives in one place, and is easy for users to view for reference that represents the actual default behavior and override if they want to.
Re: #3981, #4034 , #4061