Closed meatballspaghetti closed 5 days ago
In progress of fixing Upgrader11to12Test which failed the unit test checks.
Upgrader11to12Test has been fixed and passes now.
Commit dd8c4fa has moved all namespace mapping actions (create/put, rename, remove) into NamespaceMapping, ensured that mutateExisting() is used in all cases, and gives checks for non-existent/already-existing namespaces (depending on the action). The actual namespace node creation and removal functionalities remain in their respective classes and occur after the mapping is updated.
@meatballspaghetti - please take a look at NamespacesIT, I'm seeing 4 tests fail with the same type of error:
java.lang.IllegalArgumentException: Multiple entries with same key: =+default and =x
at com.google.common.collect.ImmutableMap.conflictException(ImmutableMap.java:382)
at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:376)
at com.google.common.collect.ImmutableSortedMap.fromEntries(ImmutableSortedMap.java:560)
at com.google.common.collect.ImmutableSortedMap.access$100(ImmutableSortedMap.java:68)
at com.google.common.collect.ImmutableSortedMap$Builder.buildOrThrow(ImmutableSortedMap.java:739)
at com.google.common.collect.ImmutableSortedMap$Builder.build(ImmutableSortedMap.java:718)
at org.apache.accumulo.core.clientImpl.NamespaceMapping.update(NamespaceMapping.java:148)
at org.apache.accumulo.core.clientImpl.NamespaceMapping.getNameToIdMap(NamespaceMapping.java:160)
at org.apache.accumulo.core.clientImpl.Namespaces.getNamespaceId(Namespaces.java:83)
at org.apache.accumulo.core.clientImpl.Namespaces.lookupNamespaceId(Namespaces.java:97)
at org.apache.accumulo.core.clientImpl.Namespaces.namespaceNameExists(Namespaces.java:110)
at org.apache.accumulo.core.clientImpl.NamespaceOperationsImpl.exists(NamespaceOperationsImpl.java:113)
at org.apache.accumulo.test.NamespacesIT.createAndDeleteNamespace(NamespacesIT.java:220)
I'm seeing 4 tests fail with the same type of error:
I didn't see that specific stack trace when running NamespacesIT in the 3.1 branch. I got different exceptions instead. Perhaps that stack trace only appears in the main branch? I did fix the test failures I saw in 3.1 with #5067. I tested a local merge to the main branch and NamespacesIT passed there also.
This PR is part of an ongoing work to improve efficiency and reduce slowdown of retrieving and writing namespace names and table names in ZooKeeper. These particular changes modify the namespace names, and a follow-up PR in the near future will modify the table names. It replaces the use of the
/name
nodes under namespaces with a namespace id-to-name mapping in a Json format stored under the/namespaces
node. With this, when any function wants to retrieve a namespace name, ZooKeeper will only need to read the single mapping instead of iterating through all/name
nodes every time.New files added for implementation:
core/src/main/java/org/apache/accumulo/core/clientImpl/NamespaceMapping.java
: Class that holds functionality for initializing, retrieving, writing, serializing, and deserializing namespace id-to-name map (and name-to-id map).Files changed for implementation:
ClientContext.java
: Allow namespace mapping to be retrieved from ZooKeeper by referencing the context.Namespaces.java
: Update namespace name retrieval functionality with new mapping.ZooKeeperInitializer.java
: Replace initialization ofdefault
andaccumulo
/name
nodes with creation of new mapping.TableManager.java
: Update namespace creation functionality to use new mapping.RenameNamespace.java
: Replace usage of/name
node to instead use new mapping.Files changed for upgrading:
Constants.java
: DeprecateZNAMESPACE_NAME
, as the/name
node will no longer be used.Upgrader11to12.java
: Build namespace id-to-name mapping in ZooKeeper out of current/name
nodes. Follow with deletion of/name
nodes.Upgrader11to12Test.java
: Add test code respective to additions made inUpgrader11to12.java
.Original Issue: #4698