Deployment of extension classes fail when Keycloak testcontainer runs on a remote docker host

[bpasson]

First of great work on creating this very useful test container! I do however hit a bump. When the Keycloak testcontainer is spawned on a remote docker host, the extension classes are never deployed. I suspect it has to do with the addFileSystemBind which is registered on the directory. This directory is not present on the host where the Keycloak testcontainer is running.

[dasniko]

Thanks for the feedback, it‘s appreciated!

Have you tested a similar approach (e.g. a generic testcontainers instance with mounted files like we do here and executed remotely) successfully? I don‘t have an environment to reproduce a remote execution. This project „just“ uses the api and methods from the upstream testcontainers project, so I think this has to be addressed there!? How do other testcontainers solve this issue?

[bpasson]

It has nothing to do with the upstream project. I did a quick test and came up with the following which uses withCopyFileToContainer from the upstream testcontainers project to recursively copy the classes to the container. I did not extensively test all cases, but it starts with the extension classes deployed on the remote docker host.

public class ExtendedKeycloakContainer extends KeycloakContainer {

    private static final Transferable WILDFLY_DEPLOYMENT_TRIGGER_FILE_CONTENT = Transferable.of("true".getBytes(StandardCharsets.UTF_8));
    private final Set<String> wildflyDeploymentTriggerFiles = new HashSet<>();

    public ExtendedKeycloakContainer(String dockerImageName) {
        super(dockerImageName);
    }

    protected void createKeycloakExtensionDeployment(String deploymentLocation, String extensionName, String extensionClassFolder) {
        Objects.requireNonNull(deploymentLocation, "deploymentLocation");
        Objects.requireNonNull(extensionName, "extensionName");
        Objects.requireNonNull(extensionClassFolder, "extensionClassFolder");

        String classesLocation = resolveExtensionClassLocation(extensionClassFolder);

        if (!new File(classesLocation).exists()) {
            return;
        }

        String explodedFolderName = extensionClassFolder.hashCode() + "-" + extensionName;
        String explodedFolderExtensionsJar = deploymentLocation + "/" + explodedFolderName;

        Path src = Paths.get(extensionClassFolder);
        try (Stream<Path> stream = Files.walk(src)) {
            stream.forEach(source -> {
                if (!Files.isDirectory(source)) {
                    withCopyFileToContainer(MountableFile.forClasspathResource(source.toString().replace(extensionClassFolder, "")),
                            explodedFolderExtensionsJar + source.toString().replace(extensionClassFolder, ""));
                }
            });
        } catch (IOException e) {
            throw new UncheckedIOException(e);
        }

        boolean wildflyDeployment = deploymentLocation.contains("/standalone/deployments");
        if (wildflyDeployment) {
            registerWildflyDeploymentTriggerFile(deploymentLocation, explodedFolderName);

            // wait for extension deployment
            setWaitStrategy(createCombinedWaitAllStrategy(Wait.forLogMessage(".* Deployed \"" + explodedFolderName + "\" .*", 1)));
        }
    }

    /**
     * Creates a {@link WaitAllStrategy} based on the current {@link #getWaitStrategy()} if present followed by the given {@link WaitStrategy}.
     *
     * @param waitStrategy
     * @return
     */
    private WaitAllStrategy createCombinedWaitAllStrategy(WaitStrategy waitStrategy) {
        WaitAllStrategy waitAll = new WaitAllStrategy();
        // startup timeout needs to be configured before calling .withStrategy(..) due to implementation in testcontainers.
        waitAll.withStartupTimeout(getStartupTimeout());
        WaitStrategy currentWaitStrategy = getWaitStrategy();
        if (currentWaitStrategy != null) {
            waitAll.withStrategy(currentWaitStrategy);
        }
        waitAll.withStrategy(waitStrategy);
        return waitAll;
    }

    /**
     * Registers a {@code extensions.jar.dodeploy} file to be created at container startup.
     *
     * @param deploymentLocation
     * @param extensionArtifact
     */
    private void registerWildflyDeploymentTriggerFile(String deploymentLocation, String extensionArtifact) {
        String triggerFileName = extensionArtifact + ".dodeploy";
        wildflyDeploymentTriggerFiles.add(deploymentLocation + "/" + triggerFileName);
    }

    @Override
    protected void containerIsStarting(InspectContainerResponse containerInfo) {
        createWildflyDeploymentTriggerFiles();
        super.containerIsStarting(containerInfo);
    }

    @Override
    protected void containerIsStopping(InspectContainerResponse containerInfo) {
        wildflyDeploymentTriggerFiles.clear();
        super.containerIsStopping(containerInfo);
    }

    /**
     * Creates a new Wildfly {@code extensions.jar.dodeploy} deployment trigger file to ensure the exploded extension
     * folder is deployed on container startup.
     */
    private void createWildflyDeploymentTriggerFiles() {
        wildflyDeploymentTriggerFiles.forEach(deploymentTriggerFile ->
                copyFileToContainer(WILDFLY_DEPLOYMENT_TRIGGER_FILE_CONTENT, deploymentTriggerFile));
    }
}

[dasniko]

Thanks for providing this information and code. I'll investigate this further in the next days/weeks.

Note: if this will make some troubles with the Wildfly environment, I'll most likely focus on the new Quarkus base for Keycloak.X. But hopefully it will be possible without problems with both approaches.

[bpasson]

Did you do some additional test work? The code I gave you was a quickly drafted extension of your KeycloakContainer implementation and is not to be assumed well-tested.

[dasniko]

There are not much other possibilities if we need to copy the resources to the container. All tests are still green. With Keycloak.X I don't copy all the single files from the directory, but create a .jar archive on the fly first and copy this file to the container (due to requirements in KC.X and the Quarkus architecture). Tests work also for KC.X.

[bpasson]

Happy to have contributed then to this very helpful testcontainer. Once the Keycloak team releases the first GA of Keycloak X I will give that a go.

[dasniko]

Thanks for contributing, I will mention you in the release notes, of course. Next release will be with KC16, hopefully soon.

[dasniko]

see https://github.com/dasniko/testcontainers-keycloak/releases/tag/1.9.0