How to connect to mongodb using FluentDocker?

[jolienai]

I am trying to use FluentDocker to run the tests against MongoDB, but I cannot connect to it, see the code below.

            [Fact]
            public async Task TestMongoDbConnection3()
            {
                const string root = "root";
                const string secret = "secret";

                using (
                    var container =
                        new Builder().UseContainer()
                            .UseImage("mongo")
                            .WithEnvironment($"MONGO_INITDB_ROOT_USERNAME:{root}")
                            .WithEnvironment($"MONGO_INITDB_ROOT_PASSWORD:{secret}")
                            .ExposePort(27018)
                            .WaitForPort("27018/tcp", 30000 /*30s*/)
                        .Build()
                        .Start())
            {
                var config = container.GetConfiguration(true);
                Assert.Equal(ServiceRunningState.Running, config.State.ToServiceState());

                const string connectionString = "mongodb://root:secret@localhost:27018";

                // insert data 
                const string collectionName = "Users";

                var mongoClient = new MongoClient(connectionString);
                var database = mongoClient.GetDatabase ("Users");
                var collection = database.GetCollection<User>(collectionName);

                const int id = 1;
                var data = new Fixture().Build<User>()
                    .With(x => x.Id, id)
                    .Create();

                // delete if exists, but got timeout after 30000 ms
                await collection.DeleteOneAsync(x => x.AccountId == data.AccountId);

                ...
            }
        }

I tried to replace the localhost for the IP from config (config.NetworkSettings.IPAddress), but still getting the timeout.

Any idea?

Many thanks,

[mariotoffia]

Hi @jolienai. Hope you have solved it by now. I had a look and did come to the following conclusions.

1) It seems that you use the incorrect port. As per docker hub, mongo image, documentation (and if you try to start it manually) the exposed port is 27017, not 27018.

2) In addition when you choose to use

.ExposePort(27018)

It will expose the port to a random host port. In order to retrieve the endpoint (IP and port), you may use the following extension function

container.ToHostExposedEndpoint("27017/tcp");

To render a proper connection string you to use that extension to get the randomly mapped port. For example

        var ep = container.ToHostExposedEndpoint("27017/tcp");
        string connectionString = $"mongodb://{root}:{secret}@{ep}";

You can see samples in the test directory for the fluent API how to use this with an Nginx server.

Hope that this helps!

Cheers, Mario :)

[jolienai]

Hi Mario, thank you a lot for replying.

I am still having issues connecting to the database, I made the changes you said, but now I am getting "Unable to authenticated using sasl protocol mechanism SCRAM-SHA-1"

I had this issue before when I was not using FluentDocker, and I solve it by add "/DatabaseName?authSource=admin&readPreference=primary&ssl=false" but it still not working.

Regarding the 27018 port, I am trying to use, even the default port is 27017 you can set another port in the docker compose file as below.

# Use root/example as user/password credentials
version: '3.1'

services:

  mongo:
    image: mongo
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: secret
     ports:
       - 27018:27017

[mariotoffia]

Hi @jolienai hmm... I'm more of an DynamoDB guy - done some mongo back in the days when it was 1.x...

Maybe test to connect using this?

Cheers, Mario

[jolienai]

HI @mariotoffia the env variables set was wrong, so I manage to fix it, now the connection is working locally. However, it is failing when the test is running inside a docker-in-docker image in the CI pipeline.

Here is the test I am trying to do

[Fact]
public void Test_Run_PingCommand_returns_True()
{
    const string root = "root";
    const string secret = "secret";

    using (
        var container =
            new Builder().UseContainer()
                .UseImage("mongo")
                .WithEnvironment(new []{$"MONGO_INITDB_ROOT_USERNAME={root}",
                    $"MONGO_INITDB_ROOT_PASSWORD={secret}"})
                .ExposePort(27017)
                .WaitForPort("27017/tcp", 30000 /*30s*/)
                .Build()
                .Start())
    {
        var config = container.GetConfiguration(true);
        Assert.Equal(ServiceRunningState.Running, config.State.ToServiceState());

        var ep = container.ToHostExposedEndpoint("27017/tcp");

        var connectionString =
            $"mongodb://{root}:{secret}@{ep}";

        var setting = new UsageDbSettings();
        var mongoClient = new MongoClient(connectionString);
        var database = mongoClient.GetDatabase(setting.DatabaseName);

        var isMongoLive = database.RunCommandAsync((Command<BsonDocument>)"{ping:1}").Wait(30000);

        Assert.True(isMongoLive);
    }
}

Now I am getting timeout

[xUnit.net 00:00:53.76] Cost.Infrastructure.Tests.DataSources.Mongodb.MongoDbConnectionTests.Test_Run_PingCommand_returns_True [FAIL]
61 X Cost.Infrastructure.Tests.DataSources.Mongodb.MongoDbConnectionTests.Test_Run_PingCommand_returns_True [52s 539ms]
62 Error Message:
63 Ductus.FluentDocker.Common.FluentDockerException : Timeout waiting for service at = 0.0.0.0 port = 38926
64---- System.Net.Internals.SocketExceptionFactory+ExtendedSocketException : Connection refused 0.0.0.0:38926
65 Stack Trace:
66 at Ductus.FluentDocker.Services.Extensions.NetworkExtensions.WaitForPort(IPEndPoint endpoint, Int64 millisTimeout)
67 at Ductus.FluentDocker.Services.Extensions.NetworkExtensions.WaitForPort(IContainerService service, String portAndProto, Int64 millisTimeout, String address)
68 at Ductus.FluentDocker.Builders.ContainerBuilder.<>c__DisplayClass73_1.<AddHooks>b__12(IService svc)
69 at Ductus.FluentDocker.Fd.DisposeOnException[T](Action`1 action, T service, String name)
70 at Ductus.FluentDocker.Builders.ContainerBuilder.<AddHooks>b__73_1(IService service)
71 at Ductus.FluentDocker.Services.Impl.ServiceHooks.Execute(IService service, ServiceRunningState state)
72 at Ductus.FluentDocker.Services.Impl.DockerContainerService.set_State(ServiceRunningState value)
73 at Ductus.FluentDocker.Services.Impl.DockerContainerService.Ductus.FluentDocker.Services.IService.Start()
74 at Ductus.FluentDocker.Services.Impl.DockerContainerService.Start()
75 at Cost.Infrastructure.Tests.DataSources.Mongodb.MongoDbConnectionTests.Test_Run_PingCommand_returns_True() in /builds/billing/cost/Cost.Infrastructure.Tests/DataSources/Mongodb/MongoDbConnectionTests.cs:line 112

76----- Inner Stack Trace -----
77 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
78 at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
79 at Ductus.FluentDocker.Services.Extensions.NetworkExtensions.WaitForPort(IPEndPoint endpoint, Int64 millisTimeout)
80Test Run Failed.
81Total tests: 1
82 Failed: 1
83 Total time: 54.7567 Seconds

[mariotoffia]

Nice @jolienai !! The problem is that you're trying to connect inside the docker container to a docker container hosted in the host. Hence you need to make sure that that traffic is forwarded to the MongoDB docker container. Now it expects the container to exist on the test(?) containers port 38926. If you need FluentDocker to resolve the network differently you can override that

My best advice is to use stack overflow for this. Maybe you can try to create a network that both resides in?

Cheers, Mario :)

Sample custom endpoint resolver:

using (
  var container =
    Fd.UseContainer()
      .UseImage("postgres:9.6-alpine")
      .WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
      .ExposePort(5432)
      .UseCustomResolver((
        ports, portAndProto, dockerUri) =>
      {
        if (null == ports || string.IsNullOrEmpty(portAndProto))
          return null;

        if (!ports.TryGetValue(portAndProto, out var endpoints))
          return null;

        if (null == endpoints || endpoints.Length == 0)
          return null;

        if (CommandExtensions.IsNative())
          return endpoints[0];

        if (CommandExtensions.IsEmulatedNative())
          return CommandExtensions.IsDockerDnsAvailable()
            ? new IPEndPoint(CommandExtensions.EmulatedNativeAddress(), endpoints[0].Port)
            : new IPEndPoint(IPAddress.Loopback, endpoints[0].Port);

        if (Equals(endpoints[0].Address, IPAddress.Any) && null != dockerUri)
          return new IPEndPoint(IPAddress.Parse(dockerUri.Host), endpoints[0].Port);

        return endpoints[0];
      })
      .WaitForPort("5432/tcp", 30000 /*30s*/)
      .Build()
      .Start())
{
  var state = container.GetConfiguration(true/*force*/).State.ToServiceState();
  Assert.AreEqual(ServiceRunningState.Running, state);
}

[mariotoffia]

Hi @jolienai, how did it go, and what CI service do you use? It would be nice, if you'd solved it, to have it here as a reference.

Cheers, Mario :)

[jolienai]

Hi @mariotoffia, I am still having the timeout issue. The idea is to use FluentDocker to use to test the MongoDB repositories so that I can insert some data, call the repo and validate the data it is returned.

I am using GitLab, so in the build step I am using the dind image from the docker file similar to this:

FROM mcr.microsoft.com/dotnet/sdk:3.1
COPY nuget.config /root/.nuget/NuGet/NuGet.Config

ENV nuget_api_key XXXXXXXXXXXXXXXXXXXXXX

# Install Docker from Docker Inc. repositories.
RUN curl -sSL https://get.docker.com/ | sh && \
    apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y docker-ce && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

 # Install docker-compose
RUN curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
RUN chmod +x /usr/local/bin/docker-compose
RUN /usr/local/bin/docker-compose --version

# Install the magic wrapper.
ADD ./wrapdocker /usr/local/bin/wrapdocker
RUN chmod +x /usr/local/bin/wrapdocker

# Define additional metadata for our image.
VOLUME /var/lib/docker
EXPOSE 2375
ENV LOG=file
ENTRYPOINT ["wrapdocker"]
CMD []