Add ability to start mongo single replica set

[vhatsura]

The goal of changes

Not often, but in some cases, it's needed to bootstrap MongoDB for test purposes in replica set mode. One of the features is change streams. In addition, to bootstrap MongoDB replica set isn't an easy task and some preparation to use such instance is needed.

1. Firstly, Mongo DB should be run in replica set mode. It can be achieved with entry point customization "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0". By this reason I've added opportunity to specify entrypoint
2. The second challenge is in the process of establishing connection with the replica set. More details about such a process can be found at MongoDB documentation. In our case, Mongo DB returns host and ports which is defined inside a docker container. E.g., we define 32017:27017 port mapping. It means that Mongo DB will be available for use with the 32017 port, which works perfectly in standalone mode due to no need in the server selection mechanism. In case of replica set mode, where selection mechanism is involved, MongoDB established a connection with the client, but after the server, the selection will return 27017 port instead of 32017 due to such value is placed in replica set configuration. After that client will be disconnected by timeout. This is the exact reason why static hardcoded 27017:27017 port mappings are used.
3. The last thing which is also important is the replica set configuration. To configure Mongo DB programmatically, driver should connect to the node directly. A connection string for direct connect to node is defined in GetDirectNodeConnectionString. After direct connection, replica set must be initialized with replSetInitiate command. The logic for replica set initialization is addressed in MongoSingleReplicaSetContainerInitializer.

In addition, the auth mechanism via environment variables doesn't work. More details can be found at https://github.com/docker-library/mongo/issues/339

P.S.: should be merged after #33

[Deffiss]

@vhatsura I saw you introduced IContainerInitializer. @Hellevar what do you think?

[vhatsura]

@Deffiss, @Hellevar, thanks for your review and comments. Everything is valid and this is exact reason why the PR is in draft state. My initial intention was to introduce a concept for you with a detailed description. Unfortunately, I didn't have time to fill in the description before your review. I'll update the description in few hours and will ping you

[vhatsura]

@Deffiss, @Hellevar, I've updated the description of PR. The PR has issue with container reuse functionality. I'm going to fix it soon.

[Deffiss]

@Deffiss, @Hellevar, thanks for your review and comments. Everything is valid and this is exact reason why the PR is in draft state. My initial intention was to introduce a concept for you with a detailed description. Unfortunately, I didn't have time to fill in the description before your review. I'll update the description in few hours and will ping you

Nobody likes to provide descriptions :) the code looks self-explained and the intend was clear anyway.

[vhatsura]

Failure message from AppVeyor:

Docker.DotNet.DockerApiException : Docker API responded with status code=InternalServerError, response={"message":"driver failed programming external connectivity on endpoint test-env-my-mongo-replicaSet (f856e93f14dd07cf80baf85383563d7e4b32425eaa724c7db05b95d12cd1758f): Error starting userland proxy: listen tcp 0.0.0.0:27017: bind: address already in use"}

Looks like smth is run on 27017 port. From documentation, AppVeyor agents have preinstalled MongoDB. But it shouldn't be run by default. according to the same documentation. Link to documentation: https://www.appveyor.com/docs/services-databases/. Also, looks like default MongoDB port can be customized. I'll try to figure out is it possible or not in our use case and will return with additional details.

[Deffiss]

Failure message from AppVeyor:

Docker.DotNet.DockerApiException : Docker API responded with status code=InternalServerError, response={"message":"driver failed programming external connectivity on endpoint test-env-my-mongo-replicaSet (f856e93f14dd07cf80baf85383563d7e4b32425eaa724c7db05b95d12cd1758f): Error starting userland proxy: listen tcp 0.0.0.0:27017: bind: address already in use"}

Looks like smth is run on 27017 port. From documentation, AppVeyor agents have preinstalled MongoDB. But it shouldn't be run by default. according to the same documentation. Link to documentation: https://www.appveyor.com/docs/services-databases/. Also, looks like default MongoDB port can be customized. I'll try to figure out is it possible or not in our use case and will return with additional details.

You can try to use port binding to avoid the issue.

[vhatsura]

You can try to use port binding to avoid the issue.

As I mentioned, private docker container port should be equal to public docker container port due to the logic of how server selection works.

The second challenge is in the process of establishing connection with the replica set. More details about such a process can be found at MongoDB documentation. In our case, Mongo DB returns host and ports which is defined inside a docker container. E.g., we define 32017:27017 port mapping. It means that Mongo DB will be available for use with the 32017 port, which works perfectly in standalone mode due to no need in the server selection mechanism. In case of replica set mode, where selection mechanism is involved, MongoDB established a connection with the client, but after the server, the selection will return 27017 port instead of 32017 due to such value is placed in replica set configuration. After that client will be disconnected by timeout. This is the exact reason why static hardcoded 27017:27017 port mappings are used.

[Deffiss]

You can try to use port binding to avoid the issue.

As I mentioned, private docker container port should be equal to public docker container port due to the logic of how server selection works.

The second challenge is in the process of establishing connection with the replica set. More details about such a process can be found at MongoDB documentation. In our case, Mongo DB returns host and ports which is defined inside a docker container. E.g., we define 32017:27017 port mapping. It means that Mongo DB will be available for use with the 32017 port, which works perfectly in standalone mode due to no need in the server selection mechanism. In case of replica set mode, where selection mechanism is involved, MongoDB established a connection with the client, but after the server, the selection will return 27017 port instead of 32017 due to such value is placed in replica set configuration. After that client will be disconnected by timeout. This is the exact reason why static hardcoded 27017:27017 port mappings are used.

If it works locally, we probably could skip this test until we migrate to GitHub actions if it is not possible to solve the issue.

[vhatsura]

If it works locally, we probably could skip this test until we migrate to GitHub actions if it is not possible to solve the issue.

It would be nice to fix it due to some other devs can face it in their own CIs. BTW, I've fixed it with custom Mongo DB port. Let's see the output from AppVeyor.

[Deffiss]

In order to publish new version of Mongo lib we need to increase the version in .csproj. Could you please do that?

I'll do that.

[vhatsura]

@Deffiss, it's also required to increase version of TestEnvironment.Docker due to it was some changes there

[Deffiss]

All published

[vhatsura]

@Deffiss , despite of fact that additional parameter in constructor is backward compatible on source code level, it’s breaking change on binary level. After updating NuGet packages, Elasticsearch container cannot be instantiated. Could you update versions for other projects too, please? In addition, I suggest looking into GitVersion, which can help to publish new versions in automatic way.