Closed d-m4m4n closed 2 years ago
Thanks for your feature request. Supporting Azure Cosmos DB would be great.
Right now, .NET Testcontainers does not ship any 3rd party database or message broker libraries etc. This makes it easier to maintain the library, but it has a disadvantage. All vendor specific calls, features, etc. go straight to the container. We do not use any client libraries at all.
I'd like to move modules in the future (databases, message brokers, etc.) to their own library. This allows much better support incl. features for databases, message brokers, etc.
What do you think about a specific project, like DotNet.Testcontainers.CosmosDB
?
Thanks for your feature request. Supporting Azure Cosmos DB would be great.
Right now, .NET Testcontainers does not ship any 3rd party database or message broker libraries etc. This makes it easier to maintain the library, but it has a disadvantage. All vendor specific calls, features, etc. go straight to the container. We do not use any client libraries at all.
I'd like to move modules in the future (databases, message brokers, etc.) to their own library. This allows much better support incl. features for databases, message brokers, etc.
What do you think about a specific project, like
DotNet.Testcontainers.CosmosDB
?
That would be great! Really anything to take advantage of all the awesome work you have done here. Maybe DotNet.Testcontainers.CosmosDB
and DotNet.Testcontainers.Azurite
?
Yep, sounds good.
Built in integration would be a god send!
I'm having trouble trying to spin up an instance of CosmosDb - I'm not able to access the cosmos explorer, do you know if the cosmos image should be compatible with this library?
Running this command works fine:
docker run --name=Cosmos_DB -d -p 8081:8081 -p 10251:10251 -p 10252:10252 -p 10253:10253 -p 10254:10254 -e AZURE_COSMOS_EMULATOR_PARTITION_COUNT=29 -e AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1 -e AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
However the following creates the container but I can't connect to the explorer:
var cosmosDbContainerBuilder = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
.WithName("Cosmos_DB")
.WithPortBinding(8081, 8081)
.WithPortBinding(10251, 10251)
.WithPortBinding(10252, 10252)
.WithPortBinding(10253, 10253)
.WithPortBinding(10254, 10254)
.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "3")
.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(8081));
await using (var cosmosDbContainer = cosmosDbContainerBuilder.Build())
{
await cosmosDbContainer.StartAsync();
//...
}
You need to expose the port 8081
too. It's not exposed by the Dockerfile.
Then I recommend assigning random ports.
In addition to that, it's not enough to just wait for port 8081
. You'll need a custom strategy to figure out when the container is up and running:
private readonly ITestcontainersContainer container = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
.WithName("azure-cosmos-emulator")
.WithExposedPort(8081)
.WithPortBinding(8081, true)
.WithPortBinding(10251, true)
.WithPortBinding(10252, true)
.WithPortBinding(10253, true)
.WithPortBinding(10254, true)
.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
.WithWaitStrategy(Wait.ForUnixContainer()
.UntilPortIsAvailable(8081))
.Build();
[Fact]
public async Task Issue()
{
// TODO: You need to replace this with a proper wait strategy. Port 8081 is accessible before the container is ready.
await Task.Delay(TimeSpan.FromSeconds(30))
.ConfigureAwait(false);
using (var handler = new HttpClientHandler())
{
handler.ServerCertificateCustomValidationCallback = (_, _, _, _) => true;
using (var client = new HttpClient(handler))
{
var mappedPort = this.container.GetMappedPublicPort(8081);
var response = await client.GetAsync($"https://localhost:{mappedPort}/_explorer/emulator.pem")
.ConfigureAwait(false);
var pem = await response.Content.ReadAsStringAsync()
.ConfigureAwait(false);
Debug.WriteLine(pem);
}
}
}
public Task InitializeAsync()
{
return this.container.StartAsync();
}
public Task DisposeAsync()
{
return this.container.DisposeAsync().AsTask();
}
You need to expose the port
8081
too. It's not exposed by the Dockerfile. Then I recommend assigning random ports.In addition to that, it's not enough to just wait for port
8081
. You'll need a custom strategy to figure out when the container is up and running:private readonly ITestcontainersContainer container = new TestcontainersBuilder<TestcontainersContainer>() .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator") .WithName("azure-cosmos-emulator") .WithExposedPort(8081) .WithPortBinding(8081, true) .WithPortBinding(10251, true) .WithPortBinding(10252, true) .WithPortBinding(10253, true) .WithPortBinding(10254, true) .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1") .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1") .WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false") .WithWaitStrategy(Wait.ForUnixContainer() .UntilPortIsAvailable(8081)) .Build(); [Fact] public async Task Issue() { // TODO: You need to replace this with a proper wait strategy. Port 8081 is accessible before the container is ready. await Task.Delay(TimeSpan.FromSeconds(30)) .ConfigureAwait(false); using (var handler = new HttpClientHandler()) { handler.ServerCertificateCustomValidationCallback = (_, _, _, _) => true; using (var client = new HttpClient(handler)) { var mappedPort = this.container.GetMappedPublicPort(8081); var response = await client.GetAsync($"https://localhost:{mappedPort}/_explorer/emulator.pem") .ConfigureAwait(false); var pem = await response.Content.ReadAsStringAsync() .ConfigureAwait(false); Debug.WriteLine(pem); } } } public Task InitializeAsync() { return this.container.StartAsync(); } public Task DisposeAsync() { return this.container.DisposeAsync().AsTask(); }
Incredible work, will be looking at implementing this shortly!
Thanks so much! In case anyone else stumbles across this I managed to get a global instance of cosmos running for my integration tests with the following snippet:
[CollectionDefinition(nameof(IntegrationTestCollection))]
public class IntegrationTestCollection : ICollectionFixture<IntegrationTestCollectionFixture> { }
public sealed class IntegrationTestCollectionFixture : IDisposable
{
private readonly TestcontainersContainer _cosmosContainer;
public IntegrationTestCollectionFixture()
{
var outputConsumer = Consume.RedirectStdoutAndStderrToStream(new MemoryStream(), new MemoryStream());
var waitStrategy = Wait.ForUnixContainer().UntilMessageIsLogged(outputConsumer.Stdout, "Started");
_cosmosContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
.WithName("azure-cosmos-emulator")
.WithExposedPort(8081)
.WithExposedPort(10251)
.WithExposedPort(10252)
.WithExposedPort(10253)
.WithExposedPort(10254)
.WithPortBinding(8081, true)
.WithPortBinding(10251, true)
.WithPortBinding(10252, true)
.WithPortBinding(10253, true)
.WithPortBinding(10254, true)
.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "30")
.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
.WithOutputConsumer(outputConsumer)
.WithWaitStrategy(waitStrategy)
.Build();
_cosmosContainer.StartAsync().GetAwaiter().GetResult();
}
public void Dispose()
{
_ = _cosmosContainer.DisposeAsync();
}
}
Also the following is needed to setup the CosmosClient
services.AddSingleton(sp =>
{
var cosmosClientBuilder = new CosmosClientBuilder(connString);
cosmosClientBuilder.WithHttpClientFactory(() =>
{
HttpMessageHandler httpMessageHandler = new HttpClientHandler()
{
ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
};
return new HttpClient(httpMessageHandler);
});
cosmosClientBuilder.WithConnectionModeGateway();
return cosmosClientBuilder.Build();
});
You need to be aware of the async operations. Your IntegrationTestCollectionFixture
is not right. Use the IAsyncLifetime interface of xUnit.net.
public sealed class IntegrationTestCollectionFixture : IAsyncLifetime, IDisposable
{
private readonly IOutputConsumer consumer = Consume.RedirectStdoutAndStderrToStream(new MemoryStream(), new MemoryStream());
private readonly ITestcontainersContainer container;
public IntegrationTestCollectionFixture()
{
this.container = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
.WithName("azure-cosmos-emulator")
.WithExposedPort(8081)
.WithExposedPort(10251)
.WithExposedPort(10252)
.WithExposedPort(10253)
.WithExposedPort(10254)
.WithPortBinding(8081, true)
.WithPortBinding(10251, true)
.WithPortBinding(10252, true)
.WithPortBinding(10253, true)
.WithPortBinding(10254, true)
.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "30")
.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
.WithOutputConsumer(this.consumer)
.WithWaitStrategy(Wait.ForUnixContainer()
.UntilMessageIsLogged(this.consumer.Stdout, "Started"))
.Build();
}
public Task InitializeAsync()
{
return this.container.StartAsync();
}
public Task DisposeAsync()
{
return this.container.DisposeAsync().AsTask();
}
public void Dispose()
{
this.consumer.Dispose();
}
}
What do you think about a specific project, like
DotNet.Testcontainers.CosmosDB
?
@HofmeisterAn Do you mean new project in same repo or in dedicated repo? As mentioned in https://github.com/testcontainers/testcontainers-dotnet/issues/421#issuecomment-1039979866
I think it makes sense to keep modules in this repository, but don't include them in the Testcontainers dependency anymore. Each module should be an independent package.
Maybe we can avoid using the Cosmos client SDK (at least for now) by using the REST API (see: https://docs.microsoft.com/en-us/rest/api/cosmos-db/querying-cosmosdb-resources-using-the-rest-api). Would this cover the usecase?
Would love to try my hand at this for a bit if noone else is.
OC, we can use the version without client as MVP and enhance it as soon as the new structure is available.
This thread has been incredibly helpful. Thank you!
I've got my test container starting, and I can see the cosmos db emulator container start up in the docker desktop dashboard however I am struggling to get the test to connect to it.
I think it is down to the connection string.
In the example above ...
services.AddSingleton(sp =>
{
var cosmosClientBuilder = new CosmosClientBuilder(connString);
cosmosClientBuilder.WithHttpClientFactory(() =>
{
HttpMessageHandler httpMessageHandler = new HttpClientHandler()
{
ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
};
return new HttpClient(httpMessageHandler);
});
cosmosClientBuilder.WithConnectionModeGateway();
return cosmosClientBuilder.Build();
});
The string for connString
is mentioned but not constructed. Is this the default dev emulator detail or do I need to update it to point to a different port or something? If it's not this, then I am lost 😕
Any help would be appreciated.
Thanks
EDIT
I've updated the code above so that the connection string is created based on the default development one but the port is updated before being provided to the CosmosClientBuilder
.
var mappedPort = _dbContainer.GetMappedPublicPort(8081);
var updated = string.Format(connString, mappedPort);
var cosmosClientBuilder = new CosmosClientBuilder(updated);
If I debug the test and pause the test after containers have spun up etc. and I have found the port number from the above I can browse to the data explorer!
Next issue, creating db/containers on the fly for the tests ...
one but the port is updated before being provided to the
CosmosClientBuilder
.
I assume you refer to the random assigned host port? This is a best practice and avoid clashes. If you assign a random host port WithPortBinding(8081, true)
(notice the true
arg), you need to get the public mapped port first. Like you have done with GetMappedPublicPort(8081)
. See this example too: https://github.com/testcontainers/testcontainers-dotnet/discussions/438.
Next issue, creating db/containers on the fly for the tests ...
Usually, that's done via a client dependency or a cli command (ExecAsync
).
@HofmeisterAn I opened a Draft PR #549 with some progress, managed to get it up and running and create a database in the container at least. I havn't done any OS before so some early-ish feedback is much appreciated!
@Yeseh @HofmeisterAn I got CosmosDB running in a testcontainer but have some issues with performance. I reached out on twitter and was put in contact with a PM at Microsoft on the cosmos db emulator team. I put together a demo repo to show the setup. Just waiting for a reply.
Please take a look at the repo - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test
Hope this helps. I will review the Draft PR as I am interested in seeing how this can be done 😄
Found this which has some good examples of constructing the web requests to call CosmosDB directly which could aid with not using the cosmos client - https://github.com/Azure-Samples/cosmos-db-rest-samples/blob/main/Program.cs
In the examples on how to run the cosmosdb linux emulator in docker it specifies setting the memory and cpu levels ...
docker run \
--publish 8081:8081 \
--publish 10251-10254:10251-10254 \
--memory 3g --cpus=2.0 \
--name=test-linux-emulator \
--env AZURE_COSMOS_EMULATOR_PARTITION_COUNT=10 \
--env AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true \
--env AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=$ipaddr \
--interactive \
--tty \
mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
Docker compose examples seem to suggest the same.
version: '3.4'
services:
db:
container_name: cosmosdb
image: "mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator"
tty: true
restart: always
mem_limit: 2G
cpu_count: 2
environment:
- AZURE_COSMOS_EMULATOR_PARTITION_COUNT=10
- AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true
ports:
- "8081:8081"
- "8900:8900"
- "8901:8901"
- "8979:8979"
- "10250:10250"
- "10251:10251"
- "10252:10252"
- "10253:10253"
- "10254:10254"
- "10255:10255"
- "10256:10256"
- "10350:10350"
volumes:
- vol_cosmos:/data/db
volumes:
vol_cosmos:
How can these be specified in the testcontainer setup? Thanks
I havn't done any OS before so some early-ish feedback is much appreciated!
@Yeseh OC, thanks for your contribution. I'll take a look at it in the next days.
Please take a look at the repo - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test
@WestDiscGolf please notice, I'm not an CosmosDB expert 😃, but OC, I'll take a look at it.
How can these be specified in the testcontainer setup? Thanks
You can use WithCreateContainerParametersModifier
, see #503 and #508.
Cool, thanks @HofmeisterAn. Will have a go with the WithCreateContainerParametersModifier
😄
Found this which has some good examples of constructing the web requests to call CosmosDB directly which could aid with not using the cosmos client - https://github.com/Azure-Samples/cosmos-db-rest-samples/blob/main/Program.cs
Good stuff, that is very useful. Thanks! I'll go implemnent a couple of those at least .
What should the user interface of the Testcontainer be? The above example seems pretty complete, would it be desirable to have all these operations available to the user in the final version (with or without client)?
I see CosmosDB is in progress but Azurite isn't. Are there any objections if I will try work on AzuriteTestcontainer?
@HofmeisterAn Thanks for your feedback on setting up Testcontainers with CosmosDB emulator.
I'm using EF Core Cosmos provider in my application so in xUnit during the test initialization phase I am calling EnsureDeletedAsync() and EnsureCreatedAsync() methods but I am getting these errors:
Gateway HttpRequestException Endpoint not reachable. Failed Location: https://127.0.0.1:8081/; ResourceAddress: dbs/Database
Endpoint https://127.0.0.1:8081/ unavailable for Write added/updated to unavailableEndpoints with timestamp 09/19/2022 15:39:36
I wonder if I would need to specify other local IP address in override of environment variable? Really weird as I am running in Gateway mode
I wonder if I would need to specify other local IP address in override of environment variable? Really weird as I am running in Gateway mode
@kasparas12 Is 127.0.0.1
a const value? Try the Hostname
property and get the random assigned host port via GetMappedPublicPort
.
See the CosmosDb emulator PR for more details.
This should work as well:
` public sealed class CosmosDbFixture : IDisposable, IAsyncLifetime { private readonly IOutputConsumer consumer = Consume.RedirectStdoutAndStderrToStream(new MemoryStream(), new MemoryStream()); private readonly ITestcontainersContainer _container;
public CosmosDbFixture()
{
_container = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
.WithExposedPort(8081)
.WithExposedPort(10251)
.WithExposedPort(10252)
.WithExposedPort(10253)
.WithExposedPort(10254)
.WithPortBinding(8081, true)
.WithPortBinding(10251, true)
.WithPortBinding(10252, true)
.WithPortBinding(10253, true)
.WithPortBinding(10254, true)
.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
.WithOutputConsumer(consumer)
.WithWaitStrategy(Wait.ForUnixContainer()
.UntilMessageIsLogged(consumer.Stdout, "Started"))
.Build();
}
public int Port => _container.GetMappedPublicPort(8081);
public string Host => _container.Hostname;
public Task InitializeAsync()
{
return _container.StartAsync();
}
public Task DisposeAsync()
{
return _container.DisposeAsync().AsTask();
}
public void Dispose()
{
consumer.Dispose();
}
} `
Together with a DelegatingHandler which rewrites the url: request.RequestUri = new Uri($"https://{_host}:{_portNumber}{request.RequestUri.PathAndQuery}");
This is the same way I did it with creating the default db and container as well - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/blob/main/tests/Api.Tests/Infrastructure/DataContainerFixture.cs
The key for me was the delegating handler to make sure the updated uri was mapped correctly as the default port seemed hardcoded somewhere in the emulator - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/blob/8c04fd345fc272123f4ac964beaea0d0d690ca41/tests/Api.Tests/Infrastructure/FixRequestLocationHandler.cs
So I've tried the above solution to running CosmosDB
emulator but I keep getting the following error:
System.Net.Http.HttpRequestException : No connection could be made because the target machine actively refused it. (localhost:49174)
This exception is thrown when executing the following line:
var response = await _client.CreateDatabaseAsync("mydatabase", 4000);
Any ideas?
I'm using version 2.2.0-beta.3123645733
@wbuck Can you add more information about your host setup (OS, Docker version, etc.)? How do you set up your client?
@HofmeisterAn Yeah of course.
The code I'm using to setup the client is identical to what is shown here except that I've exposed more ports in an attempt to actually connect with the container.
I've also tried different addresses (instead of localhost
).
The code for the FixRequestLocationHandler
can be found here.
Like I mentioned previously I copied the code that @WestDiscGolf had shown.
I should also mention that the container is starting successfully, I just can't seem to connect to it. I have tried disabling all firewalls as well to no avail.
[TestFixture]
internal class ProjectServiceTests
{
private readonly IOutputConsumer _consumer =
Consume.RedirectStdoutAndStderrToStream(new MemoryStream(), new MemoryStream());
private const string AccountKey =
"C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";
private ITestcontainersContainer _container;
private CosmosClient _client;
[OneTimeSetUp]
public async Task InitializeAsync()
{
_container = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
.WithExposedPort(8081)
.WithExposedPort(8900)
.WithExposedPort(8901)
.WithExposedPort(8902)
.WithExposedPort(10250)
.WithExposedPort(10251)
.WithExposedPort(10252)
.WithExposedPort(10253)
.WithExposedPort(10254)
.WithExposedPort(10255)
.WithExposedPort(10256)
.WithExposedPort(10350)
.WithPortBinding(8081, true)
.WithPortBinding(8900, true)
.WithPortBinding(8901, true)
.WithPortBinding(8902, true)
.WithPortBinding(10250, true)
.WithPortBinding(10251, true)
.WithPortBinding(10252, true)
.WithPortBinding(10253, true)
.WithPortBinding(10254, true)
.WithPortBinding(10255, true)
.WithPortBinding(10256, true)
.WithPortBinding(10350, true)
.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "10")
.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
.WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
.WithOutputConsumer(_consumer)
.WithWaitStrategy(Wait.ForUnixContainer()
.UntilMessageIsLogged(_consumer.Stdout, "Started"))
.Build();
await _container.StartAsync().ConfigureAwait(false);
var mappedPort = _container.GetMappedPublicPort(8081);
var updated = $"https://localhost:{mappedPort}";
var cosmosClientBuilder = new CosmosClientBuilder(updated, AccountKey);
cosmosClientBuilder.WithHttpClientFactory(() =>
{
var handler = new HttpClientHandler
{
ServerCertificateCustomValidationCallback =
HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
};
return new HttpClient(new FixRequestLocationHandler(mappedPort, handler));
});
cosmosClientBuilder.WithConnectionModeGateway();
_client = cosmosClientBuilder.Build();
var response = await _client.CreateDatabaseAsync("mydatabase", 4000)
.ConfigureAwait(false);
var container = await response.Database.CreateContainerAsync("organization", "/organizationId", 4000)
.ConfigureAwait(false);
}
[OneTimeTearDown]
public async Task TearDownAsync()
{
await _container.StopAsync().ConfigureAwait(false);
}
[Test]
public void DoNothing()
{
Debugger.Break();
}
}
OS: Windows 10 Enterprise, version 21H2, build 19044.2006
Docker: version 20.10.17, build 100c701
The Linux containers are backed by WSL 2
.
I've also tried different addresses (instead of localhost).
Do not use localhost
. Use the Hostname
property instead. This applies to FixRequestLocationHandler
too. localhost
is not always right. Can you double-check if the Hostname
property resolves to the correct hostname / ip of the container? Does the IpAddress
property work?
The HostName
is localhost
with an IpAddress
of 172.17.0.3
which does correspond to the containers IP address.
I've tried using both the HostName
and the IpAddress
as the host, but I get the below error message (sub out the IP address for localhost
when I use HostName
instead of the IpAddress
).
System.Net.Http.HttpRequestException : A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. (172.17.0.3:49320)
Ok, let's test something. Can you add an empty .dockerenv
file to the root directory of your host / OS and test it again?
OK, so after creating the .dockerenv
in root the HostName
property is now host.docker.internal
.
If I use the HostName
I get the following exception:
System.Net.Http.HttpRequestException : No connection could be made because the target machine actively refused it. (host.docker.internal:49380)
I also want to note that I still have this line: .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
specified on the builder. I'm unsure if that should be removed.
@wbuck If you have some time tomorrow, ping me on Slack. Maybe the Hostname
resolves the wrong name or address in your case and we can figure it out together.
@HofmeisterAn Maybe release a new beta and use the new cosmosdb testcontainer. It should work directly without any extra config.
@wbuck or just check the code in develop. Use CosmosDbTestcontainer and related classes.
@ktjn Funny I was thinking the same thing. For my usual day-to-day I do a lot of system driver programming and due to that I have a bunch of VM's and virtual switches set up. I have a suspicion that's what's messing me up. I'll grab the develop
branch and give it a go.
I tried the CosmosDbTestcontainer
but unfortunately it failed for the same reason.
Tonight, after I put my kids down, I'll do some more digging into the matter and try to figure out what it is about my machine that's giving me grief.
OK, so currently I have the following network adapters:
In order to connect with the test container, I had to first disable the vEthernet (External VM Switch) Hyper-V
adaptor.
I then had to enable Internet Protocol Version 4 (TCP/IPv4)
for my actual NIC (this was disabled on the actual NIC because I was sharing the network adaptor with the VM's through the external VM switch).
I have some really distant memory that Hyper-V and Docker have had some issues, but I can't remember what that was. Anyways, the issue seems resolved. Thank-you for everyone's help.
In order to connect with the test container, I had to first disable the vEthernet (External VM Switch) Hyper-V adaptor. I then had to enable Internet Protocol Version 4 (TCP/IPv4) for my actual NIC (this was disabled on the actual NIC because I was sharing the network adaptor with the VM's through the external VM switch).
@kiview @eddumelendez @mdelapenya how do you deal in other languages with the network settings? Some devs have issues to connect to containers. Usually, I cannot reproduce it. Any idea how to make it more reliable or convenient for the dev to figure out what is going wrong e.g. see #607?
I have tried running my example - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/ - with hyper v running as well as docker desktop and the tests are successful. I have tried commenting out the following line in the DataContainerFixture
:
//.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "127.0.0.1")
The test runs continue to function.
I have the Default Switch Hyper-V NIC and not the "external switch" configured NIC. I am running Windows 10 21H1 if that helps.
Unsure if any of this information is helpful but thought I'd mention it just in case :smile:
I had to create the external switch manually via the virtual switch manager. You can use the following guide to set that up.
I think the key is to enable Allow management operating system to share this network adapter
.
Also I'm using gen 1 VM's (due to HCK, HLK limitations), but I don't believe that's an issue in this case.
If I re-enable my external VM switch I lose the ability to connect to the container.
If I re-enable my external VM switch I lose the ability to connect to the container.
To any container? Are you able to connect through a different IP or something like that?
I'll test that after I drop my kids off at school. I have a SQL Server Linux container that I can try and connect to. I'll try connecting to it via this library and manually via Azure Data Studio.
Getting an error while running the test application - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test
System.TypeInitializationException HResult=0x80131534 Message=The type initializer for 'DotNet.Testcontainers.Configurations.TestcontainersSettings' threw an exception. Source=Testcontainers StackTrace: at DotNet.Testcontainers.Configurations.TestcontainersSettings.get_OS() at DotNet.Testcontainers.Builders.TestcontainersBuilder`1..ctor() at Api.Tests.Infrastructure.DataContainerFixture..ctor() in C:\CosmosDB-Testcontainers-Test-main\tests\Api.Tests\Infrastructure\DataContainerFixture.cs:line 21
This exception was originally thrown at this call stack: [External Code]
Inner Exception 1: InvalidOperationException: Sequence contains no elements
@rrr-ganesh This looks like Docker is not running or Testcontainers for .NET cannot detect your Docker host configuration. The latest snapshot release should show an error message with more context (see Custom Configuration too).
Think I am having similar issues with the stock CosmosDbTestcontainer (version 2.2.0, running .Net 7 RC2) - container is running, but connection is refused. Happens on both Windows 11 with Docker installed, and in GitHub Actions Linux environment. Here's the stack trace for Actions:
warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
Failed to determine the https port for redirect.
warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
Failed to determine the https port for redirect.
[xUnit.net 00:00:05.35] TESTNAME [FAIL]
Failed FUNCTIONNAME at CODELOCATION [2 s]
Error Message:
System.Net.Http.HttpRequestException : Connection refused (localhost:8081)
---- System.Net.Sockets.SocketException : Connection refused
Stack Trace:
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint)
at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAndUpdateAccountPropertiesAsync(Uri endpoint)
at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAccountPropertiesAsync()
at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()
at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()
at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()
at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync(ITrace trace)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request, ITrace trace)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(String resourceUriString, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerInternal cosmosContainerCore, FeedRange feedRange, Stream streamPayload, Action`1 requestEnricher, ITrace trace, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.CosmosClient.<>c__DisplayClass50_0.<<CreateDatabaseIfNotExistsAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](ITrace trace, Func`2 task)
at Microsoft.Azure.Cosmos.ClientContextCore.<>c__DisplayClass31_0`1.<<OperationHelperWithRootTraceWithSynchronizationContextAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.Azure.CosmosRepository.Services.DefaultCosmosContainerService.GetContainerAsync(Type itemType, Boolean forceContainerSync)
at Microsoft.Azure.CosmosRepository.Providers.DefaultCosmosContainerProvider`1.<>c__DisplayClass1_0.<<-ctor>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at FUNCTIONNAME in CODELOCATION
at FUNCTIONNAME in CODELOCATION
at FUNCTIONNAME in CODELOCATION
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|281_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
[xUnit.net 00:00:41.29] TESTNAME [FAIL]
Failed TESTNAME [3 s]
Error Message:
System.Net.Http.HttpRequestException : Connection refused (localhost:49154)
---- System.Net.Sockets.SocketException : Connection refused
Stack Trace:
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint)
at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAndUpdateAccountPropertiesAsync(Uri endpoint)
at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAccountPropertiesAsync()
at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()
at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()
at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()
at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync(ITrace trace)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request, ITrace trace)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(String resourceUriString, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerInternal cosmosContainerCore, FeedRange feedRange, Stream streamPayload, Action`1 requestEnricher, ITrace trace, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.CosmosClient.<>c__DisplayClass50_0.<<CreateDatabaseIfNotExistsAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](ITrace trace, Func`2 task)
at Microsoft.Azure.Cosmos.ClientContextCore.<>c__DisplayClass31_0`1.<<OperationHelperWithRootTraceWithSynchronizationContextAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.Azure.CosmosRepository.Services.DefaultCosmosContainerService.GetContainerAsync(Type itemType, Boolean forceContainerSync)
at Microsoft.Azure.CosmosRepository.Providers.DefaultCosmosContainerProvider`1.<>c__DisplayClass1_0.<<-ctor>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.Azure.CosmosRepository.DefaultRepository`1.ExistsAsync(String id, PartitionKey partitionKey, CancellationToken cancellationToken)
at FUNCTIONNAME in /home/runner/work/CODELOCATION
at FUNCTIONNAME in /home/runner/work/CODELOCATION
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|281_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
What can I do to troubleshoot or provide more information?
There is a known problem with the cosmos emulator running in docker. Sometimes it's stuck in starting. This is due to running under a certain intel arch with ubuntu 20 or 22. Can you see in the log that it's actually started?
There is a known problem with the cosmos emulator running in docker. Sometimes it's stuck in starting. This is due to running under a certain intel arch with ubuntu 20 or 22. Can you see in the log that it's actually started?
@ktjn thanks for your reply - below is what I see in Docker Desktop for Windows.
TestContainers container log:
2022/11/03 18:18:02 Pinging Docker...
2022/11/03 18:18:02 Docker daemon is available!
2022/11/03 18:18:02 Starting on port 8080...
2022/11/03 18:18:02 Started!
2022/11/03 18:18:02 New client connected: 172.17.0.1:46316
2022/11/03 18:18:02 Received the first connection
2022/11/03 18:18:02 Adding {"label":{"testcontainers.resource-reaper-session=9002e6f2-ea85-47b1-8858-924a3eff9fa9":true}}
Cosmos Emulator log:
This is an evaluation version. There are [11] days left in the evaluation period.
Starting
Started 1/3 partitions
Started 2/3 partitions
Started 3/3 partitions
Started
And here is the code I'm using (changing the connection string to my local CosmosDb emulator succeeds):
using DotNet.Testcontainers.Builders;
using DotNet.Testcontainers.Configurations;
using DotNet.Testcontainers.Containers;
using Microsoft.Azure.CosmosRepository;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using System.Runtime.InteropServices;
namespace IntegrationTests.Features.Thing;
public class CosmosDbThing { public string Id { get; init; } = Guid.NewGuid().ToString(); }
public class CosmosDbThingServiceTests : IAsyncLifetime
{
private CosmosDbTestcontainer? _dbContainer;
private readonly List<string> _createdThingIds = new();
private CosmosDbThingService? _cosmosDbThingService;
public async Task InitializeAsync()
{
_dbContainer =
new TestcontainersBuilder<CosmosDbTestcontainer>()
.WithDatabase(new CosmosDbTestcontainerConfiguration())
.WithWaitStrategy(RuntimeInformation.IsOSPlatform(OSPlatform.Windows) switch
{
true => Wait.ForWindowsContainer(),
_ => Wait.ForUnixContainer(),
})
.Build();
await _dbContainer.StartAsync();
var cosmosThingRepository =
new ServiceCollection()
.AddSingleton<IConfiguration>(new ConfigurationBuilder().Build())
.AddOptions()
.AddCosmosRepository(
options =>
{
options.CosmosConnectionString = _dbContainer?.ConnectionString ?? "";
options.ContainerId = "thing-store";
options.DatabaseId = "main";
options.ContainerPerItemType = true;
options.ContainerBuilder.Configure<CosmosDbThing>(containerOptions => containerOptions
.WithContainer("things")
.WithPartitionKey("/id")
.WithSyncableContainerProperties()
);
})
.BuildServiceProvider()
.GetRequiredService<IRepository<CosmosDbThing>>();
_cosmosDbThingService = new CosmosDbThingService();
}
public async Task DisposeAsync()
{
foreach (var createdThingId in _createdThingIds)
{
await _cosmosDbThingService!.DeleteAsync(createdThingId);
}
await (_dbContainer?.StopAsync() ?? Task.CompletedTask);
}
[Fact]
public async Task CreateCosmosDbThingAsync_CreatesCosmosDbThing()
{
// Arrange
var thing = new CosmosDbThing();
// Act
var success = await _cosmosDbThingService!.CreateCosmosDbThingAsync(thing);
_createdThingIds.Add(thing.Id);
// Assert
success.Should().BeTrue();
}
}
You should use a IClassFixture, othwise the container will be recreated for each test. Anyways... Use something similar to: https://github.com/testcontainers/testcontainers-dotnet/blob/develop/tests/Testcontainers.Tests/Fixtures/Containers/Unix/Modules/Databases/CosmosDbFixture.cs
I think your waitstategy is the problem. Use the correct one.
See this for how to use: https://github.com/testcontainers/testcontainers-dotnet/blob/develop/tests/Testcontainers.Tests/Unit/Containers/Unix/Modules/Databases/CosmosDbTestcontainerTest.cs
Is your feature request related to a problem? Please describe. Not related to a problem
Describe the solution you'd like I would like to see setups for CosmosDB Emulator and Azurite to allow usage for testing storage or cosmosDB functions.
Describe alternatives you've considered I have one implemented locally before I found this repo. I can contribute and add these changes but need to make sure I find everything I need to make sure it works. I see there is ExecuteScripts functions and I don't think they come with CLIs in the images so I would have to check.
Additional context