aws / aws-sdk-net

The official AWS SDK for .NET. For more information on the AWS SDK for .NET, see our web site:
http://aws.amazon.com/sdkfornet/
Apache License 2.0
2.05k stars 852 forks source link

Most unit tests are not run on .NET Core. #1181

Open akdor1154 opened 5 years ago

akdor1154 commented 5 years ago

Expected Behavior

When running dotnet test "AWSDK.CoreCLR.sln", I would expect most of the unit tests in the project to be listed. However, the given list is actually very small, suggesting most unit tests are not included in this solution. (and making it difficult for developers on .NET Core-only platforms to contribute to this project.) Alternatively it's suggesting that I am running the unit tests incorrectly...

Current Behavior

The current list is:

output of `dotnet test "AWSSDK.CoreCLR.sln"`

``` Test run for /home/jarrad/src/ow/web/aws-sdk-net/sdk/test/CoreCLR/UnitTests/bin/Debug/netcoreapp1.0/UnitTests.dll(.NETCoreApp,Version=v1.0) Build completed. Test run for /home/jarrad/src/ow/web/aws-sdk-net/sdk/test/SmokeTests/bin/Debug/netcoreapp2.0/AWSSDK.SmokeTests.CoreCLR.dll(.NETCoreApp,Version=v2.0) Microsoft (R) Test Execution Command Line Tool Version 15.9.0 Copyright (c) Microsoft Corporation. All rights reserved. Microsoft (R) Test Execution Command Line Tool Version 15.9.0 Copyright (c) Microsoft Corporation. All rights reserved. The following Tests are available: The following Tests are available: AWSSDK.SmokeTests.TranscribeService.SmokeTest.ListSecretsSmokeTest AWSSDK.SmokeTests.SecretsManager.SmokeTest.ListSecretsSmokeTest AWSSDK.SmokeTests.Neptune.SmokeTest.DescribeDbInstancesTest AWSSDK.SmokeTests.Cloud9.SmokeTest.ListEnvironmentSmokeTest AWSSDK_CoreCLR.UnitTests.RegionEndpointV3Tests.CustomServicesTest AWSSDK_CoreCLR.UnitTests.RegionEndpointV3Tests.V2RegressionTest AWSSDK_CoreCLR.UnitTests.RegionEndpointV3Tests.EnumerableAllRegionsTest AWSSDK_CoreCLR.UnitTests.RegionEndpointV3Tests.NonstandardRegionTest AWSSDK_CoreCLR.UnitTests.RegionEndpointV3Tests.UnknownEndpointTest AWSSDK_CoreCLR.UnitTests.RegionEndpointV3Tests.S3SignatureTest AWSSDK_CoreCLR.UnitTests.EndpointCustomizationTestClass.EndpointCustomizationTest AWSSDK_CoreCLR.UnitTests.InternalSDKUtilsTests.RuntimeInformationTest AWSSDK_CoreCLR.UnitTests.InternalSDKUtilsTests.GetValidSubstringOrUnknownTest AWSSDK_CoreCLR.UnitTests.CloudFrontTests.SignURLValidation AWSSDK_CoreCLR.UnitTests.CloudFrontTests.SignURLCannedValidation UnitTests.ClientConfigTests.CheckForNewConfigProperties UnitTests.ClientConfigStaticMethodTests.CacheHttpClients_False_NoFactory UnitTests.ClientConfigStaticMethodTests.CacheHttpClients_True_NoFactory UnitTests.ClientConfigStaticMethodTests.CacheHttpClients_False_BasicFactory UnitTests.ClientConfigStaticMethodTests.CacheHttpClients_True_BasicFactory UnitTests.ClientConfigStaticMethodTests.CacheHttpClients_False_AdvancedFactory UnitTests.ClientConfigStaticMethodTests.CacheHttpClients_True_AdvancedFactory UnitTests.ClientConfigStaticMethodTests.DisposeHttpClients_False_NoFactory UnitTests.ClientConfigStaticMethodTests.DisposeHttpClients_True_NoFactory UnitTests.ClientConfigStaticMethodTests.DisposeHttpClients_False_BasicFactory UnitTests.ClientConfigStaticMethodTests.DisposeHttpClients_True_BasicFactory UnitTests.ClientConfigStaticMethodTests.DisposeHttpClients_False_AdvancedFactory UnitTests.ClientConfigStaticMethodTests.DisposeHttpClients_True_AdvancedFactory UnitTests.ClientConfigStaticMethodTests.CreateConfigUniqueString_NoFactory UnitTests.ClientConfigStaticMethodTests.CreateConfigUniqueString_BasicFactory UnitTests.ClientConfigStaticMethodTests.CreateConfigUniqueString_AdvancedFactory UnitTests.ClientConfigStaticMethodTests.UseGlobalHttpClientCache_True_NoFactory UnitTests.ClientConfigStaticMethodTests.UseGlobalHttpClientCache_False_NoFactory1 UnitTests.ClientConfigStaticMethodTests.UseGlobalHttpClientCache_False_NoFactory2 UnitTests.ClientConfigStaticMethodTests.UseGlobalHttpClientCache_True_Factory UnitTests.ClientConfigStaticMethodTests.UseGlobalHttpClientCache_False_Factory UnitTests.HttpRequestMessageFactoryTests.NoFactory_NoCache UnitTests.HttpRequestMessageFactoryTests.NoFactory_LocalCache UnitTests.HttpRequestMessageFactoryTests.NoFactory_GlobalCache UnitTests.HttpRequestMessageFactoryTests.WithFactory_NoCache UnitTests.HttpRequestMessageFactoryTests.WithFactory_SDKLocalCache UnitTests.HttpRequestMessageFactoryTests.WithFactory_SDKGlobalCache UnitTests.HttpRequestMessageFactoryTests.WithFactory_ExternalCache Build completed. Test run for /home/jarrad/src/ow/web/aws-sdk-net/sdk/test/CoreCLR/IntegrationTests/bin/Debug/netcoreapp1.0/IntegrationTests.dll(.NETCoreApp,Version=v1.0) Microsoft (R) Test Execution Command Line Tool Version 15.9.0 Copyright (c) Microsoft Corporation. All rights reserved. The following Tests are available: AWSSDK_DotNet.IntegrationTests.Tests.GetFederationTokenTest.TestGetFederationToken AWSSDK_DotNet.IntegrationTests.Tests.GetFederationTokenTest.TestGetFederationTokenAsync AWSSDK_DotNet.IntegrationTests.Tests.GetSessionToken.TestGetSessionToken AWSSDK_DotNet.IntegrationTests.Tests.GetSessionToken.TestGetSessionTokenRegional AWSSDK_DotNet.IntegrationTests.Tests.GetSessionToken.TestGetSessionTokenAsync AWSSDK_DotNet.IntegrationTests.Tests.RDS.BasicDescribes.TestDescribeDBEngineVersions AWSSDK_DotNet.IntegrationTests.Tests.RDS.BasicDescribes.TestDescribeDBEngineVersionsForFamily AWSSDK_DotNet.IntegrationTests.Tests.RDS.BasicDescribes.TestDescribeDBParameterGroups AWSSDK_DotNet.IntegrationTests.Tests.RDS.BasicDescribes.TestDescribeDBInstances AWSSDK_DotNet.IntegrationTests.Tests.IAM.AccessKeyTests.TestCreateAccessKey AWSSDK_DotNet.IntegrationTests.Tests.IAM.AccessKeyTests.TestCreateAccessKeyNonExistentUserException AWSSDK_DotNet.IntegrationTests.Tests.IAM.AccessKeyTests.TestListAccessKeys AWSSDK_DotNet.IntegrationTests.Tests.IAM.AccessKeyTests.TestDeleteAccessKey AWSSDK_DotNet.IntegrationTests.Tests.IAM.AccessKeyTests.TestDeleteNonExistentAccessKeyException Amazon.DNXCore.IntegrationTests.ArchiveTransferManagerTests.ListGlacierCalls Amazon.DNXCore.IntegrationTests.ArchiveTransferManagerTests.TestSimpleUpload Amazon.DNXCore.IntegrationTests.AutoScaling.DescribeAccountLimitsTest Amazon.DNXCore.IntegrationTests.AutoScaling.NoParamOperationsTest Amazon.DNXCore.IntegrationTests.AutoScaling.NoParamDescribeTerminationTests Amazon.DNXCore.IntegrationTests.AWSPublicIpAddressRangesTests.TestLoadAndParse Amazon.DNXCore.IntegrationTests.AWSPublicIpAddressRangesTests.TestQueryByServiceKey Amazon.DNXCore.IntegrationTests.AWSPublicIpAddressRangesTests.TestQueryByRegion Amazon.DNXCore.IntegrationTests.Beanstalk.ListAvailableSolutionStacksTest Amazon.DNXCore.IntegrationTests.Beanstalk.CheckDNSAvailabilityTest Amazon.DNXCore.IntegrationTests.CloudFormation.TestValidateTemplateURL Amazon.DNXCore.IntegrationTests.CloudFormation.TestGetTemplateSummary Amazon.DNXCore.IntegrationTests.CloudFormation.EstimateCostTest Amazon.DNXCore.IntegrationTests.CloudFormation.TestCreateStack Amazon.DNXCore.IntegrationTests.CloudFront.ListDistributions Amazon.DNXCore.IntegrationTests.CloudFront.OriginTests Amazon.DNXCore.IntegrationTests.CloudHSM.TestSimpleMethods Amazon.DNXCore.IntegrationTests.CloudSearch.TestCreateDomains Amazon.DNXCore.IntegrationTests.CloudWatchLogs.CRUDLogGroup Amazon.DNXCore.IntegrationTests.CodeDeploy.TestListApplications Amazon.DNXCore.IntegrationTests.CodeDeploy.TestDeleteApplication Amazon.DNXCore.IntegrationTests.CredentialsTests.TestSessionCredentials Amazon.DNXCore.IntegrationTests.DataPipeline.TestListPipelines Amazon.DNXCore.IntegrationTests.DirectoryService.TestGetDirectoryLimits Amazon.DNXCore.IntegrationTests.ECS.ListClusters Amazon.DNXCore.IntegrationTests.General.TestClientDispose Amazon.DNXCore.IntegrationTests.Kinesis.KinesisCRUD Amazon.DNXCore.IntegrationTests.ProtocolTests.TestRestJson Amazon.DNXCore.IntegrationTests.ProtocolTests.TestRestXml Amazon.DNXCore.IntegrationTests.ProtocolTests.TestQuery Amazon.DNXCore.IntegrationTests.Route53.TestRoute53 Amazon.DNXCore.IntegrationTests.Route53.HealthCheckTests Amazon.DNXCore.IntegrationTests.Route53.DelegationSetTests Amazon.DNXCore.IntegrationTests.Route53.VPCTests Amazon.DNXCore.IntegrationTests.Route53Domains.CheckDomainAvailabilityTest Amazon.DNXCore.IntegrationTests.Route53Domains.TestTagging Amazon.DNXCore.IntegrationTests.NotificationTests.SetTopicConfigurationTests Amazon.DNXCore.IntegrationTests.NotificationTests.SetQueueConfigurationTests Amazon.DNXCore.IntegrationTests.SimpleSystemsManagement.TestListDocuments Amazon.DNXCore.IntegrationTests.SNS.CRUDTopics Amazon.DNXCore.IntegrationTests.SNS.FindTopic Amazon.DNXCore.IntegrationTests.SNS.IsMessageSignatureValid Amazon.DNXCore.IntegrationTests.SNS.TestQueueSubscription Amazon.DNXCore.IntegrationTests.SQS.SQSDLQTest Amazon.DNXCore.IntegrationTests.SQS.SimpleSend Amazon.DNXCore.IntegrationTests.SQS.AuthorizeS3ToSendMessageAsyncTest Amazon.DNXCore.IntegrationTests.SQS.TestGetQueueUrl Amazon.DNXCore.IntegrationTests.SQSTests.ListQueues Amazon.DNXCore.IntegrationTests.SQSTests.SendAndReceiveMessage Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetFileUsingMetadataMode Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetFileUsingInstructionFileMode Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetStreamUsingMetadataMode Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetStreamUsingInstructionFileMode Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetContentUsingMetadataMode Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetZeroLengthContentUsingMetadataMode Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetNullContentContentUsingMetadataMode Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetContentUsingInstructionFileMode Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetFileUsingInstructionFileModeKMS Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetStreamUsingMetadataModeKMS Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetStreamUsingInstructionFileModeKMS Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetContentUsingMetadataModeKMS Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetZeroLengthContentUsingMetadataModeKMS Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetNullContentContentUsingMetadataModeKMS Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.PutGetContentUsingInstructionFileModeKMS Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.MultipartEncryptionTestMetadataMode Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.MultipartEncryptionTestInstructionFile Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.MultipartEncryptionTestMetadataModeKMS Amazon.DNXCore.IntegrationTests.S3.EncryptionTests.MultipartEncryptionTestInstructionFileKMS Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.SimpleTest Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.TestHttpErrorResponseUnmarshalling Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.SimplePutObjectTest(useChunkEncoding: True) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.SimplePutObjectTest(useChunkEncoding: False) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.SimplePathPutObjectTest(useChunkEncoding: True) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.SimplePathPutObjectTest(useChunkEncoding: False) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.GzipTest(useChunkEncoding: True) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.GzipTest(useChunkEncoding: False) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.PutObjectWithContentEncoding(useChunkEncoding: True) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.PutObjectWithContentEncoding(useChunkEncoding: False) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.PutObjectWithContentEncodingIdentity(useChunkEncoding: True) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.PutObjectWithContentEncodingIdentity(useChunkEncoding: False) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.PutObjectWithoutContentEncoding(useChunkEncoding: True) Amazon.DNXCore.IntegrationTests.S3.PutObjectTest.PutObjectWithoutContentEncoding(useChunkEncoding: False) Amazon.DNXCore.IntegrationTests.S3.S3ServiceClientTests.TestS3PutWithBodyProperty Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.SimpleUploadTest Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.SimpleUploadProgressTest Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.SimpleUpload Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.MultipartUploadProgressTest Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.UploadDirectoryWithProgressTracker Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.DownloadDirectoryProgressTest Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.DownloadProgressTest Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.OpenStreamTest Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.TestZeroLengthDownloadToNonExistingPath Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.UploadAsyncCancellationTest Amazon.DNXCore.IntegrationTests.S3.TransferUtilityTests.UploadDirectoryWithTagsTest Amazon.DNXCore.IntegrationTests.EC2.BasicDescribes.TestDescribeAmazonImages Amazon.DNXCore.IntegrationTests.EC2.BasicDescribes.TestDescribeAmazonImagesCancellationTest Amazon.DNXCore.IntegrationTests.EC2.ImageUtilitiesTest.ImageTest Amazon.DNXCore.IntegrationTests.EC2.ImageUtilitiesTest.GetDescriptors Amazon.DNXCore.IntegrationTests.EC2.TaggingTests.TestNullTags Amazon.DNXCore.IntegrationTests.DynamoDB.DynamoDBTests.TestDerivedPropertyOverrides Amazon.DNXCore.IntegrationTests.DynamoDB.DynamoDBTests.TestContext Amazon.DNXCore.IntegrationTests.DynamoDB.DynamoDBTests.TestTableOperations Amazon.DNXCore.IntegrationTests.DynamoDB.DynamoDBTests.TestJSON Amazon.DNXCore.IntegrationTests.DynamoDB.DynamoDBTests.TestTableCalls Amazon.DNXCore.IntegrationTests.DynamoDB.DynamoDBTests.TestDataCalls Amazon.DNXCore.IntegrationTests.DynamoDB.DynamoDBTests.TestDynamoDBStreamWithServiceURL Amazon.DNXCore.IntegrationTests.IntegrationTests.ApplicationAutoscaling.EndpointReachableTest Amazon.DNXCore.IntegrationTests.IntegrationTests.DynamoDBTruncateError.DynamoDBTruncateErrorTest

```

Possible Solution

??? The sln and project structure is quite complicated for a newcomer like me to understand. Maybe this complexity is what led to the tests not being included in the first place?

Your Environment

.NET Core Info

Runtime Environment: OS Name: ubuntu OS Version: 18.04 OS Platform: Linux RID: ubuntu.18.04-x64 Base Path: /usr/share/dotnet/sdk/2.2.103/

Host (useful for support): Version: 2.2.1 Commit: 878dd11e62

.NET Core SDKs installed: 1.1.11 [/usr/share/dotnet/sdk] 2.1.503 [/usr/share/dotnet/sdk] 2.2.103 [/usr/share/dotnet/sdk]

.NET Core runtimes installed: Microsoft.AspNetCore.All 2.1.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.2.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All] Microsoft.AspNetCore.App 2.1.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.2.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 1.1.10 [/usr/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 2.2.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs: https://aka.ms/dotnet-download

github-actions[bot] commented 4 years ago

We have noticed this issue has not recieved attention in a year. We will close this issue for now. If you think this is in error, please feel free to comment and reopen the issue.

akdor1154 commented 4 years ago

still an issue. Most tests still don't even run on dot net core. This is terrifying tbh.