Closed JeremyKuhne closed 3 years ago
I don't know the process of taking an issue, but I volunteer to increase the test coverage.
@karelz can you assign this to @dhoehna?
@dhoehna collaborator invite sent. Please ping us when you accept and we will assign it to you. Assigning to @JeremyKuhne in the meantime ;-)
@JeremyKuhne ca you please add issue type? (I assume test enhancement)
@karelz I have accepted and enabled two-factor authentication.
@dhoehna this may help https://github.com/dotnet/corefx/blob/master/Documentation/building/code-coverage.md
@karelz Now I have accepted the invitation.
Reassigned to you @dhoehna, thanks for your contribution in the space!
@karelz Thank you. What is the priority for this issue and what does the priority mean? Also, is there a deadline when this issue has to be resolved?
Milestone is Future, which means we (.NET team) don't plan to invest into it during 2.0 timeframe. Other than that, no priority or deadline. It's nice to update the issue every week or two to communicate progress is being made - if there's no progress, others might want to pick it up.
Okay. Thank you.
Another question. I only see a folder called System.Configuration.ConfigurationManager. Is this the folder that needs better test coverage, or am I looking in the wrong spot?
@dhoehna can I join this issue? It looks like a great start point to dive into corefx contribution.
@FireAlkazar I don't mind. Once you are added, we can divvy up the work. @karelz Is it possible to add another contributor?
@dhoehna System.Configuration.ConfigurationManager is where all of the code is. We couldn't use the same exact assembly name as an existing .NET assembly, this was our best compromise. :)
@FireAlkazar added, please coordinate the work and/or tell us (me and @JeremyKuhne) if you need help or guidance.
@FireAlkazar How to you want to divvy it up? I'm thinking of alphabetically.
@dhoehna I want to try at first to cover ConnectionStrings property of Configuration. Write some simple tests and check if they are ok. And move forward after getting some feedback.
Okay. I'll start from the beginning than.
Am I doing something wrong? I can't build System.Configuration.ConfigurationManager. I get errors like "Missing compiler required member 'System.ParamArrayAttribute..ctor'" and "Predefined type System.Enum is not defined or imported" and "Predefined type System.Int32 is not defined or imported"
Here is what I have done
I've done 4 years of development on VS and I've done VS development on my current computer, so I don't know what is wrong.
Also, I am using Visual Studio 2015.
EDIT: I'm assuming the project builds successfully from master.
I don't want to add any references in fear of messing something up..
EDIT2: +@karelz
@dhoehna you need to run the build.cmd from the root once. In order to run the tests you'll need to run the build-tests.cmd once. After that you'll be able to build with the projects directly either through the command line (e.g. msbuild.exe) or in VS.
To run all of the tests from VS you need to set the test project as the startup project. We don't have things set up to use the test explorer yet.
Oh. I thought I did run build.cmd. I am running it now. I'll let you know if I have any problems.
@JeremyKuhne I am still having issues.
Build.cmd ran to completion. build-tests.cmd gave me yellow and red text at the end, saying some tests failed.
I opened visual studio and got the errors as described above but I was able to build all projects in System.Configuration.ConfigurationManager.
I do get an error when I try to run the test class though. I pasted the image from my command prompt.
It would be better if you pasted the output rather than an image. (Just block it in ```.)
What solution configuration is active?
You should try building the test project from the command line. Run the developer command prompt and in the test directory run corefx\src\System.IO.Ports\tests>msbuild System.IO.Ports.Tests.csproj /t:BuildAndTest
.
If things are messed up it is also worth trying /t:RebuildAndTest
.
If you can run tests fine from the command line that would mean VS is in the way. Possibly the configuration isn't set right.
The spew from my command line run is:
Command(s):
call P:\repos\corefx\Tools\testdotnetcli\dotnet.exe xunit.console.netcore.exe System.IO.Ports.Tests.dll -xml testRes
ults.xml -notrait Benchmark=true -notrait category=nonnetcoreapp1.1tests -notrait category=OuterLoop -notrait catego
ry=failing -notrait category=nonwindowstests
xUnit.net console test runner (64-bit .NET Core)
Copyright (C) 2014 Outercurve Foundation.
Discovering: System.IO.Ports.Tests
Discovered: System.IO.Ports.Tests
Starting: System.IO.Ports.Tests
Close.OpenFillBuffersClose [SKIP]
Condition(s) not met: \"HasNullModem\"
Close.CloseWithoutOpen [SKIP]
Condition(s) not met: \"HasOneSerialPort\"
Close.OpenClose [SKIP]
Condition(s) not met: \"HasOneSerialPort\"
Close.OpenCloseNewInstanceOpen [SKIP]
Condition(s) not met: \"HasOneSerialPort\"
Finished: System.IO.Ports.Tests
=== TEST EXECUTION SUMMARY ===
System.IO.Ports.Tests Total: 17, Errors: 0, Failed: 0, Skipped: 4, Time: 0.325s
Finished running tests. End time=15:27:35.00, Exit code = 0
It is also possible you're in a bad state in general (with all of the constant build changes we're making this can happen). Run clean.cmd
in the root, then build.cmd
and build-tests.cmd
again (I just had to to get building again in VS.).
Here is my console when running in VS:
xUnit.net console test runner (64-bit .NET Core)
Copyright (C) 2014 Outercurve Foundation.
Discovering: System.IO.Ports.Tests
Discovered: System.IO.Ports.Tests
Starting: System.IO.Ports.Tests
Close.OpenFillBuffersClose [SKIP]
Condition(s) not met: \"HasNullModem\"
Close.CloseWithoutOpen [SKIP]
Condition(s) not met: \"HasOneSerialPort\"
Close.OpenClose [SKIP]
Condition(s) not met: \"HasOneSerialPort\"
Close.OpenCloseNewInstanceOpen [SKIP]
Condition(s) not met: \"HasOneSerialPort\"
Finished: System.IO.Ports.Tests
=== TEST EXECUTION SUMMARY ===
System.IO.Ports.Tests Total: 17, Errors: 0, Failed: 0, Skipped: 4, Time: 3.742s
Press enter key to continue...
Er, sorry about using System.IO.Ports as the example. Same goes for the other project. I'm just in the middle of looking at Ports. :)
I'll do all that. In the meantime I have a question. Can I run the test just for SYstem.COnfiguration.ConfigurationManager, or do I have to run the tests for all projects?
If I can run the test for only System.Configuration.ConfigurationManger, how would I do that since build-tests.cmd is in the root?
@dhoehna you can do the following: in dev command promt navigate to Tools folder, execute msbuild.cmd /t:buildandtest ..\src\System.Configuration.ConfigurationManager\tests\System.Configuration.ConfigurationManager.Tests.csproj
I personally run only System.Configuration.ConfigurationManager.Tests doing this issue.
If your working directory is the test directory, then msbuild.cmd /t:buildandtest
suffices.
You do have to have built all the projects and tests first (and since the last pull that changed anything related to the projects, which is hard to second guess but obvious when building just the test project no longer works), but any changes that are just your own subsequently need only have the project in question built.
msbuild /t:BuildAndTest /p:Coverage=true
is very useful too, as it runs a coverage on those tests. which you can then look at at bin/tests/coverage/index.htm
Personally I like to use it a bit like I would a bad spell-checker in that it might catch things I've missed, but it's best not to trust it 100%. So if there's a big red patch it definitely shows either something that needs more testing or an area that should be proven unreachable and either removed (if dead code) or marked [ExcludeFromCodeCoverage]
if necessary e.g. to fulfil an interface, but it's best ignored while actually writing the tests to help ensure that all possibilities are considered as well as just all lines and branches.
There is pending work to have the test directory set up without actually having to run all of the tests (build-tests.cmd). You can CTRL-C out of them once they start going for now.
Or build-tests -skiptests
to build them and not run them.
@JeremyKuhne
summary
I think the error came from me opening the solution file through a regular command prompt and not the Visual Studio command prompt.
solution configuration is debug.
msbuild System.Configuration.ConfigurationManager.csproj /t:BuildAndTest
success. /t:rebuildandtest
also worked.
Opened System.Configuration.ConfigurationManager.sln from visual studio command prompt. Still the same errors.
clean.cmd
in vs command prompt is a success
build.cmd
in vs command prompt is undefined. Here is the message:
`C:\Program Files (x86)\MSBuild\14.0\bin\Microsoft.Common.CurrentVersion.targets
(1819,5): warning MSB3277: Found conflicts between different versions of the sa
me dependent assembly that could not be resolved. These reference conflicts ar
e listed in the build log when log verbosity is set to detailed. [C:\Users\Test
\Documents\GitHub\corefx\src\Tools\CoreFx.Tools\CoreFx.Tools.csproj]
Command execution succeeded.`
build-tests.cmd
in vs command prompt.
Yellow text
`C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(210,5): warning :
System.IO.Compression.Tests Total: 206, Errors: 0, Failed: 2, Skipped: 0, Time
: 29.617s [C:\Users\Test\Documents\GitHub\corefx\src\System.IO.Compression\test
s\System.IO.Compression.Tests.csproj]
C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(210,5): warning MSB30
73: The command "C:\Users\Test\Documents\GitHub\corefx\bin/Windows_NT.AnyCPU.De
bug/System.IO.Compression.Tests/netstandard//RunTests.cmd C:\Users\Test\Documen
ts\GitHub\corefx\Tools/testdotnetcli" exited with code 1. [C:\Users\Test\Docume
nts\GitHub\corefx\src\System.IO.Compression\tests\System.IO.Compression.Tests.c
sproj]
C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(210,5): warning :
System.Runtime.Extensions.Tests Total: 1426, Errors: 0, Failed: 1, Skipped: 0,
Time: 9.539s [C:\Users\Test\Documents\GitHub\corefx\src\System.Runtime.Extensi
ons\tests\System.Runtime.Extensions.Tests.csproj]
C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(210,5): warning MSB30
73: The command "C:\Users\Test\Documents\GitHub\corefx\bin/Windows_NT.AnyCPU.De
bug/System.Runtime.Extensions.Tests/netcoreapp//RunTests.cmd C:\Users\Test\Docu
ments\GitHub\corefx\Tools/testdotnetcli" exited with code 1. [C:\Users\Test\Doc
uments\GitHub\corefx\src\System.Runtime.Extensions\tests\System.Runtime.Extensi
ons.Tests.csproj]
C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(210,5): warning :
System.Net.NameResolution.Functional.Tests Total: 53, Errors: 0, Failed: 1, Sk
ipped: 0, Time: 23.905s [C:\Users\Test\Documents\GitHub\corefx\src\System.Net.N
ameResolution\tests\FunctionalTests\System.Net.NameResolution.Functional.Tests.
csproj]
C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(210,5): warning MSB30
73: The command "C:\Users\Test\Documents\GitHub\corefx\bin/Windows_NT.AnyCPU.De
bug/System.Net.NameResolution.Functional.Tests/netstandard//RunTests.cmd C:\Use
rs\Test\Documents\GitHub\corefx\Tools/testdotnetcli" exited with code 1. [C:\Us
ers\Test\Documents\GitHub\corefx\src\System.Net.NameResolution\tests\Functional
Tests\System.Net.NameResolution.Functional.Tests.csproj]
C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(210,5): warning :
System.Net.Sockets.Tests Total: 580, Errors: 0, Failed: 6, Skipped: 0, Time: 2
6.544s [C:\Users\Test\Documents\GitHub\corefx\src\System.Net.Sockets\tests\Func
tionalTests\System.Net.Sockets.Tests.csproj]
C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(210,5): warning MSB30
73: The command "C:\Users\Test\Documents\GitHub\corefx\bin/AnyOS.AnyCPU.Debug/S
ystem.Net.Sockets.Tests/netstandard//RunTests.cmd C:\Users\Test\Documents\GitHu
b\corefx\Tools/testdotnetcli" exited with code 1. [C:\Users\Test\Documents\GitH
ub\corefx\src\System.Net.Sockets\tests\FunctionalTests\System.Net.Sockets.Tests
.csproj]`
Red text
`C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(218,5): error : One o
r more tests failed while running tests from 'System.IO.Compression.Tests' plea
se check C:\Users\Test\Documents\GitHub\corefx\bin/Windows_NT.AnyCPU.Debug/Syst
em.IO.Compression.Tests/netstandard/testResults.xml for details! [C:\Users\Test
\Documents\GitHub\corefx\src\System.IO.Compression\tests\System.IO.Compression.
Tests.csproj]
C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(218,5): error : One o
r more tests failed while running tests from 'System.Runtime.Extensions.Tests'
please check C:\Users\Test\Documents\GitHub\corefx\bin/Windows_NT.AnyCPU.Debug/
System.Runtime.Extensions.Tests/netcoreapp/testResults.xml for details! [C:\Use
rs\Test\Documents\GitHub\corefx\src\System.Runtime.Extensions\tests\System.Runt
ime.Extensions.Tests.csproj]
C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(218,5): error : One o
r more tests failed while running tests from 'System.Net.NameResolution.Functio
nal.Tests' please check C:\Users\Test\Documents\GitHub\corefx\bin/Windows_NT.An
yCPU.Debug/System.Net.NameResolution.Functional.Tests/netstandard/testResults.x
ml for details! [C:\Users\Test\Documents\GitHub\corefx\src\System.Net.NameResol
ution\tests\FunctionalTests\System.Net.NameResolution.Functional.Tests.csproj]
C:\Users\Test\Documents\GitHub\corefx\Tools\tests.targets(218,5): error : One o
r more tests failed while running tests from 'System.Net.Sockets.Tests' please
check C:\Users\Test\Documents\GitHub\corefx\bin/AnyOS.AnyCPU.Debug/System.Net.S
ockets.Tests/netstandard/testResults.xml for details! [C:\Users\Test\Documents\
GitHub\corefx\src\System.Net.Sockets\tests\FunctionalTests\System.Net.Sockets.T
ests.csproj]`
Command execution failed with exit code 1.
Opened System.Configuration.ConfigurationManager.sln from VS command prompt, everything worked.
Closed VS, than tried opening with the solution file with a regular command prompt. I had the same errors as before.
I think my error was that I wasn't using the VS command prompt to open the solution file. Is the VS command prompt required? If so, where is that stated?
@dhoehna It is stated in the docs that you have to do a command line build from the VS command prompt. Once you've built, however, VS should work fine with the solutions no matter how you launch it. I usually launch the solutions from the most recent on the task bar (I pin VS).
@JeremyKuhne The developer guide says you can use a regular command prompt to do the builld. I am able to do builds with a regular command prompt.
Is there a different document that states you need a VS command prompt for the build?
sigh Still not working. Here is what I tried
Warning CS8032 An instance of analyzer Microsoft.DotNet.CodeAnalysis.Analyzers.ResourceUsageAnalyzer cannot be created from C:\Users\Test\Documents\GitHub\corefx\Tools/net46/analyzers/Microsoft.DotNet.CodeAnalysis.dll: Could not load file or assembly 'Microsoft.CodeAnalysis, Version=1.3.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies.
Here is what I am going to try now.
Sorry about the difficulties. I'll help figure out what is going on. As mentioned, the build environment is changing pretty fast- you apparently are hitting some quirk or bug in our tools/docs.
Sometimes the VS files for a solution gets messed up, wiping out .user files and the hidden .vs folder is a useful clean step.
Make sure that the test project is the startup project. If you're getting a command window that vanishes you can check what is in the debug property page for the project to see what the command is and try running it directly if it seems ok.
If you turn diagnostic logging on in the tools menu (Tools: Options: Projects and Solutions: Build and Run
) you can see the state of the project when you execute. The relevant ones to startup are:
3>StartAction = Program
3>StartArguments = xunit.console.netcore.exe System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -notrait Benchmark=true -notrait category=nonnetcoreapp1.1tests -wait -parallel none
3>StartProgram = P:\repos\corefx\Tools/testdotnetcli\dotnet.exe
3>StartWorkingDirectory = P:\repos\corefx\bin/AnyOS.AnyCPU.Debug/System.Configuration.ConfigurationManager.Tests/netstandard/
If the solution works fine from within the developer prompt but not otherwise, it would seem likely that your environment variables are messing with the build. We can look at the diff between a developer prompt set
and and regular cmd window to look for clues.
I'll try your recommendation. I am now re-installing Visual Studio on my laptop. I am also putting the corfx project on my desktop. If my desktop version has the same error than it might be I am doing something wrong.
EDIT: I successfully opened up System.Configuration.ConfigurationManager.sln on my desktop via double clicking on the .sln file. No compilation errors :). But I did get the following error when I ran the test project: Discovering: System.COnfiguration.COnfigurationManger.Tests System.BadImageFormatException: COuld not load file or assembly 'netstandard' Version=2.0.0.0, CUlter=neutral, publicKeyTOken=cc7b13ffcd2ddd51'. Referebce assemblies should not be loaded for execution. They can only be loaded in the Reflaction-only loader context. (Exception from HRESULT: 0x80131058)
This error appeared when I opened the solution via the .sln file and via an admin, VS command prompt.
EDIT 2: Visual Studio also has 12 warnings. They all say that an instance of an analyzer can't be created and the System.Collection.Immutable can't be found or one of it's dependencies.
EDIT 3: I get this warning when I run build.cmd C:\Program Files (x86)\MSBuild\14.0\bin\Microsoft.Common.CurrentVersion.targets (1820,5): warning MSB3277: Found conflicts between different versions of the sa me dependent assembly that could not be resolved. These reference conflicts ar e listed in the build log when log verbosity is set to detailed. [C:\git\corefx \src\Tools\CoreFx.Tools\CoreFx.Tools.csproj]
Also, I'm in Everett. If this gets too complicated I can come to Microsoft and we can work this out 1 on 1.
EDIT 4: ReBuilt with diagnostic messaging. StartAction, StartArguments, StartProgram, and StartWorkingDIrectory are all good and correct.
EDIT 5: Successfully re-installed Visual Studio 2015. Successfully cloned the corefx repo. Ran build.cmd and got these errors ` '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\....\VC\vc varsall.bat"' is not recognized as an internal or external command, operable program or batch file. Commencing build of native components
-- The C compiler identification is unknown -- The CXX compiler identification is unknown CMake Error at CMakeLists.txt:28 (project): No CMAKE_C_COMPILER could be found.
CMake Error at CMakeLists.txt:28 (project): No CMAKE_CXX_COMPILER could be found.
-- Configuring incomplete, errors occurred! See also "C:/git/corefx/bin/obj/Windows_NT.x64.Debug/native/CMakeFiles/CMakeOutp ut.log". See also "C:/git/corefx/bin/obj/Windows_NT.x64.Debug/native/CMakeFiles/CMakeErro r.log". Failed to generate native component build project!`
I encountered this error on my desktop as well. Currently I am uninstalling CMake and will reinstall it with admin privileges. If that does not work than I'll add CC and CXX to my PATH.
EDIT 6: Got the same error after re-installing CMake with admin privileges. Added CC and CXX to my environment variables. Still same error. I am going to remove Visual Studio 2013 and see if that works.
EDIT 7: Renamed my Microsoft Visual Studio 14.0
folder to Microsoft Visual Studio 14.0_bak
and repaired my Microsoft Visual Studio 2015 installation. Once VS installed I re-ran build.cmd and got the same errors that I did in Edit 5.
Still figuring out what is wrong. I am making progress. Hopefully my efforts will help someone else down the road.
@FireAlkazar Sorry I'm not helping. It seems like both my computers have something wrong with them and won't let me run corefx code.
@dhoehna that's ok, I hope you'll figure it out.
I also take a pause for a week approximately as I'm changing the company where I work now and it takes lots of time.
@FireAlkazar @JeremyKuhne
I got it working. I found an assumption that wasn't said in the documentation, I will double-check to make sure I am correct than post an issue.
Now I am getting run-time errors when I run the solution from Visual studio, is this supposed to happen?
EDIT 1: By errors, I mean I just get runtime errors on in different classes. After five runs, I haven't encountered the same error.
Now I am getting run-time errors when I run the solution from Visual studio, is this supposed to happen?
Not that I'm aware of. Can you give an example?
@JeremyKuhne Sure.
Run 1: File: SystemAttributeCollection.cs Line: 76 Method: ThrowIfReadOnly Error: ConfigurationErrorsException Message: An exception of type 'System.Configuration.ConfigurationErrorsException' occurred in System.Configuration.ConfigurationManager.dll but was not handled in user code
Additional information: The configuration is read only.
Run 2: File: ValidatorUtils.cs Line: 17 Method: HelperParamValidation Error: ArgumentException Message: An exception of type 'System.ArgumentException' occurred in System.Configuration.ConfigurationManager.Tests.dll but was not handled in user code
Additional information: The supplied value is not of type which the validator can process.
Run 3: File: RegexStringValidator.cs Line: 35 Method: Validate Error: ArgumentException Message: An exception of type 'System.ArgumentException' occurred in System.Configuration.ConfigurationManager.dll but was not handled in user code
Additional information: The value does not conform to the validation regex string '[a-z]+'.
Run 4: File: TimeSpanSecondsConverter.cs Line: 30 Method: ConvertFrom Error: ArgumentException Message: An exception of type 'System.ArgumentException' occurred in System.Configuration.ConfigurationManager.dll but was not handled in user code
Additional information: The expected format is an integer value in seconds
Here is how I am running the tests 1 Open an admin VS command prompt
I can't really tell without a call stack, but I assume this is either normal (e.g. you're in the middle of a test that is looking for an exception) or possibly your start command in your test project isn't quite right.
Btw, when you're including blocks of output, using code fencing would make your posts a lot easier to read. :)
@JeremyKuhne Ah, so sorry I forgot about a call stack. I'll include those. I'll also take a look at code fencing.
I'll post more later today with a call stack.
@JeremyKuhne I rant he test project again. Since the error happened in the dll there is not stack trace.
I've had this happen before in other projects. I'll try to find where the main code is so I can put a breakpoint there.
EDIT 1: I don't know if this is the same issue or not, but I am also getting this warning.
An instance of analyzer Microsoft.DotNet.CodeAnalysis.Analyzers.ResourceUsageAnalyzer cannot be created from C:\git\corefx\Tools/net46/analyzers/Microsoft.DotNet.CodeAnalysis.dll: Could not load file or assembly 'System.Collections.Immutable, Version=1.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
EDIT 2: FYI OS: Windows 7 VS version: Community.
@FireAlkazar How are you running the tests?
@dhoehna
Initially I run corefx\build-managed.cmd
and corefx\build-tests.cmd
(usually there are some errors).
In Developer command prompt for VS2015 I navigate to corefx\src\System.Configuration.ConfigurationManager\tests
folder and run msbuild /t:buildandtest System.Configuration.ConfigurationManager.Tests.csproj
.
After that I open '.sln' in VS, select test project as startup and Ctrl+F5 or F5.
Also I avoid clean solution button in VS as it breaks compilation for me.
OS: Windows 7 VS version: Community.
@FireAlkazar @dhoehna if you found steps which don't work in VS, please file a new bug (and CC me) - we need to either fix the infra or change our docs to guide devs via supported path. Thanks!
@karelz I was going to file a new bug. I want to get the solution working before I do to make sure I have covered everything from first installation to final build.
dotnet/runtime#19296 ported the code for System.Configuration to facilitate porting existing .NET code. Test coverage is relatively sparse as this effort was meant to be demand driven. Adding additional tests (unit and scenario) is a good opportunity to get one's feet wet in CoreFX.
ConfigurationPermission
is the only class that wasn't ported (Code Access Security isn't supported in CoreFX). The only known non-working class isRsaProtectedConfigurationProvider
which is dependent on System.Security.Cryptography.Xml working (#14950).