Windows Protocol Test Suites provide interoperability testing against the implementation of Windows open specifications including File Services, Identity Management, Remote Desktop and etc.
Originally developed for in-house testing of the Microsoft Open Specifications, Microsoft Protocol Test Suites have been used extensively during Plugfests and interoperability labs to test against partner implementations. A Test Suite evaluates whether a protocol or protocol family implementation meets certain interoperability requirements. Test Suites do not cover every protocol requirement and in no way certify an implementation, even if all tests pass. However, each test suite provides users with a useful indication of interoperability.
Windows Protocol Test Suites contain below components:
Windows Protocol Test Suites are based on .NET so they can be developed and run across different platforms. You should install the software listed below based on your testing purpose, including their own dependencies.
.NET and related components
a. For Windows, Linux and macOS, install .NET 6.0 SDK to build or run test suites.
b. For those who work on Windows and prefer IDE, install Visual Studio 2022 or higher (Visual Studio 2022 Community recommended), together with these individual components from the installer:
Section | Individual Component in Visual Studio 2022 | Run Windows Protocol Test Suites | Build Windows Protocol Test Suites from source code |
---|---|---|---|
.NET | .NET SDK | Required | Required |
Compilers, build tools, and runtime | C# and Visual Basic Roslyn compilers | Required | |
Compilers, build tools, and runtime | MSVC v143 - VS 2022 C++ x64/x86 build tools (Latest) | Required1 | |
Compilers, build tools, and runtime | C++/CLI support for v143 build tools (Latest) | Required1 | |
Compilers, build tools, and runtime | C++ 2022 Redistributable Update | Required1 | Required1 |
Development Activities | C++ core features | Required1 | |
SDKs, libraries, and frameworks | Windows 10 SDK (10.0.19041.0) | Required1 |
Note:
[1]: This individual component is required by ADFamily and MS-SMBD which have C++ code.
Protocol Test Framework v2.3 (build 2.3.0)
Protocol Test Framework is referenced by projects of ProtoSDK and TestSuites as NuGet packages.
From NetworkDirect_DDK.zip
extract ndspi.h
and ndstatus.h
into project path ProtoSDK\RDMA\include
. This is to build SMBD test suite.
This is required only when user want to use PowerShell Core Remoting over SSH.
This is required only when user want to use PowerShell Core Remoting over SSH for Windows platform.
This is required only when user want to use PowerShell implementation on Windows Server 2012R2 for ISutCommonControlAdapter in CommonTestSuite.ptfconfig.
a. If you choose PowerShell implementation for ISutCommonControlAdapter in a domain environment where the DC runs Windows Server 2012R2, in order to get SID from the DC, you need to install WMF 5.1 on the DC, for other Windows Server versions newer than Windows Server 2012R2, you do not need to install WMF 5.1 on the DC.
b. If you choose PowerShell implementation for ISutCommonControlAdapter on Windows platforms (including Windows Server 2012R2 and newer versions) in workgroup environment, you do not need to install WMF 5.1 on the SUT.
c. If you choose managed implementation for ISutCommonControlAdapter on Windows platforms (including Windows Server 2012R2 and newer versions), it will use LDAP queries to get SID and only supports domain environment.
If your work on Windows, you can use the script in InstallPrerequisites
folder to automatically download and install these software.
Tips when using the script in InstallPrerequisites
folder:
cd WindowsProtocolTestSuites\InstallPrerequisites
.\InstallPrerequisites.ps1
Set-ExecutionPolicy RemoteSigned
You could run the command below to verify if the Execution Policy is correctly set:
Get-ExecutionPolicy
Then rerun the script.
After you clone a copy of this repo, you can run build.ps1
in PowerShell or build.sh
in shell for each test suite separately after you have installed all the softwares required for build listed in Prerequisites.
For example, if you want to build FileServer test suite:
cd WindowsProtocolTestSuites\TestSuites\FileServer\src
build.ps1
After the build succeeds, the common folder structure should be generated in the folder WindowsProtocolTestSuite\drop\TestSuites\[TestSuiteName]\
.
Bin
: all the built binaries including ProtoSDK, adapters and test suites.Batch
: batch files (.ps1, .sh) which can be used to launch tests.Scripts
: scripts which can be used to configure test environment.Utils
: some utilities which can be used in tests.Before running a test suite, you need do either of below:
On macOS, the FileServer test suite uses AesCcm and AesGcm classes which require OpenSSL, so if there is not OpenSSL 1.1 on your macOS please install OpenSSL 1.1 and set the environment variable as following before you run the FileServer test suite on macOS,
brew install openssl@1.1
export DYLD_LIBRARY_PATH="/usr/local/opt/openssl@1.1/lib:$DYLD_LIBRARY_PATH"
Note:
brew
is not installed on your macOS, you can install it according to brew.DYLD_LIBRARY_PATH
environment variable or not. Once you install OpenSSL 1.1 on your macOS the crypto library location is /usr/local/Cellar/openssl@1.1/1.1.1m/lib/libcrypto.1.1.dylib
and /usr/local/opt/openssl
links to the /usr/local/Cellar/openssl@1.1/1.1.1m
directory by default.In the Batch
folder under root path of the test suite, there are several scripts you can use to launch tests.
Run all test cases
Execute RunAllTestCases.ps1
in PowerShell, or RunAllTestCases.sh
in shell directly.
Run test cases by filters
Execute RunTestCasesByFilter.ps1 -Filter [your filter expression]
in PowerShell, or RunTestCasesByFilter.sh [your filter expression]
in shell directly.
For example, you can run below command if you want to run test cases with test category BVT
and SMB311
:
RunTestCasesByFilter.sh "TestCategory=BVT&TestCategory=SMB311"
For more information about how to construct the filter expression, you can refer to Filter option details.
Dry run
If you want to list the test cases before running them actually, you could add -DryRun
switch to .ps1
scripts or pass a non-empty string as the last argument to .sh
scripts.
For example, you can run below command if you want to list test cases with test category BVT
and SMB311
:
RunTestCasesByFilter.sh "TestCategory=BVT&TestCategory=SMB311" "list"
Take File Server test suite as an example, you can learn how to configure and run test suite by Protocol Test Manager referring to this tutorial.
You could set up the test environment and configure the test suite according to its User Guide.
Each test suite has its own User Guide in the WindowsProtocolTestSuites\TestSuites\[TestSuiteName]\docs
folder.
There are two more kinds of documents in the same folder:
You can find contributing guide here.
Windows Protocol Test Suites are under the MIT license.
The following resources are for Windows protocol test suite news, discussion, and support:
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.