EvotecIT / Testimo

Testimo is a PowerShell module for running health checks for Active Directory against a bunch of different tests
549 stars 58 forks source link

Testimo not properly importing dependency modules, commands not found #155

Closed tastyratz closed 2 years ago

tastyratz commented 2 years ago

This is something that broke for me a long time ago. I've tried using this at multiple environments, I want to say this was around the time RSAT became required. I dug in today and only now realized the problem.

I can not get testimo to work across any domain/site/customer I try it on.

If I invoke-testimo with known dependencies, many of the items will fail and give me an error such as "The term 'Get-WinADLastBackup' is not recognized as the name of a cmdlet..."

So, I went looking. If I try running those commands on their own, I get an error. If I were to import-module adessentials then I can run those commands manually. It's installed, I have it, they work independently.

After importing, however, I still get errors within Testimo. It appears that testimo is not loading the modules to allow their commands to run successfully, so, the whole thing fails.

I suspect that issue 134 https://github.com/EvotecIT/Testimo/issues/134 is actually similarly related. I am not running in portable mode but do experience similar results anywhere I try this.

PrzemyslawKlys commented 2 years ago

Are you using Install-Module Testimo?

Testimo requires RSAT AD, RSAT GPO, RSAT DNS. Without it, it won't work.

This means:

Get-ADForest
Get-GPO -All
Get-DnsServer

All 3 have to be there and working.

tastyratz commented 2 years ago

Correct on all of the above, same results I'm running this directly from domain controllers, 2012R2 & 2016 are the same.

(p.s. I might update the readme documentation for Testimo to include the add-windowscapability commands for all 3 if all 3 is required. There are 2 mentioned on Gpozaurr, none on the Testimo readme)

PrzemyslawKlys commented 2 years ago

Ok, what are the errors? What happens if you do Import-Module Tesitmo -Force -Verbose

tastyratz commented 2 years ago

If I run all 3 commands above, they work.

It's behaving as if the ad module is not importing. If I run invoke-Testimo, most of the adessentials functions tell me that the command isn't found and say "The term 'Get-WinADLastBackup' is not recognized as the name of a cmdlet..." And it will say that for others like get-winaddhcp, etc.

If I import-module adessentials and then attempt those same commands manually, they run in a normal powershell window. If testimo attempts to run the commands it's as if the module isn't getting imported and they don't exist.

PrzemyslawKlys commented 2 years ago

What does that $PSModuleAutoloadingPreference show?

https://devblogs.microsoft.com/scripting/powertip-turn-off-powershell-module-autoload/

Seems to me like auto-loading AD module is broken.

tastyratz commented 2 years ago

The value for that preference is blank on all machines I checked it against. From what I can find online, that's correct.

Note also that if I run Invoke-Gpozaurr it works for me. Gpozaurr relies on ADEssentials. If this was a problem with loading modules, Gpozaurr would behave similarly, would it not?

PrzemyslawKlys commented 2 years ago

GPOZaurr doesn't rely on ADEssentials, at least when you install proper version from PowerShellGallery. My auto-merging process takes care of that dependency.

image

ADEssentials is only required for Testimo. ADEssentials has in psm1 logic to load ActiveDirectory and DHCPServer modules.

image

My guess is - for whatever reason ActiveDirectory module somehow doesn't get imported and so things fail to auto-load.

I don't have this problem on multiple servers, on multiple clients so it's something on your end :-)

tastyratz commented 2 years ago

Is it possible to get an old copy? This was last working for me at the end of January, 2020. Would I be able to pull down the old code somewhere so I could test that against current revisions? I've run this in a bunch of environments this last week without luck but I know it used to work great.

PrzemyslawKlys commented 2 years ago

install-module testimo -version xx.

https://www.powershellgallery.com/packages/Testimo/0.0.73

All published versions

tastyratz commented 2 years ago

I did -requiredversion 0.0.55 but it's still pulling down updated versions of all the dependencies even though I've cleared them out. I'd love to test it as it was with whatever versions existed at that time, but, I didn't know if I would have to hand install each dependency legacy version and figure out what was released around that same time period.

I'm trying to run some of the individual modules to test and running into a lot of weird things. testing the pester piece: test-adpester is telling me it "is not a ps1 file" and erroring out. It looks like that one was formatted differently and PesterInfrastructureTests was last updated in 2019. The manifest is completely different.

If I invoke-testimo I'm getting error line 5212 char 5 that it cannot convert null to type "system.ConsoleColor" due to enumeration values that are not valid.

So, I uninstalled testimo completely using the command on the testimo git main readme:

$Modules = @('Testimo', 'PSWinDocumentation.AD','PSWinDocumentation.DNS','ADEssentials', 'PSSharedGoods','PSWriteColor', 'Connectimo', 'DSInternals','Emailimo','PSWriteHTML' ) foreach ($Module in $Modules) { Uninstall-Module $Module -Force -AllVersions }

I did change Uninstall-Module $Module -Force -AllVersions to include -erroraction continue which makes life easier and also lists out discrepencies (might be worth adding to the readme?)

When I ran that, I was told no package was found for the following packages: PSWinDocumentation.AD DSInternals Emailimo

When I checked the testimo manifest, I'm not seeing any of those 3 called out in the psd1 file. Did I also need those installed and if so, should they be part of the testimo manifest?

I did just try installing them manually, then installing testimo, and then running invoke-testimo- but, ran into same errors mentioned. I tried manipulating the PSModuleAutoLoadingPreference to be All or None, but, that did not make a difference.

To the point in a previous post, however, GPOZaurr still required PSWriteHTML yet it functions in all my environments. Even if it doesn't require ADessentials, it's loading and using PSWriteHTML successfully. That indicates that I am, in fact, autoloading required dependencies - Just not all for Testimo. After Testimo finishes if I don't get the color error out I do get an html output (that's just missing most tests/test data).

PrzemyslawKlys commented 2 years ago

In my opinion your problem comes from ADEssentials

Install-Module ADEssentials -Force -Verbose
Import-module ADEssentials -Force
Get-Command -Module ADEssentials

Do you get like 50 commands or more like 7? If more like 7 it means the Active Directory module is not getting detected properly and you need to reinstall it.

Push here some screenshots because I have a feeling we're running in circles.

PrzemyslawKlys commented 2 years ago

Also Test-ADPester doesn't belong to testimo, it was just there in the article to show you the old version of things.

That's the current list of modules requirements

But you all get that using Install-Module Testimo -Force -Verbose with the exception of ActiveDirectory, ServerManager, GroupPolicy, and DHCP server. Those 4 come from RSAT and need to be installed manually.

Please let me know:

$PSVersionTable
Get-module ActiveDirectory,GroupPolicy,DHCPServer,ServerManager -ListAvailable
tastyratz commented 2 years ago

I managed to get it working on a few environments but others, not so much and I'm not sure why. I ripped it down, reinstalled Pester, and it seemed to start working on 2 different domains! The next domain? Trouble! I ran the commands and pasted below for reference:


PS C:\Users\Username> get-command -module adessentials

CommandType     Name                                               Version    Source                                                                                     
-----------     ----                                               -------    ------                                                                                     
Alias           Get-WinADDomainRoles                               0.0.138    ADEssentials                                                                               
Alias           Get-WinADForestObjectsConflict                     0.0.138    ADEssentials                                                                               
Alias           Get-WinADForestTomebstoneLifetime                  0.0.138    ADEssentials                                                                               
Alias           Get-WinADPriviligedObjects                         0.0.138    ADEssentials                                                                               
Alias           Get-WinADRoles                                     0.0.138    ADEssentials                                                                               
Alias           Get-WinADSubnet                                    0.0.138    ADEssentials                                                                               
Alias           Get-WinADTrusts                                    0.0.138    ADEssentials                                                                               
Alias           Get-WinADUsersFP                                   0.0.138    ADEssentials                                                                               
Alias           Get-WinDNSServerIP                                 0.0.138    ADEssentials                                                                               
Alias           Set-WinDNSServerIP                                 0.0.138    ADEssentials                                                                               
Alias           Show-ADGroupMember                                 0.0.138    ADEssentials                                                                               
Alias           Show-ADGroupMemberOf                               0.0.138    ADEssentials                                                                               
Alias           Show-ADTrust                                       0.0.138    ADEssentials                                                                               
Alias           Show-ADTrusts                                      0.0.138    ADEssentials                                                                               
Alias           Show-WinADTrusts                                   0.0.138    ADEssentials                                                                               
Function        Add-ADACL                                          0.0.138    ADEssentials                                                                               
Function        Copy-ADOUSecurity                                  0.0.138    ADEssentials                                                                               
Function        Get-ADACL                                          0.0.138    ADEssentials                                                                               
Function        Get-ADACLOwner                                     0.0.138    ADEssentials                                                                               
Function        Get-DNSServerIP                                    0.0.138    ADEssentials                                                                               
Function        Get-WinADACLConfiguration                          0.0.138    ADEssentials                                                                               
Function        Get-WinADACLForest                                 0.0.138    ADEssentials                                                                               
Function        Get-WinADBitlockerLapsSummary                      0.0.138    ADEssentials                                                                               
Function        Get-WinADComputerACLLAPS                           0.0.138    ADEssentials                                                                               
Function        Get-WinADComputers                                 0.0.138    ADEssentials                                                                               
Function        Get-WinADDelegatedAccounts                         0.0.138    ADEssentials                                                                               
Function        Get-WinADDFSHealth                                 0.0.138    ADEssentials                                                                               
Function        Get-WinADDHCP                                      0.0.138    ADEssentials                                                                               
Function        Get-WinADDiagnostics                               0.0.138    ADEssentials                                                                               
Function        Get-WinADDomain                                    0.0.138    ADEssentials                                                                               
Function        Get-WinADDuplicateObject                           0.0.138    ADEssentials                                                                               
Function        Get-WinADDuplicateSPN                              0.0.138    ADEssentials                                                                               
Function        Get-WinADForest                                    0.0.138    ADEssentials                                                                               
Function        Get-WinADForestControllerInformation               0.0.138    ADEssentials                                                                               
Function        Get-WinADForestOptionalFeatures                    0.0.138    ADEssentials                                                                               
Function        Get-WinADForestReplication                         0.0.138    ADEssentials                                                                               
Function        Get-WinADForestRoles                               0.0.138    ADEssentials                                                                               
Function        Get-WinADForestSchemaProperties                    0.0.138    ADEssentials                                                                               
Function        Get-WinADForestSites                               0.0.138    ADEssentials                                                                               
Function        Get-WinADForestSubnet                              0.0.138    ADEssentials                                                                               
Function        Get-WinADGroupMember                               0.0.138    ADEssentials                                                                               
Function        Get-WinADGroupMemberOf                             0.0.138    ADEssentials                                                                               
Function        Get-WinADLastBackup                                0.0.138    ADEssentials                                                                               
Function        Get-WinADLDAPBindingsSummary                       0.0.138    ADEssentials                                                                               
Function        Get-WinADLMSettings                                0.0.138    ADEssentials                                                                               
Function        Get-WinADObject                                    0.0.138    ADEssentials                                                                               
Function        Get-WinADPrivilegedObjects                         0.0.138    ADEssentials                                                                               
Function        Get-WinADProtocol                                  0.0.138    ADEssentials                                                                               
Function        Get-WinADProxyAddresses                            0.0.138    ADEssentials                                                                               
Function        Get-WinADServiceAccount                            0.0.138    ADEssentials                                                                               
Function        Get-WinADSharePermission                           0.0.138    ADEssentials                                                                               
Function        Get-WinADSiteConnections                           0.0.138    ADEssentials                                                                               
Function        Get-WinADSiteLinks                                 0.0.138    ADEssentials                                                                               
Function        Get-WinADTomebstoneLifetime                        0.0.138    ADEssentials                                                                               
Function        Get-WinADTrust                                     0.0.138    ADEssentials                                                                               
Function        Get-WinADTrustLegacy                               0.0.138    ADEssentials                                                                               
Function        Get-WinADUserPrincipalName                         0.0.138    ADEssentials                                                                               
Function        Get-WinADUsers                                     0.0.138    ADEssentials                                                                               
Function        Get-WinADUsersForeignSecurityPrincipalList         0.0.138    ADEssentials                                                                               
Function        Get-WinADWellKnownFolders                          0.0.138    ADEssentials                                                                               
Function        Invoke-ADEssentials                                0.0.138    ADEssentials                                                                               
Function        New-ADSite                                         0.0.138    ADEssentials                                                                               
Function        Remove-ADACL                                       0.0.138    ADEssentials                                                                               
Function        Remove-WinADDuplicateObject                        0.0.138    ADEssentials                                                                               
Function        Remove-WinADSharePermission                        0.0.138    ADEssentials                                                                               
Function        Rename-WinADUserPrincipalName                      0.0.138    ADEssentials                                                                               
Function        Repair-WinADACLConfigurationOwner                  0.0.138    ADEssentials                                                                               
Function        Repair-WinADEmailAddress                           0.0.138    ADEssentials                                                                               
Function        Repair-WinADForestControllerInformation            0.0.138    ADEssentials                                                                               
Function        Set-ADACLOwner                                     0.0.138    ADEssentials                                                                               
Function        Set-DnsServerIP                                    0.0.138    ADEssentials                                                                               
Function        Set-WinADDiagnostics                               0.0.138    ADEssentials                                                                               
Function        Set-WinADForestACLOwner                            0.0.138    ADEssentials                                                                               
Function        Set-WinADReplication                               0.0.138    ADEssentials                                                                               
Function        Set-WinADReplicationConnections                    0.0.138    ADEssentials                                                                               
Function        Set-WinADShare                                     0.0.138    ADEssentials                                                                               
Function        Set-WinADTombstoneLifetime                         0.0.138    ADEssentials                                                                               
Function        Show-WinADGroupMember                              0.0.138    ADEssentials                                                                               
Function        Show-WinADGroupMemberOf                            0.0.138    ADEssentials                                                                               
Function        Show-WinADTrust                                    0.0.138    ADEssentials                                                                               
Function        Sync-DomainController                              0.0.138    ADEssentials                                                                               
Function        Test-ADDomainController                            0.0.138    ADEssentials                                                                               
Function        Test-ADRolesAvailability                           0.0.138    ADEssentials                                                                               
Function        Test-ADSiteLinks                                   0.0.138    ADEssentials                                                                               
Function        Test-DNSNameServers                                0.0.138    ADEssentials                                                                               
Function        Test-FSMORolesAvailability                         0.0.138    ADEssentials                                                                               
Function        Test-LDAP                                          0.0.138    ADEssentials                                                                               

PS C:\Users\Username> #get-command -module adessentials
$PSVersionTable
Get-module ActiveDirectory,GroupPolicy,DHCPServer,ServerManager -ListAvailable

Name                           Value                                                                                                                                     
----                           -----                                                                                                                                     
PSVersion                      5.1.14393.4583                                                                                                                            
PSEdition                      Desktop                                                                                                                                   
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                                   
BuildVersion                   10.0.14393.4583                                                                                                                           
CLRVersion                     4.0.30319.42000                                                                                                                           
WSManStackVersion              3.0                                                                                                                                       
PSRemotingProtocolVersion      2.3                                                                                                                                       
SerializationVersion           1.1.0.1                                                                                                                                   

Name              : ActiveDirectory
Path              : C:\Windows\system32\WindowsPowerShell\v1.0\Modules\ActiveDirectory\ActiveDirectory.psd1
Description       : 
Guid              : 43c15630-959c-49e4-a977-758c5cc93408
Version           : 1.0.0.0
ModuleBase        : C:\Windows\system32\WindowsPowerShell\v1.0\Modules\ActiveDirectory
ModuleType        : Manifest
PrivateData       : 
AccessMode        : ReadWrite
ExportedAliases   : {}
ExportedCmdlets   : {[Add-ADCentralAccessPolicyMember, Add-ADCentralAccessPolicyMember], [Add-ADComputerServiceAccount, Add-ADComputerServiceAccount], 
                    [Add-ADDomainControllerPasswordReplicationPolicy, Add-ADDomainControllerPasswordReplicationPolicy], [Add-ADFineGrainedPasswordPolicySubject, 
                    Add-ADFineGrainedPasswordPolicySubject]...}
ExportedFunctions : {}
ExportedVariables : {}
NestedModules     : {}

Name              : DhcpServer
Path              : C:\Windows\system32\WindowsPowerShell\v1.0\Modules\DhcpServer\DhcpServer.psd1
Description       : 
Guid              : 90eaa9df-133a-450c-8728-91055cd946a1
Version           : 2.0.0.0
ModuleBase        : C:\Windows\system32\WindowsPowerShell\v1.0\Modules\DhcpServer
ModuleType        : Manifest
PrivateData       : 
AccessMode        : ReadWrite
ExportedAliases   : {[Reconcile-DhcpServerv4IPRecord, Reconcile-DhcpServerv4IPRecord]}
ExportedCmdlets   : {}
ExportedFunctions : {[Add-DhcpServerInDC, Add-DhcpServerInDC], [Add-DhcpServerv4Class, Add-DhcpServerv4Class], [Add-DhcpServerv4ExclusionRange, 
                    Add-DhcpServerv4ExclusionRange], [Add-DhcpServerv4Failover, Add-DhcpServerv4Failover]...}
ExportedVariables : {}
NestedModules     : {PS_DhcpServerAuditLog_v1.0.0, PS_DhcpServerDatabase_v1.0.0, PS_DhcpServerInDC_v1.0.0, PS_DhcpServerSetting_v1.0.0...}

Name              : GroupPolicy
Path              : C:\Windows\system32\WindowsPowerShell\v1.0\Modules\GroupPolicy\GroupPolicy.psd1
Description       : 
Guid              : 03e49e3a-be77-4422-9d97-8fe355c2284c
Version           : 1.0.0.0
ModuleBase        : C:\Windows\system32\WindowsPowerShell\v1.0\Modules\GroupPolicy
ModuleType        : Manifest
PrivateData       : 
AccessMode        : ReadWrite
ExportedAliases   : {[Get-GPPermissions, Get-GPPermissions], [Set-GPPermissions, Set-GPPermissions]}
ExportedCmdlets   : {[Backup-GPO, Backup-GPO], [Block-GPInheritance, Block-GPInheritance], [Copy-GPO, Copy-GPO], [Get-GPInheritance, Get-GPInheritance]...}
ExportedFunctions : {}
ExportedVariables : {}
NestedModules     : {}

Name              : ServerManager
Path              : C:\Windows\system32\WindowsPowerShell\v1.0\Modules\ServerManager\ServerManager.psd1
Description       : 
Guid              : d8e0cae9-8e9b-45bc-bfed-0aad50938af0
Version           : 2.0.0.0
ModuleBase        : C:\Windows\system32\WindowsPowerShell\v1.0\Modules\ServerManager
ModuleType        : Script
PrivateData       : 
AccessMode        : ReadWrite
ExportedAliases   : {[Add-WindowsFeature, Add-WindowsFeature], [Remove-WindowsFeature, Remove-WindowsFeature]}
ExportedCmdlets   : {[Get-WindowsFeature, Get-WindowsFeature], [Install-WindowsFeature, Install-WindowsFeature], [Uninstall-WindowsFeature, Uninstall-WindowsFeature], 
                    [Enable-ServerManagerStandardUserRemoting, Enable-ServerManagerStandardUserRemoting]...}
ExportedFunctions : {[Enable-ServerManagerStandardUserRemoting, Enable-ServerManagerStandardUserRemoting], [Disable-ServerManagerStandardUserRemoting, 
                    Disable-ServerManagerStandardUserRemoting]}
ExportedVariables : {}
NestedModules     : {}

After that, I ran it again just as invoke-testimo and it did NOT work. I then did the full uninstall, including Pester, and then deployed the new pester and installed Testimo again. Same issue.

Then I opened a new Powershell admin window and just ran invoke-testimo... Worked!

I wonder if this is some kind of problem when installing from an administrative ISE window instead of a normal powershell window?

PrzemyslawKlys commented 2 years ago

Pester is not required...

Testimo doesn't require to be run as admin, if you install as admin it's made available to all users. That's the only difference.

tastyratz commented 2 years ago

Sorry, I may have confused things there when I mentioned it was administrative. I run from an elevated window anyways in case I need to install anything. I think it may be more differences in using a powershell window vs a powershell ISE window. I have some other scripts that don't work in ISE due to certain commands. Maybe ISE has been part of the problem. I'll try some more next week.

PrzemyslawKlys commented 2 years ago

Could we assume this is solved now?

tastyratz commented 2 years ago

I think ISE was a big part of the issue and I also find that sometimes, removing modules and re-loading them seems to resolve issues when they are found on servers.

I've integrated all of this into my front end wrapper for testimo and seem to be having MUCH better luck now.

It's a bit of a mess here because I've hodgepodged it together and it's good enough since it works so forgive the mess but I'll share the code to help. The rsat/capabilities piece can be very OS sensitive on formatting, available commands, and methods. I have some disabled sections in case I decide to revisit them.

https://github.com/tastyratz/Public_scripts/blob/main/ADHealthReport.ps1

PrzemyslawKlys commented 2 years ago

Why are you not using built-in reports and creating custom made HTML?

tastyratz commented 2 years ago

I've been using this wrapper for a long time. The custom generated HTML section on the bottom was originally copied from your git readme file or blog a while back. It's just still in the script. I just ran one without it and it looks pretty identical so I'm guessing I'm just duplicating what you ended up committing upsteam. I'll comment that out on the copy I use.

PrzemyslawKlys commented 2 years ago

The new version is more advanced with more information, more states, and solutions - and it's part of the package.

tastyratz commented 2 years ago

Awesome. I'll have to dive into it a bit more then and check out the changes. I suspect revelations here might resolve #134 as well @blockanz