dsccommunity / StorageDsc

DSC resource module is used to manage storage on Windows Servers.
https://dsccommunity.org
MIT License
66 stars 51 forks source link

OpticalDiskDriveLetter: DVD Drive for Windows Server Gen2 (UEFI) OS Images filtered #285

Closed AjKundnani closed 5 months ago

AjKundnani commented 6 months ago

Problem description

Windows Server Gen2 OS Images (like Windows Server 2022 Azure Edition) use caption Microsoft Virtual DVD-ROM for optical drives.

This caption is getting filtered with DSC_OpticalDiskDriveLetter causing DSC to skip the drive letter assignment for Optical drives in Gen2 OS images.

This issue is not occurring for Gen1 (BIOS) OS images as DVD drive caption is “Msft Virtual CD/ROM ATA Device”

Verbose logs

VERBOSE: Perform operation 'Invoke CimMethod' with following parameters, ''methodName' =                                                                                                                                                  SendConfigurationApply,'className' = MSFT_DSCLocalConfigurationManager,'namespaceName' =                                                                                                                                                  root/Microsoft/Windows/DesiredStateConfiguration'.                                                                                                                                                                                        VERBOSE: An LCM method call arrived from computer ws2022vm01 with user sid                                                                                                                                                                                                                                                                                                                            VERBOSE: [ws2022vm01]: LCM:  [ Start  Set      ]                                                                                                                                                                                          VERBOSE: [ws2022vm01]: LCM:  [ Start  Resource ]  [[OpticalDiskDriveLetter]MoveOpticalDrive]                                                                                                                                              VERBOSE: [ws2022vm01]: LCM:  [ Start  Test     ]  [[OpticalDiskDriveLetter]MoveOpticalDrive]                                                                                                                                              VERBOSE: [ws2022vm01]:                            [[OpticalDiskDriveLetter]MoveOpticalDrive] Get-OpticalDiskDriveLetter: Using Get-CimInstance to get the drive letter of optical disk 1 in the system.                                   VERBOSE: [ws2022vm01]:                            [[OpticalDiskDriveLetter]MoveOpticalDrive] Perform operation 'Enumerate CimInstances' with following parameters, ''namespaceName' = root\cimv2,'className' = Win32_CDROMDrive'.         VERBOSE: [ws2022vm01]:                            [[OpticalDiskDriveLetter]MoveOpticalDrive] Operation 'Enumerate CimInstances' complete.                                                                                                 VERBOSE: [ws2022vm01]:                            [[OpticalDiskDriveLetter]MoveOpticalDrive] Get-OpticalDiskDriveLetter: The optical disk 1 could not be found in the system.                                                             VERBOSE: [ws2022vm01]: LCM:  [ End    Test     ]  [[OpticalDiskDriveLetter]MoveOpticalDrive]  in 1.0320 seconds.                                                                                                                          VERBOSE: [ws2022vm01]: LCM:  [ Skip   Set      ]  [[OpticalDiskDriveLetter]MoveOpticalDrive]                                                                                                                                              VERBOSE: [ws2022vm01]: LCM:  [ End    Resource ]  [[OpticalDiskDriveLetter]MoveOpticalDrive]                                                                                                                                              VERBOSE: [ws2022vm01]: LCM:  [ Start  Resource ]  [[WaitForDisk]DataDisk]                                                                                                                                                                 VERBOSE: [ws2022vm01]: LCM:  [ Start  Test     ]  [[WaitForDisk]DataDisk]                                                                                                                                                                 VERBOSE: [ws2022vm01]:                            [[WaitForDisk]DataDisk] Test-TargetResource: Checking for disk with Number '2'.                                                                                                         VERBOSE: [ws2022vm01]:                            [[WaitForDisk]DataDisk] Test-TargetResource: Found disk with Number '2' named 'Msft Virtual Disk'.                                                                                      VERBOSE: [ws2022vm01]: LCM:  [ End    Test     ]  [[WaitForDisk]DataDisk]  in 6.0310 seconds.                                                                                                                                             VERBOSE: [ws2022vm01]: LCM:  [ Skip   Set      ]  [[WaitForDisk]DataDisk]
VERBOSE: [ws2022vm01]: LCM:  [ End    Resource ]  [[WaitForDisk]DataDisk]
VERBOSE: [ws2022vm01]: LCM:  [ Start  Resource ]  [[Disk]DataVolume]
VERBOSE: [ws2022vm01]: LCM:  [ Start  Test     ]  [[Disk]DataVolume]
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Test-TargetResource: Testing disk with Number '2' status for drive letter 'E'.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Test-TargetResource: Checking if disk with Number '2' is initialized.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Test-TargetResource: Disk with Number '2' is initialized with partition style 'RAW' but 'GPT' is required.
VERBOSE: [ws2022vm01]: LCM:  [ End    Test     ]  [[Disk]DataVolume]  in 0.1250 seconds.
VERBOSE: [ws2022vm01]: LCM:  [ Start  Set      ]  [[Disk]DataVolume]
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: Setting disk with Number '2' status for drive letter 'E'.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: Checking disk with Number '2' partition style.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: Initializing disk with Number '2' as 'GPT'.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: Disk with Number '2' does not contain a partition assigned to drive letter 'E'.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Disk with Number '2' already contains partitions, but size parameter is not specified.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: Creating partition on disk with Number '2' with drive letter 'E' using all free space.
The requested access path is already in use.
Activity ID: {83f40266-80e0-496c-8170-ab72a846ed47}
    + CategoryInfo          : InvalidArgument: (StorageWMI:) [], CimException
    + FullyQualifiedErrorId : StorageWMI 42002,New-Partition
    + PSComputerName        : localhost

VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: New partition '' on disk with Number '2' is readonly. Waiting for it to become writable.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: New partition '' on disk with Number '2' is readonly. Waiting for it to become writable.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: New partition '' on disk with Number '2' is readonly. Waiting for it to become writable.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: Changing volume 'E' label to 'NFV'.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: Assigning drive letter 'E'.
VERBOSE: [ws2022vm01]:                            [[Disk]DataVolume] Set-TargetResource: Successfully initialized 'E'.
VERBOSE: [ws2022vm01]: LCM:  [ End    Set      ]  [[Disk]DataVolume]  in 4.5620 seconds.
The PowerShell DSC resource '[Disk]DataVolume' with SourceInfo 'C:\Users\Downloads\DSC.ps1::73::9::Disk' threw one or more non-terminating errors while running the Set-TargetResource functionality. These errors are
logged to the ETW channel called Microsoft-Windows-DSC/Operational. Refer to this channel for more details.
    + CategoryInfo          : InvalidOperation: (:) [], CimException
    + FullyQualifiedErrorId : NonTerminatingErrorFromProvider
    + PSComputerName        : localhost

VERBOSE: [ws2022vm01]: LCM:  [ End    Set      ]
The SendConfigurationApply function did not succeed.
    + CategoryInfo          : NotSpecified: (root/Microsoft/...gurationManager:String) [], CimException
    + FullyQualifiedErrorId : MI RESULT 1
    + PSComputerName        : localhost

VERBOSE: Operation 'Invoke CimMethod' complete.
VERBOSE: Time taken for configuration job to complete is 16.656 seconds

DSC configuration

LocalConfigurationManager {
            ConfigurationModeFrequencyMins = 60
            ConfigurationMode              = 'ApplyAndAutoCorrect'
            RebootNodeIfNeeded             = $false
            AllowModuleOverwrite           = $true
            DebugMode                      = 'ForceModuleImport'
        }

        OpticalDiskDriveLetter MoveOpticalDrive {
            DiskId      = 1
            DriveLetter = 'Z'
            Ensure      = "Absent"
        }

        WaitforDisk DataDisk {
            DependsOn        = "[OpticalDiskDriveLetter]MoveOpticalDrive"
            DiskId           = $DataDiskLUN
            RetryIntervalSec = 60
            RetryCount       = 60
        }

        Disk DataVolume {
            DiskId      = $DataDiskLUN
            DriveLetter = $InstallationDrive
            FSLabel     = 'NFV'
            DependsOn   = '[WaitForDisk]DataDisk'
        }
    }

Suggested solution

Remove filter for caption Microsoft Virtual DVD-ROM from DSC for Gen2 OS images.

# Get the optical disk matching the Id
$opticalDisks = Get-CimInstance -ClassName Win32_CDROMDrive |
    Where-Object -FilterScript {
    -not (($_.DeviceID.Split('\')[-1]).Length -gt 20)
}

Operating system the target node is running

OsName               : Microsoft Windows Server 2022 Datacenter Azure Edition
OsOperatingSystemSKU : 407
OsArchitecture       : 64-bit
WindowsVersion       : 2009
WindowsBuildLabEx    : 20348.1.amd64fre.fe_release.210507-1500
OsLanguage           : en-US
OsMuiLanguages       : {en-US}

PowerShell version and build the target node is running

Name                           Value
----                           -----
PSVersion                      5.1.20348.2227
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.20348.2227
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

StorageDsc version

5.1.0
AjKundnani commented 5 months ago

Sample device IDs:

Azure Gen1 (BIOS) VM – Windows Server 2022 21H2 Caption = 'Msft Virtual CD/ROM ATA Device' DeviceID = 'IDE\CDROMMSFT_VIRTUAL_CD/ROM____1.0\5&CFB56DE&0&1.0.0'

Azure Gen2 (UEFI) VM – Windows Server 2022 Azure Edition 21H2 Caption = 'Microsoft Virtual DVD-ROM' DeviceID = 'SCSI\CDROM&VEN_MSFT&PROD_VIRTUAL_DVD-ROM\5&394B69D0&0&000002'

PlagueHO commented 5 months ago

Working on this now @AjKundnani - I'll aim to submit PR tomorrow or the next day (as haven't had the time to finish it tonight).

PlagueHO commented 5 months ago

@AjKundnani - can you confirm how you're mounting the OD in the Azure VM? I've spun one up to do some further investigation, but need to know the process you're following to create the Virtual DVD-ROM.

From looking at the DeviceID, it matches the pattern of an ISO that has been mounted rather than a Virtual DVD-ROM, so it is difficult to differentiate between a mounted ISO and a Virtual DVD-ROM.

AjKundnani commented 5 months ago

@AjKundnani - can you confirm how you're mounting the OD in the Azure VM? I've spun one up to do some further investigation, but need to know the process you're following to create the Virtual DVD-ROM.

@PlagueHO - The Virtual DVD-ROM is added by default at time of Azure VM creation. It's not being added manually.

AjKundnani commented 5 months ago

Additional mounted ISO properties:

Size                        : 6812706816
TransferRate                : 0
VolumeName                  : CCCOMA_X64FRE_EN-US_DV9
VolumeSerialNumber          : D534E15C
PlagueHO commented 5 months ago

I've been working on a better way to determine if a CIM Win32_CDROMDrive device is a mounted ISO or a virtualized device that is managed by the Host OS. It looks like the current method (using Caption + DeviceID length) just isn't reliable anymore.

I put together a table and this is what I came up with:

Type Caption DeviceID Manage using
Mounted ISO in Windows Server 2019 Microsoft Virtual DVD-ROM SCSI\CDROM&VEN_MSFT&PROD_VIRTUAL_DVD-ROM\2&1F4ADFFE&0&000004 DSC_MountImage*
Physical device MATSHITA BD-MLT UJ260AF SCSI\CDROM&VEN_MATSHITA&PROD_BD-MLT_UJ260AF\4&23A5A6AC&0&000200 DSC_OpticalDiskDriveLetter
Hyper-V Gen1 (BIOS/IDE) VM - Windows Server 2019 Msft Virtual CD/ROM ATA Device IDE\CDROMMSFT_VIRTUAL_CD/ROM____1.0\5&CFB56DE&0&1.0.0 DSC_OpticalDiskDriveLetter
Hyper-V Gen2 (UEFI/SCSI) VM - Windows Server 2019 Microsoft Virtual DVD-ROM SCSI\CDROM&VEN_MSFT&PROD_VIRTUAL_DVD-ROM\000001 DSC_OpticalDiskDriveLetter
Hyper-V Gen1 (BIOS/IDE) VM - Windows Server 2022 Msft Virtual CD/ROM ATA Device IDE\CDROMMSFT_VIRTUAL_CD/ROM____1.0\5&CFB56DE&0&1.0.0 DSC_OpticalDiskDriveLetter
Hyper-V Gen2 (UEFI/SCSI) VM - Windows Server 2022 Microsoft Virtual DVD-ROM ... DSC_OpticalDiskDriveLetter
Azure Gen1 (BIOS/IDE) VM – Windows Server 2022 Azure Edition Msft Virtual CD/ROM ATA Device IDE\CDROMMSFT_VIRTUAL_CD/ROM____1.0\5&CFB56DE&0&1.0.0 DSC_OpticalDiskDriveLetter
Azure Gen2 (UEFI/SCSI) VM – Windows Server 2022 Azure Edition Microsoft Virtual DVD-ROM SCSI\CDROM&VEN_MSFT&PROD_VIRTUAL_DVD-ROM\5&394B69D0&0&000002 DSC_OpticalDiskDriveLetter

The first item in the table should not be managed by DSC_OpticalDiskDriveLetter (instead managed by DSC_MountImage).

I don't think this is specifically an Azure issue.

One technique I have found that might detect if the device is a mounted ISO is to:

  1. For the CIM Instance, get the Drive letter: $driveLetter = ($cimInstance.Drive - replace ":$")
  2. If the Drive letter is set, query the volume information for the Device path: $devicePath = (Get-Volume -DriveLetter $driveLetter).Path -replace "\\$"
  3. Look up the Disk Image for the DevicePath: Get-DiskImage -DevicePath $devicePath - if no error occurs then it is a mounted ISO. If a "The specified disk is not a virtual disk." error occurs then it is not an ISO and can be managed by this resource.

image

Notes: If a drive letter isn't assigned to a win32_cdromdrive instance then will need to fall back to some other method (need to investigate this).

There are no other differences in win32_cdromdrive when the same ISO is mounted using Mount-DiskImage or mounted from the host OS (e.g., Hyper-V). The only property differences are DeviceID.

This is still not elegant - but it should work (where the existing method really doesn't any more).

If no objections (or better suggestions), I'll implement later tonight.

PlagueHO commented 5 months ago

Still working on this. I've completely removed the use of Caption and DeviceID from being used to detect an ISO as it's completely unreliable after I ran tests across HyperV and Azure.

I've replaced it with this function:

<#
    .SYNOPSIS
        This helper function determines if an optical disk can be managed
        or not by this resource.

    .PARAMETER OpticalIdsk
        The cimv2:Win32_CDROMDrive instance of the optical disk that should
        be checked.

    .OUTPUTS
        System.Boolean

    .NOTES
        This function will use the following logic when determining if a drive is
        a mounted ISO and therefore should **not** be mnanaged by this resource:

        - Get the Drive letter assigned to the drive in the `cimv2:Win32_CDROMDrive`
          instance
        - If the drive letter is set, query the volume information for the device
           using drive letter and get the device path.
        - Look up the disk image using the device path.
        - If no error occurs then the device is a mounted ISO and should not be
          used with this resource.
          If a "The specified disk is not a virtual disk." error occurs then it
          is not an ISO and can be managed by this resource.
#>
function Test-OpticalDiskCanBeManaged
{
    [CmdletBinding()]
    [OutputType([System.Boolean])]
    param
    (
        [Parameter(Mandatory = $true)]
        [Microsoft.Management.Infrastructure.CimInstance]
        $OpticalDisk
    )

    $diskCanBeManaged = $false
    $driveLetter = ($OpticalDisk.Drive -replace ":$")
    $devicePath = (Get-Volume -DriveLetter $driveLetter).Path -replace "\\$"

    try
    {
        <#
            If the device is not a mounted ISO then this will throw an
            exception with the message "The specified disk is not a virtual disk."
        #>
        Get-DiskImage -DevicePath $devicePath -ErrorAction Stop | Out-Null
    }
    catch [Microsoft.Management.Infrastructure.CimException]
    {
        if ($_.Exception.MessageId -eq 'HRESULT 0xc03a0015')
        {
            # This is not a mounted ISO, so can manage
            $diskCanBeManaged = $true
        }
    }

    return $diskCanBeManaged
}

But will need to build a new suite of unit tests to validate.

PlagueHO commented 5 months ago

@AjKundnani - the PR has been created. Recommend you pull my branch (https://github.com/PlagueHO/StorageDsc/tree/Issue-285) down onto your Azure VMs and run the integration tests with:

.\build.ps1 -Tasks Build -ResolveDependency
invoke-Pester -Script .\tests\Integration\DSC_OpticalDiskDriveLetter.Integration.Tests.ps1

You may need to enable PS remoting on the VM with winrm quickconfig as the integration tests usually need this.

AjKundnani commented 5 months ago
.\build.ps1 -Tasks Build -ResolveDependency

Getting below error:

[build] Parsing defined tasks
[build] Loading Configuration from \StorageDsc-Issue-285\StorageDsc-Issue-285\build.yaml
\StorageDsc-Issue-285\output\RequiredModules\InvokeBuild\5.11.0\Invoke-Build.ps1 : The term 'Set-BuildHeader' is not recognized as the name of a cmdlet, function, script file,
or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At \StorageDsc-Issue-285\build.ps1:377 char:9
+         Invoke-Build @PSBoundParameters -Task $Tasks -File $MyInvocat ...
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Invoke-Build.ps1], Exception
    + FullyQualifiedErrorId : Invoke-Build.ps1
PlagueHO commented 5 months ago

@AjKundnani - that is strange that you got that error. I'll give it a try later on today.

But in the meantime, this PR has been merged and a new preview version is available (thanks @johlju for reviewing!): https://www.powershellgallery.com/packages/StorageDsc/6.0.0-preview0002

@bbonaby - Should I wait to get your PR #279 merged before releasing 6.0.0 final? I don't think your PR is a breaking change though, so we can always release your one as 6.1.0.

AjKundnani commented 5 months ago

But in the meantime, this PR has been merged and a new preview version is available (thanks @johlju for reviewing!): https://www.powershellgallery.com/packages/StorageDsc/6.0.0-preview0002

@PlagueHO - Validated the preview release successfully with DVD Drive & ISO mounted. Mounted ISO was not impacted by the change. What is the ETA for merge with production release please?

PlagueHO commented 5 months ago

Hi @AjKundnani - a production release has just been published: https://www.powershellgallery.com/packages/StorageDsc/6.0.0