If `--io-manager=native`, `withHandleToHANDLE` does not work for a 'console' `Handle`

[mpilgrem]

This issue posting picks up a discussion, involving @bgamari and @Mistuke, originating in https://github.com/UnkindPartition/ansi-terminal/issues/114.

Absent GHC 9 series option --io-manager=native, System.Win32.Types.withHandleToHANDLE returns the (Windows) HANDLE of a (GHC) Handle, irrespective of whether the Handle was a 'console' Handle (like stdout) or a 'file' Handle.

Now, with --io-manager=native set, System.Win32.Types.withHandleToHANDLENative uses GHC.IO.Handle.Windows.handleToHANDLE and that latter function will not return the referenced HANDLE if Handle is not a 'file' Handle.

-- | Turn an existing Handle into a Win32 HANDLE. This function throws an
-- IOError if the Handle does not reference a HANDLE
handleToHANDLE :: Handle -> IO Win.HANDLE
handleToHANDLE h = case h of
  FileHandle _ mv -> do
    Handle__{haDevice = dev} <- readMVar mv
    case (cast dev :: Maybe (Win.Io Win.NativeHandle)) of
      Just hwnd -> return $ Win.toHANDLE hwnd
      Nothing   -> throwErr "not a file HANDLE"  -- <<< Throws this error if h is stdout :: Handle
  DuplexHandle{} -> throwErr "not a file handle"
  where
    throwErr msg = ioException $ IOError (Just h)
      InappropriateType "handleToHANDLE" msg Nothing Nothing

This causes a problem on ansi-terminal because it assumes that if a Handle ('console' or 'file') references a HANDLE, withHandleToHANDLE will return that HANDLE. (ansi-terminal then goes on to examine what sort of HANDLE has been referenced, using the Windows API and packages such as mintty.)

My personal preference is that withHandleToHANDLE should do what it currently does when --io-manager=native is not set (irrespective of the setting), and that if a more specific function is required that effectively 'filters out' a non-'file' Handle, that should be a distinct function.

As an aside, the current Haddock documentation for System.Win32.Types.withHandleToHANDLE does not identify the change in behaviour if --io-manager=native is set.

[Mistuke]

Thank you! I'm currently away but will fix it this weekend. Is there a testsuite I can run too?

[mpilgrem]

@Mistuke, by testsuite you may mean something more sophisticated than this, but a simple program:

module Main where

import System.Console.ANSI

main :: IO ()
main = clearScreen

should compile (with ghc-options -with-rtsopts=--io-manager=native) and execute without error. (When run, it should just clear the screen.)

It will need a dependency on package ansi-terminal. The version of the Win32 package will need to be specified to be at least Win32-2.13.1; a requirement of mintty-0.1.3. I use stack, with stack.yaml:

resolver: nightly-2021-11-14
packages:
- .
extra-deps:
- Win32-2.13.1.0

[Mistuke]

Thanks! Working on the fix now

[mpilgrem]

I have had some success with:

-- | Turn an existing Handle into a Win32 HANDLE. This function throws an
-- IOError if the Handle does not reference a HANDLE
handleToHANDLE' :: Handle -> IO HANDLE
handleToHANDLE' h = case h of
  FileHandle _ mv -> do
    Handle__{haDevice = dev} <- readMVar mv
    case (cast dev :: Maybe (IoHandle NativeHandle)) of
      Just hwnd -> return $ toHANDLE hwnd
      Nothing   -> case (cast dev :: Maybe (IoHandle ConsoleHandle)) of
                     Just hwnd -> return $ toHANDLE hwnd
                     Nothing   -> throwErr "not a file or console HANDLE"
  DuplexHandle{} -> throwErr "not a file handle"
  where
    throwErr msg = ioException $ IOError (Just h)
      InappropriateType "handleToHANDLE" msg Nothing Nothing

[Mistuke]

@mpilgrem the changes in #192 fix things for me, if you can confirm I'll release the package and do the backports.

[mpilgrem]

@Mistuke , this is what I have done to test successfully on Windows 11 (Version 10.0.22000.318) and using terminal software (Microsoft) Windows Terminal version 1.11.2921.0:

1. Compiled and executed the simple program above with/without -with-rtsopts=--io-manager=native and with stack.yaml (GHC 9.0.1):

   resolver: nightly-2021-11-19
   packages:
   - .
   extra-deps:
   - git: https://github.com/haskell/win32.git
   commit: b379321f57883b173398a2d6a2aa321d706021e7
   flags:
   mintty:
   Win32-2-13-1: true

2. Compiled and executed the simple program above with/without -with-rtsopts=--io-manager=native and with stack.yaml (GHC 9.2.1):

   resolver: nightly-2021-11-19
   compiler: ghc-9.2.1
   packages:
   - .
   extra-deps:
   - git: https://github.com/haskell/win32.git
   commit: b379321f57883b173398a2d6a2aa321d706021e7
   flags:
   mintty:
   Win32-2-13-1: true

3. Compiled and executed the ansi-terminal-example example (test) applicaton with with/without -with-rtsopts=--io-manager=native and with stack.yaml (GHC 9.2.1):

   resolver: nightly-2021-11-19
   compiler: ghc-9.2.1
   packages:
   - '.'
   extra-deps:
   - git: https://github.com/haskell/win32.git
   commit: b379321f57883b173398a2d6a2aa321d706021e7
   flags:
   mintty:
   Win32-2-13-1: true
   ansi-terminal:
   example: true

[Mistuke]

@mpilgrem awesome, thanks for confirming! I'll release and make the backports tomorrow morning.