[SR-13591] REPL is not working on swift-5.3-RELEASE (Windows)

[swift-ci]

Previous ID	SR-13591
Radar	None
Original Reporter	gazsiazasz (JIRA User)
Type	Bug

Environment

`Windows 10 - swift-5.3-RELEASE`

Additional Detail from JIRA

| | | |------------------|-----------------| |Votes | 3 | |Component/s | | |Labels | Bug, Windows | |Assignee | @compnerd | |Priority | Medium | md5: 34aed1a92808f2bdb31c9273d970f787

is blocked by:

• https://github.com/apple/llvm-project/issues/4418

is duplicated by:

• https://github.com/apple/llvm-project/issues/4328

Issue Description:

When I issue the command swift it should launch the REPL. But instead of that it gives back the prompt immediately.

**********************************************************************
** Visual Studio 2019 Developer Command Prompt v16.7.4
** Copyright (c) 2020 Microsoft Corporation
********************************************************************** 
[vcvarsall.bat] Environment initialized for: 'x64'

C:\Program Files (x86)\Microsoft Visual Studio\2019\Community>swift

C:\Program Files (x86)\Microsoft Visual Studio\2019\Community>swift --version
compnerd.org Swift version 5.3 (swift-5.3-RELEASE)
Target: x86_64-unknown-windows-msvc

C:\Program Files (x86)\Microsoft Visual Studio\2019\Community>

[compnerd]

Thanks for the report. I believe that the backport in https://github.com/apple/llvm-project/pull/1832 should address this issue.

[compnerd]

Taking another look at this - (3) ⚙ D77287 Windows: support `DoLoadImage` (llvm.org) and Windows: support `DoLoadImage` by compnerd · Pull Request #3580 · apple/llvm-project (github.com) repair enough of LLDB to fix the module loading.

Subsequently, I can hack together enough pieces to get things working:

> swift -- -sdk %SDKROOT%
Welcome to Swift version 5.6-dev (LLVM 4fa084761bda946, Swift 2889d5149c7ca32).
Type :help for assistance.
1> :process load S:\Library\icu-69.1\usr\bin\icudt69.dll
Loading "S:\Library\icu-69.1\usr\bin\icudt69.dll"...ok
1> :process load S:\Library\icu-69.1\usr\bin\icuuc69.dll
Loading "S:\Library\icu-69.1\usr\bin\icuuc69.dll"...ok
Image 1 loaded.
1> :process load S:\Library\icu-69.1\usr\bin\icuin69.dll
Loading "S:\Library\icu-69.1\usr\bin\icuin69.dll"...ok
Image 2 loaded.
1> :process load S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\swiftCore.dll
Loading "S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\swiftCore.dll"...ok
Image 3 loaded.
1> :process load S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\swiftCRT.dll
Loading "S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\swiftCRT.dll"...ok
Image 4 loaded.
1> :process load S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\swiftWinSDK.dll
Loading "S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\swiftWinSDK.dll"...ok
Image 5 loaded.
1> :process load S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\dispatch.dll
Loading "S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\dispatch.dll"...ok
Image 6 loaded.
1> :process load S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\swiftDispatch.dll
Loading "S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\swiftDispatch.dll"...ok
Image 7 loaded.
1> :process load S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\Foundation.dll
Loading "S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\Foundation.dll"...ok
Image 8 loaded.
1> :process load S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\swift_Differentiation.dll
Loading "S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\swift_Differentiation.dll"...ok
Image 9 loaded.
1> import Foundation
2> RunLoop.current
$R0: Foundation.RunLoop = {
  Foundation.NSObject = {}
  _cfRunLoopStorage = 0x000001dd437f4c40 {
    Foundation.NSObject = {}
    _cfinfo = {
      info = 11136
      pad = 0
    }
  }
  monitoredPortsWithModesLock = {
    Foundation.NSObject = {}
    mutex = 0x1dd437dba60 {
      pointee = {
        Ptr = nil
      }
    }
    timeoutCond = 0x1dd437dba10 {
      pointee = {
        Ptr = nil
      }
    }
    timeoutMutex = 0x1dd437f37e0 {
      pointee = {
        Ptr = nil
      }
    }
    name = nil
  }
  monitoredPortsWithModes = 0 key/value pairs
  monitoredPortObservers = 0 key/value pairs
}
3>

Once the changes are merged, the next set of issues lie in the library search path setup which is why the DLLs need to be loaded manually.

+ @JDevlieghere

[compnerd]

lldb/Windows: synchronize the sources with upstream by compnerd · Pull Request #4231 · apple/llvm-project (github.com)

This corrects some invalid behaviour. It seems that the behaviour of LoadLibraryExW will not search the default search paths (i.e. Path) as we wish to search altered search paths. One option is to explicitly inject Path in the search locations and another is to change LLDB to perform a secondary lookup to search the default search path. However, there is still another issue of this code path not being executed properly 🙁.

[compnerd]

I could use a bit of help testing this, I think that we are in a slightly better position now with the REPL, though it still needs further work. In the meantime, if someone else can test and verify that that this seems to work better with the 2023-04-11-a snapshot it would be useful. In order to get this to actually work the REPL needs to be invoked with some additional parameters, namely -L... where the value is the location of the installed runtime (defaults to %Program Files%\swift\runtime-development\usr\bin). This is slightly confusing and will need to be resolved, but getting some preliminary testing on this would be helpful to ensure that the various issues are identified.

S:\>swift repl "-LS:\Program\ Files\swift\runtime-development\usr\bin"
Welcome to Swift version 5.9-dev (LLVM 91080e65df791d6, Swift a35e5ab3295f464).
Type :help for assistance.
1> import Foundation
2> ProcessInfo.processInfo
$R0: Foundation.ProcessInfo = {
  Foundation.NSObject = {}
  processName = "repl_swift.exe"
  $__lazy_storage_$__operatingSystemVersionString = nil
  _processorCount = 1
  _activeProcessorCount = 20
  _physicalMemory = 0
}
3>

You will need to ensure that python is in your path of course but that has always been a requirement. Also note the extra slash to quote the space in %ProgramFiles%.

[lxbndr]

It works with 2023-04-11 snapshot (LLVM 8277d73ee656251, Swift 4de53fc2e7cee4d). Thanks for note about extra slash in Program\ Files. Tried ProcessInfo.processInfo and RunLoop.current and got a valid output.

[compnerd]

https://github.com/apple/llvm-project/pull/6927 should be the last bit that I think that is missing. With that in place, I can now launch the REPL without any additional parameters (swift repl).

[compnerd]

The changes have been merged and should be part of the next snapshot.

[svanimpe]

Should this be working in 5.9? If I try to run swift repl, I just get an error saying python39.dll is missing.

I have Python 3.10 installed, as per the instructions on https://www.swift.org/install/windows/#installation-via-windows-package-manager

Running files with swift file.swift also still fails with the familiar "could not load the swift standard library" error. (https://github.com/apple/swift/issues/56202)

[Keithcat1]

I just got the REPL working with These instructions. It can evaluate 2+2, for example. Running files with swift h.swift now fail with errors about JIT session: symbols not found and then abunch of mangled symbol names. I'm using the Swift installer from here, which may work better than the default.

[compnerd]

swift h.swift is going to fail for now. You might be able to use swift -O h.swift for the time being. There is still more work that needs to be done on the interpreter.

[Keithcat1]

Adding -O allows a simple hprint("hello world!") to compile, but anything that tries to import foundation still can't materialize symbols. I'll just compile. When is the REPL going to not require python39.dll? This seems like the main blocker and means I need 2 installations of Python on my Windows computer.