nim-lang / Nim

Nim is a statically typed compiled systems programming language. It combines successful concepts from mature languages like Python, Ada and Modula. Its design focuses on efficiency, expressiveness, and elegance (in that order of priority).
https://nim-lang.org
Other
16.54k stars 1.47k forks source link

exported DllMain not working #21334

Open 4zv4l opened 1 year ago

4zv4l commented 1 year ago

Description

I tried to load a dynamic library (dll), when I made it from Linux with the exported DllMain like this

import winim/lean

proc NimMain() {.cdecl,importc.}

proc DllMain(hinstDLL: HINSTANCE, fdwReason: DWORD, lpvReserved: LPVOID) : BOOL
  {.stdcall, exportc, dynlib.} =

  if fdwReason == DLL_PROCESS_ATTACH:
    NimMain()
    MessageBox(0, "Hello, world !", "Nim is Powerful", 0)

  return true

it worked just fine
but when I compile it from Windows directly, when I export the DllMain, it does not work (when I just put a simple function in the dll it works, but so when I try to export DllMain, the dll cannot be loaded anymore)

I compile it like this:
nim c --app=lib --nomain .\dll_test.nim

this is the program (main.nim)

import winim/lean
import dynlib

const dll = "./dll_test.dll"

proc main() =
  let hlib = loadLib(dll)
  if hlib == nil:
    echo "could not import: ", dll

  unloadLib(hlib)

main()

Nim Version

nim -v Nim Compiler Version 1.9.1 [Windows: amd64] Compiled at 2023-02-03 Copyright (c) 2006-2023 by Andreas Rumpf

active boot switches: -d:release

Current Output

could not import: ./dll_test.dll

Expected Output

(MessageBox showing Hello World)

Possible Solution

No response

Additional Information

with the same code, cross compiling from Linux it works.

foxoman commented 1 year ago

Move the NimMain inside dllmain proc like this:

import winim/lean

proc NimMain() {.cdecl, importc.}

proc DllMain(hinstDLL: HINSTANCE, fdwReason: DWORD,
    lpvReserved: LPVOID): BOOL {.stdcall, exportc, dynlib.} =
  NimMain()

  if fdwReason == DLL_PROCESS_ATTACH:
    MessageBox(0, "Hello, world !", "Nim is Powerful", 0)

  return true

Also check your antivrus because in my system the test did not work first time because the dll was quarantined.

4zv4l commented 1 year ago

my antivirus doesn't quarantine my dll and even moving the NimMain() doesn't seem to change, but I found a way to make my code work using when isMainModule so that's fine ^^

heterodoxic commented 1 year ago

my antivirus doesn't quarantine my dll and even moving the NimMain() doesn't seem to change, but I found a way to make my code work using when isMainModule so that's fine ^^

It seems to me you're basically trying to re-implement what Nim already does for you: offering a DLL entry-point, while generalizing away its low-level implementation. As you found out yourself, this is the functional equivalent of your first attempt dll_test.nim (which could be behind a when isMainModule):

import winim/lean
MessageBox(0, "Hello, world !", "Nim is Powerful", 0)

So I'd say this is working as intended?

4zv4l commented 1 year ago

Just like here https://youtu.be/9fV8tWb2W1M?t=470 and here https://github.com/byt3bl33d3r/OffensiveNim#creating-windows-dlls-with-an-exported-dllmain, it should be possible to create your own entry point for the dll. I saw it was possible, just I couldn't make it works when I needed it so I used the default main function. But that differs a lot since the dllMain gives you arguments when it is called meanwhile main doesn't give you access to those arguments. (I sent both link about malicious way to use that but for the project of the company I was working for it was for a good purpose).

edit: the project I had to make is finished and work fine, just when I was trying to find a solution I needed the hinstDLL which I couldn't get without dllmain.

thamyekh commented 1 week ago

I can confirm that nim 2.0+ (tested on 2.0.6) broke the exported DllMain, reverting to version 1.6.10 resolved the issue:

choosenim 1.6.10
nimble install winim
nim c -d=mingw -d=release --app=lib --nomain --cpu=amd64 --nimcache=dllcache dll_test.nim

# this works afterward
rundll32 mylib.dll,DllMain

I chose 1.6.10 because any version lower you will come across could not import: SSL_get_peer_certificate when trying to run nimble.

It was this blog that also claimed nim DLLs were broken on nim 2.0 which lead me confirm if it was true: https://medium.com/@monocloud786/hijacking-and-proxying-dlls-for-fun-and-profit-712674de99aa

metagn commented 1 week ago

Related are PRs #21910 and #22323 and their issues #21501 and #22321, does adding --mm:refc fix it?