search

open-watcom / open-watcom-v2

Open Watcom V2.0 - Source code repository, Wiki, Latest Binary build, Archived builds including all installers for download.
Other
989 stars 162 forks source link

Compatibility problem with Win32 libraries #1287

Closed fafner2000 closed 6 months ago

fafner2000 commented 6 months ago

Hello,

3 years ago, I had managed to link a program against SDL.lib using Watcom (version 1.9). I tried again a few days ago but ended up with this error: Error! E3146: SDL.lib is an invalid object file

I wrote a simple program that imports SDL_init and prints its address, that fails to link. I managed to find the Windows 7 virtual machine where it had worked 3 years ago, tried to compile that program, and it linked successfully (it ran successfully too by the way). Then I substituted Watcom with the version I had troubles with (version 2.0 2024-04-10), and suddenly it failed to link too on the Windows 7 virtual machine. To me, it can mean only 2 things:

In case it is a regression bug, mind if I try to bissect the git repository to find the commit that broke the compatibility? I would have done it already, but with 2+ hours to compile a commit, I prefer to make sure there is a bug before realizing there was a simple option to add :-)

jmalak commented 6 months ago

What exact version of SDL.lib you use and for which target you linking and on which host? what command you use for linking? Did you check it with OW 1.9? Anyway SDL 2.x is tested with OW 2.0 regulary and no problem is reported.

fafner2000 commented 6 months ago

System is Windows 7. I compile without target, which in this case should be nt. Here is the script I use:

set WATCOM=C:\WATCOM
set PATH=%PATH%;%WATCOM%\binnt
set INCLUDE=%WATCOM%\h;%WATCOM%\h\nt
set LIBS=-LC:\lib\sdl2\lib\x86 SDL2.lib

owcc -Werror -Wall test.c -c -o test.o
owcc %LIBS% test.o -o test.exe

I used the last version of SDL2 (2.30.3), and with Watcom 1.9 (probably from late May 2021 if I have to believe my project files) it works.

When compiling with Watcom 2.0 2024-04-10, it gives this error: Error! E3146: SDL2.lib is an invalid object file

If I change the importation if SDL_init to SDL_init2, I get this error: Error! E2028: _SDL_Init2 is an undefined reference

This gives the impression that Watcom perfectly understands the format of the library, because when the function is wrong, it somehow knows that it doesn't contain the function. I suspect that for some obscure reason it fails at a later stage. I had thought the architecture of the library was wrong, but because Watcom 1.9 can link it I think there is no problem with that, especially since I can reproduce the same problem with 2 very different versions of SDL.

Now if you say that SDL2 is tested regularly, that means that I must do something wrong, but so far I can't understand what (hopefully, when I know, it will be slap on the face time).

jmalak commented 6 months ago

I think a problem is with owcc utility. try to use -v option to show real commands for OW tools.

fafner2000 commented 6 months ago

Here is the working compile command for 1.9 : wcc386 test.c -we -w4 -fo=test.o -fr And the file coughed up by wlink:

libpath C:\lib\sdl2\lib\x86
option quiet
debug dwarf
system nt
file test.o
name test.exe
option noextension
library SDL2.lib
option caseexact

The non-working compile command for 2.0 : wcc386 -we -w4 -fo=test.o -fr test.c And the link file:

option quiet
debug dwarf lines
libpath C:\lib\sdl2\lib\x86
file test.o
name test.exe
option noextension
library SDL2.lib

I also tried to inject the working link file to the non-working linker, but it didn't change anything (I might have done it wrong though, I just typed wlink @link.txt after editing it).

jmalak commented 6 months ago

The problem could be that system directive is missing therefore wlink decided some default value. could you generate linker map file ( -fm option ) where the desided format should be writen.

What version of OW tools you use 32-bit or 64-bit?

fafner2000 commented 6 months ago

The problem could be that system directive is missing therefore wlink decided some default value.

As mentioned above, I had tried to give the link file that works to the linker that didn't work, with no visible change. I suppose that in the process it specified the system directive.

could you generate linker map file ( -fm option ) where the desided format should be writen.

I attached the map file of version 1.9 , the only one that can generate a map file. The other one just dumps the same error as in the console. map.txt

What version of OW tools you use 32-bit or 64-bit?

As far as I know, they are 32-bits version, I didn't download the 64-bits version. If you tell me how to check that, I will make sure it is the case.

jmalak commented 6 months ago

Where you get SDL2.lib, from which package? Anyway version of OW tools is in banner. Please could you give me your SDL2.lib to be able check. I suppose you have properly setup WATCOM environment variable and also has OW2 binnt subdirectory on the path for OW2.

fafner2000 commented 6 months ago

Where you get SDL2.lib, from which package?

I downloaded it at this page: https://github.com/libsdl-org/SDL/releases/tag/release-2.30.3 File: SDL2-devel-2.30.3-VC.zip

Anyway version of OW tools is in banner.

Here is the banner of 2.0 (the one that currently doesn't work):

Open Watcom x86 32-bit Compilers Driver Program
Version 2.0 beta Apr 10 2024 02:31:13 (32-bit)
Copyright (c) 2002-2024 The Open Watcom Contributors. All Rights Reserved.
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See https://github.com/open-watcom/open-watcom-v2#readme for details.

Version 1.9 doesn't display a banner, and I didn't find how to make display it.

I suppose you have properly setup WATCOM environment variable and also has OW2 binnt subdirectory on the path for OW2.

My environment variables are above. I checked them a gazillon times, so I think they are good. But feel free to make sure I didn't miss anything: the more eyes, the better.

jmalak commented 6 months ago

But SDL2.lib is for VC which is generaly not compatible with OW. I will check SDL2.lib.

jmalak commented 6 months ago

The problem is in ORL module for PE format reading. I will fix it ASAP.

fafner2000 commented 6 months ago

I had read some time ago that Open Watcom had been made to be as compatible as possible with Visual Studio, which is the reason why I picked this one. Did I misread ? Anyway, it seemed to be the case at least for version 1.9 (maybe by accident though).

jmalak commented 6 months ago

It is general issue if no WATCOM support is included, but I thing that SDL 2 header files support Watcom that there are proper definition for WATCOM compilers that OW recognize VC definition of API entries and symbols etc. how it is defined for VC. It is same for OW1.9 and OW2.0. But the issue is in OW 2.0 COFF libraries reader. OMF and ELF libraries are OK. It is due to lot of changes in OW2.0 code to better support 64-bit formats and some new architectures against OW1.9.

fafner2000 commented 6 months ago

I have "reverted" time to Sun Jan 1 2023, commit 2acad650a61ccd8d71c3f4b555093134fc52bc0c. I can link SDL2.lib properly. Commit f1d32bc6324a8472fa2d2440165720650e7e9760 dated Jan 22 2024 doesn't link.

It takes an awful lot of time to compile and test a commit, but since the computer is doing the job while I'm doing something else, I will eventually find the commit that broke the linking. I'll let you know as soon as I find it.

fafner2000 commented 6 months ago

I think that the commit that broke linking with SDL2.lib is eb404e4c9fe686dd9120699748cf4ae608ea5e32. However, it doesn't compile because of errors that were fixed in the following commit.

doesn't link: 3b08601b08a1d3805b7631190580b70b0bc0aed6 dated Mon Mar 13 2023
    correct mistakes

syntax error: eb404e4c9fe686dd9120699748cf4ae608ea5e32 dated Mon Mar 13 2023
    cleanup orl code

links properly: 3f6717960f1e57d77ba56ade95af997cfd2530ba dated Sun Mar 12 2023
    correct pointer processing for PE file reading
    cleanup PE file processing (64-bit)
jmalak commented 6 months ago

The original code (OW 1.9) used check for "unknown" processor which was not transparent as check for import libraries. now there is explicit check for import libraries and "unknown" processor check was removed that it is transparent.

fafner2000 commented 6 months ago

Just compiled commit 1d56a6248a64e081f6dd722f7ad2393c109ddcc9 from this morning and I could link my test program on both Windows 7 and Linux (targeting Windows in both cases). Then I had to deal with missing libraries (such as MSVCR110.dll) and similar stuff, but this is standard Windows business. My test program now runs properly, as well as the libssh2 test program that can open and close the library.

jmalak commented 6 months ago

Thanks for your testing

fafner2000 commented 6 months ago

And thanks to you for the quick fix :-)

I tried to compile a complex project involving libssh2 and ran into a hell of crashes due to broken calling conventions, mostly coming from callbacks that my program provided to libssh2 (which doesn't explicitely declare them at all in callbacks). Now that I got all of them (I hope), everything is working fine.