wrong suffix for shared MODULES hdt*,altcmpsc, s370x, dyncrypt

[ghost]

macOS makes a distinction between a shared library and a shared module, and CMake knows about that

CMAKE_SHARED_LIBRARY_SUFFIX CMAKE_SHARED_MODULE_SUFFIX

define LTDL_SHLIB_EXT "@CMAKE_SHARED_MODULE_SUFFIX@"

add_library( ... MODULE ... )

[srorso]

Hi Enrico,

Right now, all of the shared code, whether loaded when execution starts (shared library) or dynamically (module), is built as a shared library for all platforms. I checked on my system; hdt*, dyncrypt, etc have a Mach-O file type of MH_DYLIB (6). Nothing carries a Mach-O file type of MH_BUNDLE (8).

On UNIX-like systems, I do not believe there is a difference between shared library and dynamically loaded module. There is no difference as far as I know on Windows, although CMake might suppress creation of the import library for the DLL if the DLL is tagged as a MODULE rather than a SHARED library.

Looking at macOS, it seems that there used to be a big difference between modules and shared libraries, notably that shared libraries could not be unloaded. That difference seems to have been erased as Apple transitioned from the dyld API for modules to the dlopen API. Hercules on macOS built using CMake reports library unload on an rmmod command, but perhaps macOS returns "unloaded" to Hercules and keeps the address space occupied. I will continue looking at this subject; the Apple documentation web sites seem to be pretty good.

(Sidebar: Hercules built using CMake on any platform does not use ltdl.c, and ltdl.c contains the only references to the dyld API.)

It is easy enough to change the CMake scripts to generate MODULE instead of SHARED libraries; it's just a few lines in Herc00_Includes.cmake function herc_Define_Shared_Lib. The function name should be changed as well if I use it to to both SHARED and MODULE. But that's mechanical, a global edit in Herc60_CreateTargets.cmake.

I will continue my research and welcome your expertise, particularly on modules versus shared libraries.

At the moment, getting a good CMake script for macOS seems pretty easy. Which means I probably don't understand the problem. (I had to learn bunches about Windows to do the Windows CMake scripting.)

Thank you for pointing this out!

Best Regards, Steve Orso

[jphartmann]

On UNIX, a module was traditionally invoked by the exec() system call, which maps the .text and .data segments and allocates storage for the .bss segment, but a module can also be opened by dlopen(); it is even possible for a module to dlopen() itself, thereby getting access to the loader symbols it contains.

So, no there is no material difference between a module and a shared library; they are handled the same by the kernel.

There is, however, a profound difference between a module linked statically and one linked dynamically.

[ivan-w]

John, You are confusing "modules" (which are intended to be loaded in an existing address space at run time), "shared libraries" (which are intended to be loaded in an address space at load time) and "executables". (which are invoked by exec() et al and replace the entire address space).

[jphartmann]

No, Ivan I am not confusing anything in UNIX. Your vendetta is tiresome. Stop it.

[ivan-w]

What vendetta ? Stop the Paranoia dude ! Chill !

[ghost]

I know that …. but since macOS keeps things distinct - at the formal level at least, I feel that there would be nothing wrong in following the logic cheers enrico

PS… no reason to change the name of the function… CMake uses the same function add_library, just with different parameters

attached is a zipped git diff to implement the change tested and working - apart the BFP error

On 11 Mar 2018, at 15:27, Stephen Orso notifications@github.com wrote:

Hi Enrico,

Right now, all of the shared code, whether loaded when execution starts (shared library) or dynamically (module), is built as a shared library for all platforms. I checked on my system; hdt*, dyncrypt, etc have a Mach-O file type of MH_DYLIB (6). Nothing carries a Mach-O file type of MH_BUNDLE (8).

On UNIX-like systems, I do not believe there is a difference between shared library and dynamically loaded module. There is no difference as far as I know on Windows, although CMake might suppress creation of the import library for the DLL if the DLL is tagged as a MODULE rather than a SHARED library.

Looking at macOS, it seems that there used to be a big difference between modules and shared libraries, notably that shared libraries could not be unloaded. That difference seems to have been erased as Apple transitioned from the dyld API for modules to the dlopen API. Hercules on macOS built using CMake reports library unload on an rmmod command, but perhaps macOS returns "unloaded" to Hercules and keeps the address space occupied. I will continue looking at this subject; the Apple documentation web sites seem to be pretty good.

(Sidebar: Hercules built using CMake on any platform does not use ltdl.c, and ltdl.c contains the only references to the dyld API.)

It is easy enough to change the CMake scripts to generate MODULE instead of SHARED libraries; it's just a few lines in Herc00_Includes.cmake function herc_Define_Shared_Lib. The function name should be changed as well if I use it to to both SHARED and MODULE. But that's mechanical, a global edit in Herc60_CreateTargets.cmake.

I will continue my research and welcome your expertise, particularly on modules versus shared libraries.

At the moment, getting a good CMake script for macOS seems pretty easy. Which means I probably don't understand the problem. (I had to learn bunches about Windows to do the Windows CMake scripting.)

Thank you for pointing this out!

Best Regards, Steve Orso

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hercules-390/hyperion/issues/253#issuecomment-372119553, or mute the thread https://github.com/notifications/unsubscribe-auth/ABUXcFTMI5mlihxuAeseCk2R4r1WQtf4ks5tdTRjgaJpZM4SlfRN.

[ghost]

looks like the attachment did not pass thru the email MODULE_patch.txt.zip

[srorso]

Hi Enrico,

Patch applied...works well. I get .so's with Mach-O type MH_BUNDLE for the dynamically loaded modules and .dylib's with Mach-O type MH_DYLIB for the shared libraries. If this makes more sense for macOS developers, then it is well worth doing and was part of the reason I asked about the differences.

Still left with the BFP issue...as expected.

Many thanks!

Best Regards, Steve Orso

[srorso]

Closed prematurely....should not be closed until correction committed. Sorry.

[srorso]

Hi Enrico,

The correction for this will be committed within a week or so; I wish to do a full round of regression and validation testing. What manner of acknowledgement, attribution, and/or copyright would you like for your contribution to this issue and to #255?

And thanks again for both patches.

Best Regards, Steve Orso

[ghost]

I did not do too much a thanks is more than enough ;-)

if You have the compelling need for the copyright/licence stuff the BOOST Licence is good enough for me

I am still experimenting with the RPATH stuff I will Keep You posted on the Results

e.

On 19 Mar 2018, at 14:51, Stephen Orso notifications@github.com wrote:

Hi Enrico,

The correction for this will be committed within a week or so; I wish to do a full round of regression and validation testing. What manner of acknowledgement, attribution, and/or copyright would you like for your contribution to this issue and to #255 https://github.com/hercules-390/hyperion/issues/255?

And thanks again for both patches.

Best Regards, Steve Orso

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/hercules-390/hyperion/issues/253#issuecomment-374218248, or mute the thread https://github.com/notifications/unsubscribe-auth/ABUXcFEY8BEe5a_lgduaOHYeFOMYQj8Jks5tf7fMgaJpZM4SlfRN.

[srorso]

Hercules builds correctly on macOS High Sierra and Mojave.