search

modelica / ModelicaSpecification

Specification of the Modelica Language
https://specification.modelica.org
Creative Commons Attribution Share Alike 4.0 International
98 stars 42 forks source link

BuildProjects to automatically compile Modelica binaries #1668

Open modelica-trac-importer opened 5 years ago

modelica-trac-importer commented 5 years ago

Modified by dietmarw on 27 Feb 2015 12:01 UTC Since some time we have had a directory Modelica/Resources/BuildProjects to make it easier to compile the binary libraries used by the package. This makes maintenance easier since you do not need to compile and keep svn/git updated with new versions of the library all the time (since the user/tool provider can easily compile the source code).

I propose that this is specified and made automatic by Modelica tools (that support external C). I even made a prototype implementation in OpenModelica.

The following uses the library name ModelicaExternalC2 instead of ModelicaExternalC because OpenModelica itself has ModelicaExternalC pre-compiled:

model M
function f
   input Real time;
   output Real r;
external "C" annotation(Library="ModelicaExternalC2");
end f;

   Real x = f(time);
end M;

The prototype works by looking in the usual library directories for a file libmylib.a or libmylib.so (as well as OPENMODELICAHOME/lib/omc OPENMODELICAHOME/lib and ~/.openmodelica/binaries/PACKAGENAME). If no library is found, it tries to look for modelica://M/Resources/BuildProjects/*/autogen.sh. It then tries to run that project using the command below.

The generated library for ModelicaTables is linked against hdf5 if the system has hdf5 installed. If it is not installed, it asks the OS if the hdf5 is available in the package manager, and if it is it asks for admin rights to install it.

[[Image(hdf5-install.png, 100%)]]

[1] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: Removed directory /tmp/omc_compile_ModelicaExternalC2_7UJy2p

[2] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: Compiled /home/marsj/.openmodelica/binaries/M/libModelicaExternalC2.so by running build project /home/marsj/tmp/Resources/BuildProjects/autotools

[3] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: Succeeded with compilation and installation of the library using: command: sh ./autogen.sh && ./configure --libdir='/home/marsj/.openmodelica/binaries/M' && make && make install
[...]
Reading package lists... Building dependency tree... Reading state information... Suggested packages: libhdf5-doc The following NEW packages will be installed: libhdf5-dev 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/4616 kB of archives. After this operation, 41.8 MB of additional disk space will be used. Selecting previously unselected package libhdf5-dev. (Reading database ... 755658 files and directories currently installed.) Preparing to unpack .../libhdf5-dev_1.8.12+docs-1.1ubuntu1_amd64.deb ... Unpacking libhdf5-dev (1.8.12+docs-1.1ubuntu1) ... Setting up libhdf5-dev (1.8.12+docs-1.1ubuntu1) ...
[...]
Libraries have been installed in: /home/marsj/.openmodelica/binaries/M
[...]

[4] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: Changed directory to /tmp/omc_compile_ModelicaExternalC2_7UJy2p/BuildProjects/autotools

[5] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: cp -a "/home/marsj/tmp/Resources"/* "/tmp/omc_compile_ModelicaExternalC2_7UJy2p"

[6] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: Created directory /tmp/omc_compile_ModelicaExternalC2_7UJy2p

Reported by sjoelund.se on 26 Feb 2015 12:41 UTC Since some time we have had a directory Modelica/Resources/BuildProjects to make it easier to compile the binary libraries used by the package. This makes maintenance easier since you do not need to compile and keep svn/git updated with new versions of the library all the time (since the user/tool provider can easily compile the source code).

I propose that this is specified and made automatic by Modelica tools (that support external C). I even made a prototype implementation in OpenModelica.

The following uses the library name ModelicaExternalC2 instead of ModelicaExternalC because OpenModelica itself has ModelicaExternalC pre-compiled:

model M
function f
   input Real time;
   output Real r;
external "C" annotation(Library="ModelicaExternalC2");
end f;

   Real x = f(time);
end M;

The prototype works by looking in the usual library directories for a file libmylib.a or libmylib.so (as well as OPENMODELICAHOME/lib/omc OPENMODELICAHOME/lib and ~/.openmodelica/binaries/PACKAGENAME). If no library is found, it tries to look for modelica://M/Resources/BuildProjects/*/autogen.sh. It then tries to run that project using the command below.

The generated library for ModelicaTables is linked against hdf5 if the system has hdf5 installed. If it is not installed, it asks the OS if the hdf5 is available in the package manager, and if it is it asks for admin rights to install it.

Image

[1] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: Removed directory /tmp/omc_compile_ModelicaExternalC2_7UJy2p

[2] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: Compiled /home/marsj/.openmodelica/binaries/M/libModelicaExternalC2.so by running build project /home/marsj/tmp/Resources/BuildProjects/autotools

[3] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: Succeeded with compilation and installation of the library using: command: sh ./autogen.sh && ./configure --libdir='/home/marsj/.openmodelica/binaries/M' && make && make install
[...]
Reading package lists... Building dependency tree... Reading state information... Suggested packages: libhdf5-doc The following NEW packages will be installed: libhdf5-dev 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/4616 kB of archives. After this operation, 41.8 MB of additional disk space will be used. Selecting previously unselected package libhdf5-dev. (Reading database ... 755658 files and directories currently installed.) Preparing to unpack .../libhdf5-dev_1.8.12+docs-1.1ubuntu1_amd64.deb ... Unpacking libhdf5-dev (1.8.12+docs-1.1ubuntu1) ... Setting up libhdf5-dev (1.8.12+docs-1.1ubuntu1) ...
[...]
Libraries have been installed in: /home/marsj/.openmodelica/binaries/M
[...]

[4] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: Changed directory to /tmp/omc_compile_ModelicaExternalC2_7UJy2p/BuildProjects/autotools

[5] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: cp -a "/home/marsj/tmp/Resources"/* "/tmp/omc_compile_ModelicaExternalC2_7UJy2p"

[6] 13:32:08 Translation Notification
[/home/marsj/tmp/a.mo: 2:1-6:6]: Created directory /tmp/omc_compile_ModelicaExternalC2_7UJy2p

Migrated-From: https://trac.modelica.org/Modelica/ticket/1668

modelica-trac-importer commented 5 years ago

Modified by dietmarw on 27 Feb 2015 12:01 UTC

modelica-trac-importer commented 5 years ago

Comment by sjoelund.se on 27 Feb 2015 20:29 UTC Talking to @thiele, we came to the conclusion that it would be good if impact could provide links to zip-files containing the binaries by platform.

  1. This makes the download size smaller (as you only download the binaries you need)
  2. This works as long as the index generator knows how to automatically compile binaries for the most common platforms, or if someone provides these binaries using github releases (possibly automated by a 3rd-party build service)

This solves the issues that some Modelica tools only partially support the Modelica language (you can include C code in the model, so a C compiler can be assumed to exist).

modelica-trac-importer commented 5 years ago

Comment by sjoelund.se on 27 Feb 2015 20:35 UTC Some issues exist for how to specify Library="..." annotations. It would be good if the Modelica tool could know which libraries are provided by the Modelica library, and which ones are system libraries. A proposal would be something like:

Library={"system:dbus-1"} // For things that require dbus, regardless of platform
Library={"win32|win64:shlwapi","linux:dl"} // Make the thing before the : a regular expression matching the platform name x86_64-linux or linux64 for example

This allows you to specify libraries that are different on each platform. Currently, the spec says you are recommended to link all dependencies. But these are often platform-specific and best practice is to only depend on a single statically linked library instead.

modelica-trac-importer commented 5 years ago

Comment by dietmarw on 15 Apr 2015 14:33 UTC Ticket retargeted after milestone closed

modelica-trac-importer commented 5 years ago

Comment by beutlich on 18 Jan 2017 21:26 UTC See also #2104 (or close it as duplicate).

modelica-trac-importer commented 5 years ago

Comment by hansolsson on 25 Oct 2017 07:41 UTC And conclusions in #2145

HansOlsson commented 5 years ago

There is use case for specifying the source code (directories) that we had not considered earlier.

The case is for generating FMUs containing source code, for models that use external functions. It might be that this should be part of a re-opened #2145 - separate from this ticket. However, they need to be considered together. In this case we need the source code so that we can copy it to the FMU.

One proposal to solve this is to have an annotation SourceDirectory (similary as LibraryDirectory and IncludeDirectory) possibly with default "modelica://LibraryName/Resources/Source". That can then be used when building a source code FMU by first copying the contents of that that source-directory to the FMU.

It seems best to also copy the build-project to the source code FMU (I assume the build-projects are general enough that they are useful also for the target-system of the FMU), which requires one of the following:

HansOlsson commented 1 year ago

Design meeting:

Seems that straightforward to standardize on CMake - and have some CMake-variable for ModelicaUtilities.h from tools.

HansOlsson commented 1 year ago

Shall we push this?

  1. Provide CMakeList.txt for MSL - even without anything in the specification, with a readme (create issue for it?)
  2. Tools can then support cmake
  3. If that happens we can standardize the existing behavior in tools in the specification

(And the cmake-variable for ModelicaUtilities.h)

casella commented 1 month ago

@beutlich just completed modelica/ModelicaStandardLibrary#4093, now a proof of concept is available in the MSL master branch.