Use legacy application with tardigrade

Al-Muhandis commented 5 years ago

When I try to use the tardigrade broker (4.0.0 tag) came across some problems that will list here. Maybe it's not a problem, and my lack of understanding of the mechanism of tardigrade. So... First I try to build sagui library for my Debian webserver.

Since there is not much difference between Ubuntu and Debian, I decided to build a library on my Lubuntu computer according to this document. Notice: that to build requires CMake 3.5 or above, but many Debian systems has CMake version below. However I think it's not a problem: I have no problem to build for Debian in Ubuntu
Next, I don't really know what to do with the compiled library, but I just copied the libsagui.so.1.0.0 library with simlinks to the application folder on my webserver.
When I trying to change an existing unit of the broker to the brooktardigradebroker FPC can't compile the app because it doesn't find the following units: BrookLibraryLoader, BrookUtility, BrookStringMap, BrookHTTPUploads, BrookHTTPRequest, BrookHTTPResponse, BrookHTTPServer. I suggest that these are in the 5.0.0 branch. But then why add this broker to 4.0.0 at all? Also note, as I pointed out earlier, the example with tardigrade also requires to add the package tardigrade.lpk, which in the branch 4.0.0 is simply not.

daecccp commented 5 years ago

Hi Silvio, @Al-Muhandis

Put here some information about my first check out the tardigrade version.

1) ‘tardigrade’ hello word project (Brook 4) compiled x32 and x64 successfully on win 10. It works, but CPU activity is 25% constantly (!).

2) My legacy application is compiled successfully. According to “Legacy applications (Win32 and 64)” wiki.

Get an error “Library 'libsagui-1.dll' not loaded.”. The file “libsagui-1.dll” is placed correctly. The proc TbrookLibraryLoader.InternalOpen was never raised. Fixed when use Application.Initialize before BrookApp.Run.
CPU activity is 25% constantly (!).

3) Examples from Brook 5 work fine.

Checked Win 10 x64
Checked Linux Mint 19 x32
Checked the cross-compile from Linux to Win i386.

Win 10 x64, Lazarus x64 1.8.4, fpc 3.0.4 Linux Mint 19 x32, Lazarus 2RC2, fpc 3.0.4

silvioprog commented 5 years ago

Answering the questions from issue 129:

With a fresh OS is problematic. The thing is that I have all these server applications running on a working Debian web server located (without Lazarus IDE or FPC), which I rent from Hetzner. That is a virtual server.

It is just to build the library, so using any Linux version in a virtual machine on your personal PC. After built the library, you just need to upload it to your Hetzner server. :-)

Probably, after this issue, it will be available in next Ubuntu versions (and installed using something like sudo apt install libsagui). I would like to do the same for Debian, but I'm not sure how to do it. Anyway, please let me know if you get any problem building Sagui. (I also can send a built version for you)

Should I build a program in C++? I'm not familiar with this language

Sagui library is an ANSI C library. Its build is well documented and can be done by CMake, anyone can easily compile it even not knowing C/CMake. Using two command lines you build the library in all systems.

You mean brookframework 5?

Yes. Its code name is Tardigrade. :-)

Debian doesn't have much difference in Ubuntu but I certainly try to help, if there are differences

Awesome. Could you change the topic from Legacy applications (Win32 and 64) to Use legacy application with Tardigrade and adding subtopics Windows 32 and 64, Ubuntu (as TODO) and Debian (as TODO)? :sweat_smile:

silvioprog commented 5 years ago

Since there is not much difference between Ubuntu and Debian, I decided to build a library on my Lubuntu computer according to this document.

Perfect. Basically, you just need these steps (on Debian):

# 1. create temporary directions to build the lbrary
mkdir build && cd build/
# 2. prepare the build scripts (please use the build summary to check if the build is OK for your system)
cmake -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_SHARED_LIBS=ON ..
# 3. build / install the binary
make sagui && sudo make sagui install/strip
# 4. update the LD cache on your system
sudo ldconfig
# 5. uninstall (optional)
sudo make sagui uninstall

and the library will be built/installed.

Notice the document mentioned above was written when the version 1.0.0 was released, so please update all 1.0.0 entries to 1.2.0 (current stable release).

Next, I don't really know what to do with the compiled library, but I just copied the libsagui.so.1.0.0 library with simlinks to the application folder on my webserver.

The step 3. above will install the library in the 'user library path' on your system (generally /usr/local/lib on Debian based systems). If you want to change it (for example, to get the generated library in a custom directory), change the command cmake -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_SHARED_LIBS=ON .. to cmake -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=./YourOwnOutputDirectory .. (also, you can pass the full directory instead of ./)

When I trying to change an existing unit of the broker to the brooktardigradebroker FPC can't compile the app because it doesn't find the following units: BrookLibraryLoader, BrookUtility, BrookStringMap, BrookHTTPUploads, BrookHTTPRequest, BrookHTTPResponse, BrookHTTPServer. I suggest that these are in the 5.0.0 branch. But then why add this broker to 4.0.0 at all?

You should to install both Brook 4 and 5 on your Lazarus IDE. The Brook 4 will provide the legacy framework to build your application; Brook 5 will provide an high-performance secure embedded HTTP server for Brook 4, allowing you left external web servers (Apache, Nginx etc.).

Also note, as I pointed out earlier, the example with tardigrade also requires to add the package tardigrade.lpk, which in the branch 4.0.0 is simply not.

After installing Brook Tardigrade, go the Lazarus menu Project | Project inspector; right click at Required packages, find the BrookTardigrade package and add it to your application project, something like this:

Anyway, the step-by-step document should be updated adding these additional information.

silvioprog commented 5 years ago

Hi @daecccp . :smiley:

Firstly, thanks a lot for sharing this info. :+1:

‘tardigrade’ hello word project (Brook 4) compiled x32 and x64 successfully on win 10. It works, but CPU activity is 25% constantly (!).

Please open a new issue showing the steps to reproduce the problem. It will be fixed quickly.

... Get an error “Library 'libsagui-1.dll' not loaded.”. The file “libsagui-1.dll” is placed correctly. The proc TbrookLibraryLoader.InternalOpen was never raised. Fixed when use Application.Initialize before BrookApp.Run.

Perfect! :-) Optionally, you can use the 'standard' Application.Run instead of BrookApp.Run.

Examples from Brook 5 work fine.

Wow. It is awesome! The OSes I'm testing constantly:

Windows XP (:stuck_out_tongue:) (Delphi Tokyo 10.2 rev3; plus custom build of libsagui)
Windows 7 - 32/64 bits (latest Lazarus/FPC stable; Lazarus/FPC from trunk)
Windows 10 - 64 bits (latest Lazarus/FPC stable; Lazarus/FPC from trunk)
Xubuntu 18.04 - 64 bits (Lazarus/FPC from trunk)
Ubuntu 18.04 server - 64 bits (Delphi Tokyo 10.2 rev3)
Latest stable Raspbian in a Raspberry PI 3 board - i686 (Lazarus 1.6; FPC 3.0.0)
Android 7.1.2 in a Xioami Redmi 5A smartphone (Delphi Tokyo 10.2 rev3)

Al-Muhandis commented 5 years ago

Probably, after this issue, it will be available in next Ubuntu versions (and installed using something like sudo apt install libsagui).

Awesome!

I would like to do the same for Debian, but I'm not sure how to do it. Anyway, please let me know if you get any problem building Sagui. (I also can send a built version for you)

As I said, there is no problem to build libsagui library on Ubuntu (I think others will do so) for Debian. In any case, there is an instruction https://github.com/risoflora/libsagui/blob/master/INSTALL.md and for Debian, it is exactly the same, with some well-known nuances: usually instead of "sudo apt..."used" aptitude..." and the CMake library in standard Debian repositories below 3.5

Awesome. Could you change the topic from Legacy applications (Win32 and 64) to Use legacy application with Tardigrade and adding subtopics Windows 32 and 64, Ubuntu (as TODO) and Debian (as TODO)? 😅

Ok

Al-Muhandis commented 5 years ago

Notice the document mentioned above was written when the version 1.0.0 was released, so please update all 1.0.0 entries to 1.2.0 (current stable release).

Thank you, very valuable remark!!!

The step 3. above will install the library in the 'user library path' on your system (generally /usr/local/lib on Debian based systems). If you want to change it (for example, to get the generated library in a custom directory), change the command cmake -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_SHARED_LIBS=ON .. to cmake -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=./YourOwnOutputDirectory .. (also, you can pass the full directory instead of ./)

Ok. Thanks!

You should to install both Brook 4 and 5 on your Lazarus IDE.

Now I see!

Al-Muhandis commented 5 years ago

Tweaked the wiki.

I tested my legacy app with new broker Tardigrade. Works on Windows 10.

On Debian 8.11all Ok too. But I did not bother with building the library on the web server because of the CMake version in Debian official repos. I built on Ubuntu (both _x8684), then upload libsagui.so.1.2.0 in the /usr/lib folder and created the corresponding symlinks libsagui.so and libsagui.so.1. It's working.

silvioprog commented 5 years ago

I have interesting to build Sagui on Debian. Which version do you commend to download? I found the following link:

https://cdimage.debian.org/cdimage/archive/8.11.0/amd64/iso-cd

But I'm not sure which ISO a need to download (debian-8.11.0-amd64-CD-1.iso?).

Which CMake version is available in Debian 8.11? I'm using some feature that is available only in 3.5 or higher, but I can change it to support older versions.

Al-Muhandis commented 5 years ago

I have interesting to build Sagui on Debian. Which version do you commend to download? I found the following link:

https://cdimage.debian.org/cdimage/archive/8.11.0/amd64/iso-cd

But I'm not sure which ISO a need to download (debian-8.11.0-amd64-CD-1.iso?).

I'm using a Hetzner distribution, but I don't think it's appropriate. The Debian version (number and bit) is the same

Which CMake version is available in Debian 8.11? I'm using some feature that is available only in 3.5 or higher, but I can change it to support older versions.

cmake version 3.0.2

silvioprog commented 5 years ago

Could you try to build it again changing the CMake script? If so, in the main CMake file (libsagui/CMakeLists.txt), change this line from:

...
cmake_minimum_required(VERSION 3.5)
...

to:

...
cmake_minimum_required(VERSION 3.0.2)
...

and inside your build/ folder (in my PC, it is in ~/dev/git/libsagui/build):

rm -rf * # remove old files from last failed build
cmake -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_SHARED_LIBS=ON .. # prepare the environment
make sagui && sudo make sagui install/strip # build / install the library
sudo ldconfig # update the LD cache

Please let me know if this change works for you. :-)

Al-Muhandis commented 5 years ago

Yes, I can. All went well until make sagui && sudo make sagui install/strip with out:

Scanning dependencies of target pcre2-10.31
[  3%] Creating directories for 'pcre2-10.31'
[  6%] Performing download step (download, verify and extract) for 'pcre2-10.31'
CMake Error at /root/libsagui-1.2.0/build/pcre2-10.31/src/pcre2-10.31-stamp/pcre2-10.31-download.cmake:16 (message):
  Command failed: 1

   '/usr/bin/cmake' '-Dmake=' '-Dconfig=' '-P' '/root/libsagui-1.2.0/build/pcre2-10.31/src/pcre2-10.31-stamp/pcre2-10.3                         1-download-impl.cmake'

  See also

    /root/libsagui-1.2.0/build/pcre2-10.31/src/pcre2-10.31-stamp/pcre2-10.31-download-*.log

CMakeFiles/pcre2-10.31.dir/build.make:83: recipe for target 'pcre2-10.31/src/pcre2-10.31-stamp/pcre2-10.31-download' fa                         iled
make[3]: *** [pcre2-10.31/src/pcre2-10.31-stamp/pcre2-10.31-download] Error 1
CMakeFiles/Makefile2:126: recipe for target 'CMakeFiles/pcre2-10.31.dir/all' failed
make[2]: *** [CMakeFiles/pcre2-10.31.dir/all] Error 2
CMakeFiles/Makefile2:220: recipe for target 'src/CMakeFiles/sagui.dir/rule' failed
make[1]: *** [src/CMakeFiles/sagui.dir/rule] Error 2
Makefile:223: recipe for target 'sagui' failed
make: *** [sagui] Error 2

silvioprog commented 5 years ago

It seems your machine fails to access the server to download PCRE2 library from its official page. On your browser, please try to download this file via HTTP: https://ftp.pcre.org/pub/pcre/pcre2-10.31.tar.gz . An alternative FTP link: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre2-10.31.tar.gz . It is just to check. If it is successfully downloaded, please provide the entire cmake log for checking. Otherwise, try to rebuild the library after removing all files inside your build/ folder. Optionally, you can open a new issue at this page.

Notice if the folder libsagui/lib/ contains the pcre2-10.31.tar.gz.

It is not recommended, but you can alternatively disable the PCRE2 building in Sagui, just passing the option -DPCRE2_JIT_SUPPORT=OFF at build preparation.

Al-Muhandis commented 5 years ago

It seems your machine fails to access the server to download PCRE2 library from its official page. On your browser, please try to download this file via HTTP: https://ftp.pcre.org/pub/pcre/pcre2-10.31.tar.gz . An alternative FTP link: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre2-10.31.tar.gz . It is just to check. If it is successfully downloaded, please provide the entire cmake log for checking. Otherwise, try to rebuild the library after removing all files inside your build/ folder. Optionally, you can open a new issue at this page.

Notice if the folder libsagui/lib/ contains the pcre2-10.31.tar.gz.

somehow when re-build and re-make the error download gave already another file libmicrohttpd-0.9.59.tar.gz. In the folder libsagui/lib/ were the library libmicrohttpd-0.9.59.tar.gz, but with zero size. The error in more detail here. In the end, I manually downloaded using wget both libraries and it seems to have successfully installed the library sagui

Al-Muhandis commented 5 years ago

So legacy app via tardigrade can work as a HTTP daemon as well (at least in Debian), am I right?

silvioprog commented 5 years ago

An unexpected discovery for me. Legacy application with tardigrade works fine as a daemon: that is, it starts and stops via the command systemctl as it was with HTTPDaemonBroker

Awesome! :smiley:

So legacy app via tardigrade can work as a HTTP daemon as well (at least in Debian), am I right?

You are right. An interesting additional info (undocumented yet): Sagui library & Brook 5+ was programmed to never crash an application, so they are very useful for daemons/services, ensuring our applications will never go offline.

I have plans to implement a feature in Sagui to register an application automatically as daemon or service, something like these new API functions: sg_svc_register(path)/sg_svc_start(handle|auto-restart)/sg_svc_stop(handle)/sg_svc_syslog(callback). And a possible way to implement it in Brook would:

Application.Service.Install;
Application.Service.Start(True); // AutoRestart = True - auto-restart if it is closed

or something like this. Another possibility would to install our applications as daemon/service performing sudo ./our-app -i or our-app.exe -i, and the systemctl files (Linux) or service API/registry (Windows) would be handled automatically by our application.

I'm anxious to finish this work to finally continue implementing these new features in Brook.

Al-Muhandis commented 5 years ago

You are right. An interesting additional info (undocumented yet): Sagui library & Brook 5+ was programmed to never crash an application, so they are very useful for daemons/services, ensuring our applications will never go offline.

There are also an option to add line in systemd unit file my_daemon.service:

[Service]
... ... ...
Restart=on-failure
... ... ...

Al-Muhandis commented 5 years ago

I have plans to implement a feature in Sagui to register an application automatically as daemon or service, something like these new API functions: sg_svc_register(path)/sg_svc_start(handle|auto-restart)/sg_svc_stop(handle)/sg_svc_syslog(callback). And a possible way to implement it in Brook would:
Application.Service.Install;
Application.Service.Start(True); // AutoRestart = True - auto-restart if it is closed
or something like this. Another possibility would to install our applications as daemon/service performing sudo ./our-app -i or our-app.exe -i, and the systemctl files (Linux) or service API/registry (Windows) would be handled automatically by our application.

That's awesome! Though it is simpler to me to install the daemon unit manually via systemd in the lib/systemd/system/ folder (Debian)

silvioprog commented 5 years ago

@Al-Muhandis , I did some changes to support CMake 3.0.2 and it seems the build was fixed in Debian 8.11.

Please test and close if ok. Thanks for reporting! :+1:

Al-Muhandis commented 5 years ago

@silvioprog thank You! It's ok!

silvioprog / brookframework

Use legacy application with tardigrade #160