make install after build gets cmake error

[billpriest]

All, Building for deployment on bare metal (so don't want docker image). Steps: git clone https://github.com/halfgaar/FlashMQ.git cd FlashMQ git checkout v1.1.0 mkdir build cd build cmake .. make sudo make install

Consolidate compiler generated dependencies of target flashmq [100%] Built target flashmq Install the project... -- Install configuration: "" -- Installing: /usr/bin/flashmq -- Up-to-date: /var/lib/flashmq -- Up-to-date: /var/log/flashmq -- Installing: /etc/flashmq/flashmq.conf -- Installing: /lib/systemd/system/flashmq.service CMake Error at cmake_install.cmake:126 (file): file INSTALL cannot find "/home/myusername/FlashMQ/man/flashmq.conf.5": No such file or directory.

It seems that I didn't have docbook2x installed. Would be nice if the README.md included the dependencies for those of us that don't intend to deploy in docker. If I missed it somehow in the documentation then I apologize.

Looks like an awesome broker.

[halfgaar]

I have had to disable documentation building in a fair amount software I use. Now I understand the gap between developer and user :).

I think I will make docbook a development dependency and commit the generated man pages to git. I'll have to devise a good flow, so allow me some time.

Edit: and: thanks :+1:

[bigsmoke]

In the simplest form, one could simply check the generated nroff/HTML man pages into Git and remove the “invoke make in man” step from build.sh. The only downside to that would be the potentiality to forget to rebuild the man page after changes.

Another option I'm looking into is to keep the “always invoke make in man from build.sh” behavior, but change the man/Makefile to only rebuild if the source files are from a later Git commit (or have uncommitted changes) than the destination files. This does head butt a bit with the fact that Git commits are not necessarily chronological, so maybe just always rebuild if the commit hashes of the destination and target change (or when the transformation support files do).

A cleaner and simpler option could be to change either the Makefile or build.sh to turn this from a fatality into a warning.

And then there is the approach to make the building of the man stuff explicitly opt-in or opt-out as your comment in the build.sh script suggests.

[billpriest]

All, I understand completely, I have to cross-compile for my target platform building everything from scratch and still there are tools dependencies that I miss until it goes to our build server that is a bare-bones container. Can a named pipe be used as a log filename? I'd like to be able to compress the log file on the fly to minimize disk useage.

Regards,

Bill Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Wiebe Cazemier @.> Sent: Tuesday, January 31, 2023 1:21:55 PM To: halfgaar/FlashMQ @.> Cc: Bill Priest @.>; Author @.> Subject: [EXTERNAL]Re: [halfgaar/FlashMQ] make install after build gets cmake error (Issue #30)

I have had to disable documentation building in a fair amount software I use. Now I understand the gap between developer and user :).

I think I will make docbook a development dependency and commit the generated man pages to git. I'll have to devise a good flow, so allow me some time.

— Reply to this email directly, view it on GitHubhttps://github.com/halfgaar/FlashMQ/issues/30#issuecomment-1410939372, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU7WZ7PMK4O5PMPCXH5O7FLWVFQ5HANCNFSM6AAAAAAUMZ3UFI. You are receiving this because you authored the thread.Message ID: @.***>

[halfgaar]

Can a named pipe be used as a log filename? I'd like to be able to compress the log file on the fly to minimize disk useage.

I've never tried it to be honest.

FlashMQ stores log entries in memory and they are written out to disk by a thread. This makes it non-blocking, so in that respect it should work fine and the event loop will never be held up. Do take into account there is no limit on the memory log lines take, currently. So if you stop consuming the named pipe, you will run out of memory.

I see you took the tag v1.1.0. You can also go one commit higher and then you have the ability to send USR1 signals to it to reopen the log files without reloading the config. You can then have an hourly rotate.

[billpriest]

Weibe, Thanks for the info I will give it a shot and let you know how it goes. Now I just need to convince my boss we should switch from mosquitto; if named pipes work it will be a slam dunk.

Thanks,

Bill

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Wiebe Cazemier @.> Sent: Wednesday, February 1, 2023 2:05:11 AM To: halfgaar/FlashMQ @.> Cc: Bill Priest @.>; Author @.> Subject: [EXTERNAL]Re: [halfgaar/FlashMQ] make install after build gets cmake error (Issue #30)

Can a named pipe be used as a log filename? I'd like to be able to compress the log file on the fly to minimize disk useage.

I've never tried it to be honest.

FlashMQ stores log entries in memory and they are written out to disk by a thread. This makes it non-blocking, so in that respect it should work fine and the event loop will never be held up. Do take into account there is no limit on the memory log lines take, currently. So if you stop consuming the named pipe, you will run out of memory.

I see you took the tag v1.1.0. You can also go one commit higher and then you have the ability to send USR1 signals to it to reopen the log files without reloading the config. You can then have an hourly rotate.

— Reply to this email directly, view it on GitHubhttps://github.com/halfgaar/FlashMQ/issues/30#issuecomment-1411619162, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU7WZ7KGPGJWYYOSNX6AIYLWVIKLPANCNFSM6AAAAAAUMZ3UFI. You are receiving this because you authored the thread.Message ID: @.***>

[halfgaar]

A quick test with cat shows that files are closed ('broken pipe'?) when one end closes the pipe. I think I'd have to make this smarter. Currently, it switches to stdout, but it could do a limited number of retries.

[halfgaar]

I did some tests. It works out of the box, but with caveats. You need to use tail on the fifo, because tail waits for more data when there is none. So:

tail -n 1 --follow /tmp/logfifo | gzip | hexdump -C

However, it takes a long time before you see anything, because it waits for a big block of data to compress.

And, you can't tail a compressed file, which is not ideal.

A simple hourly log rotater may be better. And if you go ahead one commit, or wait for the next release, you can send a SIGUSR1 to have it reopen the log files without risk.

[billpriest]

Hi Wiebe, I've switched to using xc which has options to control how big a buffer it uses and lvl of compression; also you can kill and the resulting file is still uncompressible. The pipe will be good enough for me as I need a new dated filename when the logging resumes. My trigger for a new log file isn't time but an application specific event. Thanks for a great broker that doesn't require a bunch of libraries.

Bill

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Wiebe Cazemier @.> Sent: Wednesday, February 1, 2023 1:56:48 PM To: halfgaar/FlashMQ @.> Cc: Bill Priest @.>; Author @.> Subject: [EXTERNAL]Re: [halfgaar/FlashMQ] make install after build gets cmake error (Issue #30)

I did some tests. It works out of the box, but with caveats. You need to use tail on the fifo, because tail waits for more data when there is none. So:

tail -n 1 --follow /tmp/logfifo | gzip | hexdump -C

However, it takes a long time before you see anything, because it waits for a big block of data to compress.

And, you can't tail a compressed file, which is not ideal.

A simple hourly log rotater may be better. And if you go ahead one commit, or wait for the next release, you can send a SIGUSR1 to have it reopen the log files without risk.

— Reply to this email directly, view it on GitHubhttps://github.com/halfgaar/FlashMQ/issues/30#issuecomment-1412641622, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU7WZ7J3J66YQ63WFQU56R3WVK5YBANCNFSM6AAAAAAUMZ3UFI. You are receiving this because you authored the thread.Message ID: @.***>

[halfgaar]

BTW, the way you build doesn't turn on compiler optimizations. The script build.sh is the way to go, which I'll update to deal with the docs properly.

What's missing in the cmake step is: cmake -DCMAKE_BUILD_TYPE="Release" ..

[billpriest]

I don't need/want a docker image (or the tools to build it to be required). When I build for the target, I use a toolchain file that controls the compiler optimization settings. I was evaluating FlashMQ on x86 linux to start. I didn't see a way to log all the msgs the broker receives; is this possible or do I need to use a client to get that type of logging. Thanks for the suggestion,

Bill

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Wiebe Cazemier @.> Sent: Thursday, February 2, 2023 9:42:41 AM To: halfgaar/FlashMQ @.> Cc: Bill Priest @.>; Author @.> Subject: [EXTERNAL]Re: [halfgaar/FlashMQ] make install after build gets cmake error (Issue #30)

BTW, the way you build doesn't turn on compiler optimizations. The script build.sh is the way to go, which I'll update to deal with the docs properly.

What's missing in the cmake step is: cmake -DCMAKE_BUILD_TYPE="Release" ..

— Reply to this email directly, view it on GitHubhttps://github.com/halfgaar/FlashMQ/issues/30#issuecomment-1413947921, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU7WZ7K7JZKCYOQSEPPXAMDWVPIXDANCNFSM6AAAAAAUMZ3UFI. You are receiving this because you authored the thread.Message ID: @.***>

[halfgaar]

I decided to do what @bigsmoke said and commit the generated documentation to git. Cmake/Cpack has a dependency on them, so this makes it more robust.

I don't need/want a docker image (or the tools to build it to be required)

The build.sh is not for Docker. It makes an executable binary, a .deb file, and if the tools are installed, an AppImage.

When I build for the target, I use a toolchain file that controls the compiler optimization settings

The official releases are with -O3. If you don't do that, which you don't as it stands, you'll miss out on about 40% performance gain.

I didn't see a way to log all the msgs the broker receives; is this possible or do I need to use a client to get that type of logging.

Do you mean permanently, or for debugging? With or without payload? I have already been thinking of implementing a log_publishes option, but that probably wouldn't contain the payload.

You can also write a plugin in c++. Takes a few lines of code. However, at this stage, the payload is not available there, but that may change in the near future. I'm hammering out ideas.

I'll close the ticket. The 'discussions' section can be used for for further discussions.

[billpriest]

Thanks Wiebe awesome support.

Bill

---

From: Wiebe Cazemier @.> Sent: Thursday, February 2, 2023 1:52 PM To: halfgaar/FlashMQ @.> Cc: Bill Priest @.>; Author @.> Subject: [EXTERNAL]Re: [halfgaar/FlashMQ] make install after build gets cmake error (Issue #30)

Closed #30https://github.com/halfgaar/FlashMQ/issues/30 as completed.

— Reply to this email directly, view it on GitHubhttps://github.com/halfgaar/FlashMQ/issues/30#event-8424979694, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU7WZ7L4HEG3VCA7WTN3BELWVQF7BANCNFSM6AAAAAAUMZ3UFI. You are receiving this because you authored the thread.Message ID: @.***>