search

FairRootGroup / FairMQ

C++ Message Queuing Library and Framework
GNU Lesser General Public License v3.0
87 stars 34 forks source link

Warn when going to try allocate a shmem message more times #456

Closed knopers8 closed 2 years ago

knopers8 commented 2 years ago

When FairMQ is configured to try allocate a shmem message indefinitely, we might end up in a situation when a device is stuck without any information why it could be so. This log would help us understand why a device is stuck.

rbx commented 2 years ago

As you can see three lines above, I already had a warning messages (although less detailed one, and on every iteration) for such cases. But there can be scenarios where it is valid to just wait and idle until eventually memory is available (or not). And there the warning is not useful.

So for your case I would rather let it throw the exception, catch it and add the warning there.

rbx commented 2 years ago

Would the debug severity instead of warning be sufficient for you?

knopers8 commented 2 years ago

But there can be scenarios where it is valid to just wait and idle until eventually memory is available (or not). And there the warning is not useful.

Yes, this is why we use it this way :) And by adding just one log, not for each try, we should avoid the spam.

Would the debug severity instead of warning be sufficient for you?

AFAICT, we disable debug messages from FairMQ in the production setup, so that would not help much unfortunately. I am not sure if it would make sense to enable FairMQ debug messages there just to be able to see this one.

If a warning is a "no" to you, then I will just go for the catch-throw approach. Not the cleanest to me, but should probably do.

rbx commented 2 years ago

AFAICT, we disable debug messages from FairMQ in the production setup, so that would not help much unfortunately. I am not sure if it would make sense to enable FairMQ debug messages there just to be able to see this one.

Mmm, right, there was that... Ok, let's leave it at the warning severity, at least if it is once per call it is not a spammy one (like the one I had there before).

knopers8 commented 2 years ago

Thanks! And sorry I did not realize you are using dev :)