search

ArieGato / serilog-sinks-rabbitmq

Serilog Sink for RabbitMq
Apache License 2.0
53 stars 52 forks source link

Add custom properties to message #211

Open KlishevichIlya opened 4 days ago

KlishevichIlya commented 4 days ago

Hi there!

I use RabbitMq.Sinks and I need to set custom properties to each messages. Does any functionality exist for that purpose? Is there only one way to fork your repo and add that functionality by myself? It needs becaose my consumer client API for rabbitMq requires type property for each message.

ArieGato commented 4 days ago

@KlishevichIlya

Could you explain a little bit more what you're trying to achieve?

KlishevichIlya commented 4 days ago

Sure!

I use Serilog to send JSON messages to RabbitMq queue. It works properly and messages apear in queue. On consumer side I use EasyNetQ and try to read that messages. During read process it throws and exception that EasyNetQ expects property typeName. (Exception {"typeName must not be blank\r\nParameter name: typeName"}). In RabbitMq UI I noticed that each message in queue contains Exchange, RoutingKey, Properties and Payload. As I understand I need to set type of message into typeName property. Custom properties are metadata for each message in specific queue. Is there any way to make it? I thought that there is some API functionality for adding custom property(in my case it's typeName) for messages in the queue.

ArieGato commented 3 days ago

If this works simular to MassTransit the message type is used to deserialize the json to a .net class. Do you need type to be same for all messages?

ps. Is there a specific reason why you want to use ENQ? As far as I know ENQ is designed to publish/subscribe to shared c# classes.

KlishevichIlya commented 3 days ago

@ArieGato yes, it uses for deserialize the json to .net class. I need the same type for all messages. It is used in legacy code, so I am not able to get rid of it.

ArieGato commented 3 days ago

I'll have a look at it. I don't know if it can be added in a clean way. Can you confirm that the type has to be added to the BasicProperties?

_model.BasicPublish(address, _properties, body);

KlishevichIlya commented 3 days ago

yes, you are right

KlishevichIlya commented 3 days ago

@ArieGato could you give any clue for adding custom properties to each message? I forked your repo and try to add it into my project. Is there any way to get BasicProperties?

ArieGato commented 3 days ago

I was think along the way of the RoutingKeyFunc. And I'm not sure I want to expose the basic properties. I'll investigate when I have some time.

ArieGato commented 2 days ago

@KlishevichIlya I had a good look whether we can implement it. It would mean quite some changes for something that will probably not be used very much. For the purpose of logging, I don't see the value of being able to add message properties.

My advice is to create a simple native RabbitMQ consumer. And maybe host it in a seperate (micro)service.

Do you really need the logging data in the legacy software?

KlishevichIlya commented 2 days ago

@ArieGato Good evening! I made some changes in your library and right now it accepts custom properties. I set it to sinkConfiguration. Also right now I encountered with an issue with dependent DLLs. After build it doesn’t appear in bin folder and when my project try to use your updated library it couldn’t find required library. Do you have any idea why dependent DLLs don’t appear into bin folder?

KlishevichIlya commented 2 days ago

Also I am not able to left legacy code without logs and need somehow resolve it. Hope that my changes could provide required functionality for custom properties. If it doesn’t need to your library at least it could help me :)

ArieGato commented 2 days ago

@ArieGato Good evening! I made some changes in your library and right now it accepts custom properties. I set it to sinkConfiguration. Also right now I encountered with an issue with dependent DLLs. After build it doesn’t appear in bin folder and when my project try to use your updated library it couldn’t find required library. Do you have any idea why dependent DLLs don’t appear into bin folder?

That's too little information for me to help you out. What solution are you building? The forked repo, or your code base?

ArieGato commented 2 days ago

Also I am not able to left legacy code without logs and need somehow resolve it. Hope that my changes could provide required functionality for custom properties. If it doesn’t need to your library at least it could help me :)

Where are you consuming the messages that are published through serilog? Couldn't you create a small microservice that processes these messages? That way you can log in the legacy code without any changes.

KlishevichIlya commented 2 days ago

Yes, I forked your repo and made my changes. I could create PR tomorrow and you could review it. Also I didn’t make a beautiful code yet and you could review only idea and maybe you could find out why after build process dependent DLLs doesn’t appear in bin folder. I couldn’t unfortunately shift it to microservice, because it’s a part of huge monolith. Later, we have plan to divide to micro services, but not now.

KlishevichIlya commented 2 days ago

@ArieGato could you please check my repo? I added functionality for setting custom properties in sinkConfig. After build there aren't any dependent libraries in bin folder for some reason. Why does it happen?

KlishevichIlya commented 2 days ago

I already resolved that issue. I needed to wrap dll to nuget and then connect that nuget to my project. As I understood, only nuget contains metadata about dependent libraries.

ArieGato commented 1 day ago

@ArieGato could you please check my repo? I added functionality for setting custom properties in sinkConfig. After build there aren't any dependent libraries in bin folder for some reason. Why does it happen?

I had a look and I'm wondering whether the dictionary will be configurable through appsettings.json or Web.Config?

As I said before, I don't want to add this functionality to the project. I made a simular change to evaluate: https://github.com/ArieGato/serilog-sinks-rabbitmq/tree/feature/211-add-basic-properties

ArieGato commented 1 day ago

Yes, I forked your repo and made my changes. I could create PR tomorrow and you could review it. Also I didn’t make a beautiful code yet and you could review only idea and maybe you could find out why after build process dependent DLLs doesn’t appear in bin folder. I couldn’t unfortunately shift it to microservice, because it’s a part of huge monolith. Later, we have plan to divide to micro services, but not now.

I'm not sure you understand what I'm suggesting. Could you elaborate on the architecture?

As I understand you have:

  1. Legacy app logging through Serilog (to RabbitMQ)
  2. Another legacy application consuming the messages with ENQ. The log messages are written to database/file?

What I'm suggesting is to create a new application that solely consumes RabbitMQ messages with the RabbitMQ.Client package.

KlishevichIlya commented 1 day ago

Hi @ArieGato, I extended your library and right now everything work smoothly with EasyNetQ. I opened a PR, please review it if you have time. Hope that my little changes could prevent any future issues with EasyNetQ for others.