How disable de-duplicated.

[wu-yafeng]

I have a AspNetCore Web configured with exceptionless logging by following code:

public void Configure(IServiceCollection services)
{
    services.AddLogging(b=>b.AddExceptionless(...));
}

I notice that a message were sent to exceptionless and it shows like:

but here a problem is that log message was deleted after a few seconds. I thought the reason is due to https://exceptionless.com/docs/deduplication/.

This is how I send log message to exceptionless:

_logger.LogInformation("Start processing request with id:{0}",id);

[niemyjski]

What do you mean by deleted? The server shouldn't delete any events and the client is the one that deduplicates messages. If the id's are not unique the plugin would remove it. Is this is what you are seeing?

You can remove the deduplication client by calling:

using Exceptionless;
ExceptionlessClient.Default.Configuration.RemovePlugin<Exceptionless.Plugins.Default.DuplicateCheckerPlugin>();

Please note this will only work if you are configuring Exceptionless's default client. If you are passing an API Key to AddExceptionless that will use an internal instance and the plugin will not be removed. It would be helpful to turn on the client logger as described in the troubleshooting docs to see what it's doing before removing this plugin.

[wu-yafeng]

Thanks for quick respond.

but here a problem is that log message was deleted after a few seconds

It means I'm in following case:

1. start my aspnet core application and send log to exceptionless server.
2. Access my exceptionless dashboard 'http://myip'
3. The new log shows
4. waiting for few senconds and press F5 to refresh the page

Why?

[niemyjski]

I'm not sure, you should check elasticsearch to see if the event is still there.

[wu-yafeng]

I checked the elasticsearch and no events found.

[wu-yafeng]

here are many logs about remove bot events on exceptionless.web. What is RemoveBotEvents

[14:04:56 INF] Received remove bot events work item OrganizationId=61c2d52349736816e8741004 ProjectId=61c2d52449736816e8741005, ClientIpAddress=::ffff:10.0.1.22, UtcStartDate=01/04/2022 06:00:00, UtcEndDate=01/04/2022 06:05:00
[14:04:56 INF] RemoveBotEventsWorkItem Progress 0%: Starting deleting of bot events... OrganizationId=61c2d52349736816e8741004
[14:04:56 INF] RemoveBotEventsWorkItem Progress 100%: Bot events deleted: 0 OrganizationId=61c2d52349736816e8741004
[14:04:56 INF] Auto completed RemoveBotEventsWorkItem work item queue entry: 2b854a53802544428b4bfb1a70744a5a.
[14:04:58 INF] Processing queue entry: id=e95cd6726d0c4dbaa25bc39ea22fec49 type=EventPost attempt=1
[14:04:58 INF] Processing post: id=e95cd6726d0c4dbaa25bc39ea22fec49 path=q\ca9\ca90ff4775174bd5a03e1d682774cf58.payload project=61c2d52449736816e8741005 ip=121.199.62.110 v=2 agent=exceptionless/4.6.2.0
[14:04:58 INF] Bot throttle triggered. IP: 112.64.52.122 Time: 01/04/2022 06:00:00 Project: 61c2d52449736816e8741005
[14:04:58 INF] Processing RemoveBotEventsWorkItem work item queue entry: 534c2315b2b444808342af76011339d1.
[14:04:58 INF] Received remove bot events work item OrganizationId=61c2d52349736816e8741004 ProjectId=61c2d52449736816e8741005, ClientIpAddress=112.64.52.122, UtcStartDate=01/04/2022 06:00:00, UtcEndDate=01/04/2022 06:05:00

[wu-yafeng]

This is due to ‘BotThrottleLimit’ when too many logs send to exceptionless. Throttle errors by client ip address to no more than 25 every 5 minutes by defaults. Configure BotThrottleLimit in appsettings.Production.yml such as:

BotThrottleLimit: 1000

[niemyjski]

You can also disable this behavior by going into project settings and unchecking the bot option! This setting is there to help reduce noise from bots / security scanners.