Logging not working on production

[maqeelqureshi]

I need to shift my application logs from file to elk. Logging on elk is working fine on my local environment (without credentials) but its not working on production.

On production, if i just switch appender to RollingFileAppender text file logging works.

Following is the output of internal debugging.

`log4net: log4net assembly [log4net, Version=2.0.8.0, Culture=neutral, PublicKeyToken=669e0ddf0bb1aa2a]. Loaded from [C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files\root\38ce21d9\71672efd\assembly\dl3\3a61141d\00cb2cb9_2898d201\log4net.dll]. (.NET Runtime [4.0.30319.42000] on Microsoft Windows NT 10.0.17763.0) log4net: defaultRepositoryType [log4net.Repository.Hierarchy.Hierarchy] log4net: Creating repository for assembly [MyApp.API, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] log4net: Assembly [MyApp.API, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] Loaded From [C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files\root\38ce21d9\71672efd\assembly\dl3\836bf9fe\56a04992_59c9d501\MyApp.API.dll] log4net: Assembly [MyApp.API, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] does not have a RepositoryAttribute specified. log4net: Assembly [MyApp.API, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] using repository [log4net-default-repository] and repository type [log4net.Repository.Hierarchy.Hierarchy] log4net: Creating repository [log4net-default-repository] using type [log4net.Repository.Hierarchy.Hierarchy] log4net: configuring repository [log4net-default-repository] using file [C:\Deployment\Feature\web.config] watching for file updates log4net: configuring repository [log4net-default-repository] using file [C:\Deployment\Feature\web.config] log4net: configuring repository [log4net-default-repository] using stream log4net: loading XML configuration log4net: Configuring Repository [log4net-default-repository] log4net: Configuration update mode [Merge]. log4net: Logger [root] Level string is [ALL]. log4net: Logger [root] level set to [name="ALL",value=-2147483648]. log4net: Loading Appender [EventsLog1] type: [log4net.ElasticSearch.ElasticSearchAppender,log4net.ElasticSearch] log4net: Setting Property [ConnectionString] to String value [Scheme=http;Server=[ip];Index=[index name];Port=[port];rolling=true;User=[user];Pwd=[pwd]] log4net: Setting Property [Lossy] to Boolean value [False] log4net: Setting Property [BufferSize] to Int32 value [10] log4net: Setting Property [Fix] to FixFlags value [None] log4net: Created Appender [EventsLog1] log4net: Adding appender named [EventsLog1] to logger [root]. log4net: Hierarchy Threshold []

log4net: configuring repository [log4net-default-repository] using .config file section log4net: Application config file is [C:\Deployment\Feature\web.config] log4net: Configuring Repository [log4net-default-repository] log4net: Configuration update mode [Merge]. log4net: Logger [root] Level string is [ALL]. log4net: Logger [root] level set to [name="ALL",value=-2147483648]. log4net: Loading Appender [EventsLog1] type: [log4net.ElasticSearch.ElasticSearchAppender,log4net.ElasticSearch] log4net: Setting Property [ConnectionString] to String value [Scheme=http;Server=[ip];Index=[index name];Port=[port];rolling=true;User=[user];Pwd=[pwd]] log4net: Setting Property [Lossy] to Boolean value [False] log4net: Setting Property [BufferSize] to Int32 value [10] log4net: Setting Property [Fix] to FixFlags value [None] log4net: Created Appender [EventsLog1] log4net: Adding appender named [EventsLog1] to logger [root]. log4net: Hierarchy Threshold []

log4net: Creating repository for assembly [MyApp.Utils, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] log4net: Assembly [MyApp.Utils, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] Loaded From [C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files\root\38ce21d9\71672efd\assembly\dl3\7cea7f47\847a6e5b_58c9d501\MyApp.Utils.dll] log4net: Assembly [MyApp.Utils, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] does not have a RepositoryAttribute specified. log4net: Assembly [MyApp.Utils, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] using repository [log4net-default-repository] and repository type [log4net.Repository.Hierarchy.Hierarchy] log4net: repository [log4net-default-repository] already exists, using repository type [log4net.Repository.Hierarchy.Hierarchy] log4net: ConfigureAndWatchHandler: Deleted [C:\Deployment\Feature\web.config] log4net: Shutdown called on Hierarchy [log4net-default-repository] `

Following is the configuration

<!--<root>
  <level value="ALL"/>
  <appender-ref ref="EventsLog"/>
</root>
<appender name="EventsLog" type="log4net.Appender.RollingFileAppender">
  <file type="log4net.Util.PatternString" value="C:\Log\Feature\EventsLog.xml"/>
  <staticLogFileName value="false"/>
  <preserveLogFileNameExtension value="true"/>
  <appendToFile value="true"/>
  <rollingStyle value="Date"/>
  <datePattern value="_yyyyMMdd"/>
  <layout type="MyApp.Utils.Log.XMLLayout">
    <header>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;LogEntries&gt;</header>
    <footer>&lt;/LogEntries&gt;</footer>
  </layout>
</appender>-->

Note : Browsing to ELK or curl to elasticsearch works fine from production server.

Update: I have also tried setup local elasticsearch docker instance with credentials. Logging works fine on that also. I was thinking that there might be some problem with connection string on production.

Update: When I change the logger name and user/pwd of elasticsearch, sometimes it works. I have no clue what is happening.

[bruno-garcia]

Often production server require a proxy to access the Internet. With .NET Framework you can set it up via app.config or web.config (or even machine.config). Other than that it's hard to help troubleshoot these issues since it's probably some configuration in your prod environment.

Lastly the sometimes it works is a bit more worrying. Can you reproduce that also with an instance of elastic on docker? Some steps to reproduce would help

[Abouzeid]

I faced a similar issue today. I'm posting here, as it might save someone's time (and sanity).

ElasticSearch: 7.13.2 log4net.Elasticsearch: 2.5.0 Net framework: 4.6.2 Schema: HTTP

Elasticsearch server returned the "401 unauthorized " in the GetResponse method https://github.com/bruno-garcia/log4net.ElasticSearch/blob/d5f2b884d54593a24a8fb01028de8787f1272244/src/log4net.ElasticSearch/Infrastructure/HttpClient.cs#L43

However, the exception doesn't show up in any log.

The main reason

The user, whose credential specified in the connection string, didn't have the correct index privilege permission. See ElasticSearch docs here Also, don't include any special characters in the password.

Tip: at first, give the user the superuser role, just to see the data on ElasticSearch. After that, narrow the permission down as you want. (try first with no special char in the password)

[danammeansbear]

I am having a similar issue. I am able to send logs via postman and https but not through the appender.

`

region working code from postman

//var client = new HttpClient(handler); //var request = new HttpRequestMessage(HttpMethod.Post, "https://server.com:9200/elk_test/_doc"); //request.Headers.Add("Authorization", "Basic ZWxhc"); //var content = new StringContent("{\r\n \"@timestamp\": \"2024-07-08T13:42:02.1570610Z\",\r\n \"LoggerName\": \"EventLogger.Logger\",\r\n \"HostName\": \"adabdoub-LT2\",\r\n \"ThreadName\": \"\",\r\n \"AppDomain\": \"DataLoaderService.exe\",\r\n \"Level\": \"ALL\",\r\n \"Message\": \"EsAlertTest 1\",\r\n \"log4net_HostName\": \"adabdoubS-LT2\",\r\n \"log4net_UserName\": \"CTI\\adabdoub\",\r\n \"Environment\": \"DEV\",\r\n \"EventName\": \"Elklogest\",\r\n \"EventID\": \"413fe71e-153c-407a-9727-14342068e5ef\"\r\n}", null, "application/json"); //request.Content = content; //var response = await client.SendAsync(request); //response.EnsureSuccessStatusCode(); //Console.WriteLine(await response.Content.ReadAsStringAsync());

endregion working code from postman

region Revised Code

// Ensure log4net is configured var logRepository = LogManager.GetRepository(Assembly.GetEntryAssembly());

// Reflectively modify the HttpClientHandler of the ElasticSearchAppender var elasticSearchAppender = LogManager.GetRepository(logRepository.Name).GetAppenders().OfType().FirstOrDefault(); if (elasticSearchAppender != null) { var clientField = elasticSearchAppender.GetType().BaseType.GetField("_client", BindingFlags.NonPublic | BindingFlags.Instance); if (clientField != null) { var customHandler = new HttpClientHandler { ServerCertificateCustomValidationCallback = (HttpRequestMessage, X509Certificate2, X509Chain, SslPolicyErrors) => true }; var customClient = new HttpClient(customHandler); clientField.SetValue(elasticSearchAppender, customClient); }

 elasticSearchAppender.ActivateOptions();

}

// Get logger var logger = LogManager.GetLogger(typeof(Program));

// Log messages logger.Debug("This is a DEBUG message."); logger.Info("This is an INFO message."); logger.Warn("This is a WARN message."); logger.Error("This is an ERROR message."); logger.Error("kaboom!", new ApplicationException("An error occurred")); Console.WriteLine("Log messages have been sent to Elasticsearch....maybe....");

endregion Revised Code

`