Deploying spring boot application to AWS using elasticbeanstalk; logback configuration error "There is no conversion class registered for conversion word [X]"

[dukeli-ehi]

I recently started seeing an issue with AWS deploying my spring boot application via elastic beanstalk. My existing application was running and deployed successfully, but when I increased the ASG (auto scaling group) to a higher capacity the newly spun up instance fails with the below error. All successfully deployed and running instances fails to start after bouncing them with zero java application code changes and AWS environment changes. This feels like an issue with the AWS public AMIs, but after working with and AWS support engineer we were not able to identify the root cause even after downgrading to previous AMIs that have worked in the past with the exact same code base.

2022-11-08T22:11:24.241Z app="location-api"|ERROR|org.springframework.boot.SpringApplication|main|Application run failed
java.lang.IllegalStateException: Logback configuration error detected: 
ERROR in ch.qos.logback.core.pattern.parser.Compiler@725bef66 - There is no conversion class registered for conversion word [X]
ERROR in ch.qos.logback.core.pattern.parser.Compiler@725bef66 - [X] is not a valid conversion word
ERROR in ch.qos.logback.core.pattern.parser.Compiler@725bef66 - There is no conversion class registered for conversion word [X]

I previously was using the MDC "%X" qualifier to pull the MDC set values on a per thread basis. Even after removing that form my logback configuration I still receive the error stated above. I am currently using logback-core version 1.2.9. Below is two code snippets of the logback configuration that I receive the same error for. One without the use of the MDC "%X" qualifier and one with the use of the MDC "%X" qualifier.

Without MDC "%X":

<appender class="ch.qos.logback.core.FileAppender" name="DEFAULT_APPENDER">
    <append>true</append>
    <file>${LOG_FILE_LOCATION}/locationapi.log</file>
    <encoder>
      <pattern>%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX} app="location-api"|%p|%C|%t|%m%n</pattern>
      <charset>utf8</charset>
    </encoder>
  </appender>

With MDC "%X":

<appender class="ch.qos.logback.core.FileAppender" name="DEFAULT_APPENDER">
    <append>true</append>
    <file>${LOG_FILE_LOCATION}/locationapi.log</file>
    <encoder>
      <pattern>%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX} app="location-api", ehi-calling-application="%X{ehi-calling-application:-daemonThread}", jwt-subject="%X{jwt-subject:-daemonThread}", uri="%X{uri:-daemonThread}", http-method="%X{http-method:-daemonThread}", x-b3-traceid="%X{x-b3-traceid:-daemonThread}", x-b3-spanid="%X{x-b3-spanid:-daemonThread}"|%p|%C|%t|%m%n</pattern>
      <charset>utf8</charset>
    </encoder>
  </appender>

NOTE: I have not seen this error anywhere else on the web. In most cases with the conversion class not being registered is referring to the use of springs defaults.xml file which, from what I've read in the documentation, does not contain any definition for [X]. Again this has been working for months now, but all of a sudden stopped working with zero code or environment changes.

Thanks!

[wilkinsona]

Unfortunately, I don't think we're going to be able to help with this one. The converter for X is registered by Logback as part of its defaults. For there to be no conversion class registered, either the registration of the defaults has failed or something must have removed this entry for X from the map. The former is out of Spring Boot's control and there's no code in Spring Boot that does the latter.

All successfully deployed and running instances fails to start after bouncing them with zero java application code changes and AWS environment changes.

To me, this strongly suggests that the problem is in some way environmental. Good luck with figuring it out.