search

anchore / anchore-engine

A service that analyzes docker images and scans for vulnerabilities
Apache License 2.0
1.58k stars 272 forks source link

Inconsistent Java package version detection #226

Open stewartadam opened 5 years ago

stewartadam commented 5 years ago

Is this a request for help?: No

Is this a BUG REPORT or a FEATURE REQUEST? (choose one): BUG REPORT

Version of Anchore Engine and Anchore CLI if applicable: anchore-cli, version 0.4.1

What happened: Detecting Java package dependency versions appears to be very inconsistent - while testing against images in the vulnhub/vulnhub repo, some JAR dependencies have undetected versions while others do, seemingly inconsistently:

anchore-cli image content docker.io/vulhub/jenkins:2.138 java
Package                                            Specification-Version        Implementation-Version        Location                                                                                                       
...
jce                                       1.8                          1.8.0_181                        /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.jar                                                                                                                       
jcifs                                     N/A                          N/A                              /usr/share/jenkins/jenkins.war:WEB-INF/detached-plugins/windows-slaves.hpi:WEB-INF/lib/jcifs-1.3.17-kohsuke-1.jar                                                       
jcifs                                     N/A                          N/A                              /usr/share/jenkins/jenkins.war:WEB-INF/lib/jcifs-1.3.17-kohsuke-1.jar                                                                                                   
jcl-over-slf4j                            N/A                          N/A                              /usr/share/jenkins/jenkins.war:WEB-INF/lib/jcl-over-slf4j-1.7.25.jar                                                                                                    
jcommon                                   N/A                          N/A                              /usr/share/jenkins/jenkins.war:WEB-INF/lib/jcommon-1.0.12.jar                                                                                                           
jdk-tool                                  N/A                          N/A                              /usr/share/jenkins/jenkins.war:WEB-INF/detached-plugins/jdk-tool.hpi                                                                                                    
jdk-tool                                  N/A                          N/A                              /usr/share/jenkins/jenkins.war:WEB-INF/detached-plugins/jdk-tool.hpi:WEB-INF/lib/jdk-tool.jar                                                                           
jenkins                                   N/A                          N/A                              /usr/share/jenkins/jenkins.war                                                                                                                                          
jenkins-cli                               N/A                          N/A                              /usr/share/jenkins/jenkins.war:WEB-INF/jenkins-cli.jar                                                                                                                  
jenkins-core                              N/A                          N/A                              /usr/share/jenkins/jenkins.war:WEB-INF/lib/jenkins-core-2.138.jar                                                                                                       
jenkins-war                               N/A                          N/A                              /usr/share/jenkins/jenkins.war:jenkins-war                                                                                                                              
jetty-alpn-server                         N/A                          N/A                              /usr/share/jenkins/jenkins.war:winstone.jar:jetty-alpn-server                                                                                                           
jetty-http                                N/A                          N/A                              /usr/share/jenkins/jenkins.war:winstone.jar:jetty-http                                                                                                                  
jetty-io                                  N/A                          N/A                              /usr/share/jenkins/jenkins.war:winstone.jar:jetty-io                                                                                                                    
jetty-jmx                                 N/A                          N/A                              /usr/share/jenkins/jenkins.war:winstone.jar:jetty-jmx                                                                                                                   
jetty-security                            N/A                          N/A                              /usr/share/jenkins/jenkins.war:winstone.jar:jetty-security                                                                                                              
jetty-server                              N/A                          N/A                              /usr/share/jenkins/jenkins.war:winstone.jar:jetty-server                                                                                                                
jetty-servlet                             N/A                          N/A                              /usr/share/jenkins/jenkins.war:winstone.jar:jetty-servlet                                                                                                               
jetty-util                                N/A                          N/A                              /usr/share/jenkins/jenkins.war:winstone.jar:jetty-util                                                                                                                  
jetty-webapp                              N/A                          N/A                              /usr/share/jenkins/jenkins.war:winstone.jar:jetty-webapp                                                                                                                
jetty-xml                                 N/A                          N/A                              /usr/share/jenkins/jenkins.war:winstone.jar:jetty-xml                                                                                                                   
jffi                                      N/A                          N/A                              /usr/share/jenkins/jenkins.war:WEB-INF/lib/jffi-1.2.15.jar                                                                                                              
...
thymeleaf-3.0.9.RELEASE                            3.0.9.RELEASE                3.0.9.RELEASE                 /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/thymeleaf-3.0.9.RELEASE.jar                            
thymeleaf-extras-java8time-3.0.1.RELEASE           3.0.1.RELEASE                3.0.1.RELEASE                 /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/thymeleaf-extras-java8time-3.0.1.RELEASE.jar           
thymeleaf-spring5-3.0.9.RELEASE                    3.0.9.RELEASE                3.0.9.RELEASE                 /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/thymeleaf-spring5-3.0.9.RELEASE.jar

Note how jenkins-core has no version - at first I thought nested archives were not handled, except was incorrect as thymeleaf is nested within the spring JAR and does get a version.

Another image:

anchore-cli image content docker.io/vulhub/spring-data-commons:2.0.5 java
Package                                            Specification-Version        Implementation-Version        Location                                                                                                       
HikariCP                                           N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/HikariCP-2.7.8.jar                                     
US_export_policy                                   N/A                          N/A                           /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/security/policy/limited/US_export_policy.jar                         
US_export_policy                                   N/A                          N/A                           /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/security/policy/unlimited/US_export_policy.jar                       
antlr-2.7.7                                        N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/antlr-2.7.7.jar                                        
aspectjweaver-1.8.13                               1.8.13                       1.8.13                        /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/aspectjweaver-1.8.13.jar                               
attoparser-2.0.4.RELEASE                           2.0.4.RELEASE                2.0.4.RELEASE                 /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/attoparser-2.0.4.RELEASE.jar                           
ca-certificates-java                               N/A                          N/A                           /usr/share/ca-certificates-java/ca-certificates-java.jar                                                       
charsets                                           N/A                          N/A                           /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/charsets.jar                                                         
classmate                                          1.3.4                        1.3.4                         /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/classmate-1.3.4.jar                                    
...
hibernate-core-5.2.14.Final                        5.2.14.Final                 5.2.14.Final                  /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/hibernate-core-5.2.14.Final.jar                        
hibernate-jpa-2.1-api-1.0.0.Final                  2.1                          1.0.0.Final                   /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/hibernate-jpa-2.1-api-1.0.0.Final.jar                  
hibernate-validator                                N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/hibernate-validator-6.0.7.Final.jar                    
hsqldb-2.4.0                                       2.4.0                        2.4.0                         /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/hsqldb-2.4.0.jar                                       
...
jboss-logging                                      3.3.2.Final                  3.3.2.Final                   /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/jboss-logging-3.3.2.Final.jar                          
jce                                                1.8                          1.8.0_162                     /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.jar                                                              
jsse                                               1.8                          1.8.0_162                     /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jsse.jar
...
spring-aop-5.0.4.RELEASE                           N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-aop-5.0.4.RELEASE.jar                           
spring-aspects-5.0.4.RELEASE                       N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-aspects-5.0.4.RELEASE.jar                       
spring-beans-5.0.4.RELEASE                         N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-beans-5.0.4.RELEASE.jar                         
spring-boot-2.0.0.RELEASE                          N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-boot-2.0.0.RELEASE.jar                          
spring-boot-autoconfigure-2.0.0.RELEASE            N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-boot-autoconfigure-2.0.0.RELEASE.jar            
spring-boot-starter-2.0.0.RELEASE                  N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-boot-starter-2.0.0.RELEASE.jar                  
spring-boot-starter-aop-2.0.0.RELEASE              N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-boot-starter-aop-2.0.0.RELEASE.jar              
spring-boot-starter-data-jpa-2.0.0.RELEASE         N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-boot-starter-data-jpa-2.0.0.RELEASE.jar         
spring-boot-starter-jdbc-2.0.0.RELEASE             N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-boot-starter-jdbc-2.0.0.RELEASE.jar             
spring-boot-starter-json-2.0.0.RELEASE             N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-boot-starter-json-2.0.0.RELEASE.jar             
spring-boot-starter-logging-2.0.0.RELEASE          N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-boot-starter-logging-2.0.0.RELEASE.jar          
spring-boot-starter-security-2.0.0.RELEASE         N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-boot-starter-security-2.0.0.RELEASE.jar         
spring-boot-starter-thymeleaf-2.0.0.RELEASE        N/A                          N/A                           /spring-data-web-example-2.0.0.RELEASE.jar:BOOT-INF/lib/spring-boot-starter-thymeleaf-2.0.0.RELEASE.jar
...

Note how jce from the jdk library has no version in its PATH but still gets its version detected, while others like charsets do not. Similarly, attoparser-2.0.4.RELEASE has its version detected but spring-boot-2.0.0.RELEASE does not.

Manually extracting these JARs shows they do have a manifest with version information attached.

What did you expect to happen: All JAR version were detected correctly, demonstrating known vulnerabilities (in this case, for Jenkins or Spring)

Any relevant log output from /var/log/anchore: Nothing that seems out of the ordinary:

[service:worker] 2019-07-03 22:37:46+0000 [-] twistd 18.9.0 (/opt/rh/rh-python36/root/usr/bin/python3 3.6.3) starting up.
[service:worker] 2019-07-03 22:37:46+0000 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
[service:worker] 2019-07-03 22:37:46+0000 [-] Site starting on 8228
[service:worker] 2019-07-03 22:37:46+0000 [-] Starting factory <twisted.web.server.Site object at 0x7f86d47d9f60>
[service:worker] 2019-07-03 22:38:10+0000 [-] "127.0.0.1" - - [03/Jul/2019:22:38:10 +0000] "GET /health HTTP/1.1" 200 - "-" "curl/7.29.0"
[service:worker] 2019-07-03 22:38:40+0000 [-] "127.0.0.1" - - [03/Jul/2019:22:38:40 +0000] "GET /health HTTP/1.1" 200 - "-" "curl/7.29.0"
[service:worker] 2019-07-03 22:39:10+0000 [-] "127.0.0.1" - - [03/Jul/2019:22:39:10 +0000] "GET /health HTTP/1.1" 200 - "-" "curl/7.29.0"
[service:worker] 2019-07-03 22:39:41+0000 [-] "127.0.0.1" - - [03/Jul/2019:22:39:40 +0000] "GET /health HTTP/1.1" 200 - "-" "curl/7.29.0"
[service:worker] 2019-07-03 22:40:11+0000 [-] "127.0.0.1" - - [03/Jul/2019:22:40:10 +0000] "GET /health HTTP/1.1" 200 - "-" "curl/7.29.0"
[service:worker] 2019-07-03 22:40:41+0000 [-] "127.0.0.1" - - [03/Jul/2019:22:40:40 +0000] "GET /health HTTP/1.1" 200 - "-" "curl/7.29.0"
[service:worker] 2019-07-03 22:40:47+0000 [-] [Thread-186] [anchore_engine.services.analyzer/process_analyzer_job()] [INFO] image dequeued for analysis: admin : sha256:cf842a4d075d1cef8b9ba98389000ff829c03f0a233cf2c4407afc21fea80e79
[service:worker] 2019-07-03 22:40:47+0000 [-] [Thread-186] [anchore_engine.services.analyzer/perform_analyze_nodocker()] [INFO] performing analysis on image: ['admin', 'docker.io/vulhub/spring-data-commons@sha256:cf842a4d075d1cef8b9ba98389000ff829c03f0a233cf2c4407afc21fea80e79', 'docker.io/vulhub/spring-data-commons:2.0.5']
[service:worker] 2019-07-03 22:40:47+0000 [-] [Thread-186] [anchore_engine.clients.localanchore_standalone/get_anchorelock()] [INFO] copying new config into place: /anchore_service/analyzer_config.yaml -> /home/anchore/.anchore/conf/analyzer_config.yaml
[service:worker] 2019-07-03 22:41:11+0000 [-] "127.0.0.1" - - [03/Jul/2019:22:41:10 +0000] "GET /health HTTP/1.1" 200 - "-" "curl/7.29.0"
[service:worker] 2019-07-03 22:41:41+0000 [-] "127.0.0.1" - - [03/Jul/2019:22:41:40 +0000] "GET /health HTTP/1.1" 200 - "-" "curl/7.29.0"
[service:worker] 2019-07-03 22:42:11+0000 [-] "127.0.0.1" - - [03/Jul/2019:22:42:10 +0000] "GET /health HTTP/1.1" 200 - "-" "curl/7.29.0"
[service:worker] 2019-07-03 22:42:40+0000 [-] [Thread-186] [anchore_engine.services.analyzer/perform_analyze_nodocker()] [INFO] performing analysis on image complete: docker.io/vulhub/spring-data-commons@sha256:cf842a4d075d1cef8b9ba98389000ff829c03f0a233cf2c4407afc21fea80e79
[service:worker] 2019-07-03 22:42:41+0000 [-] "127.0.0.1" - - [03/Jul/2019:22:42:40 +0000] "GET /health HTTP/1.1" 200 - "-" "curl/7.29.0"

What docker images are you using:

How to reproduce the issue:


anchore-cli image add docker.io/vulhub/spring-data-commons:2.0.5
anchore-cli image wait docker.io/vulhub/spring-data-commons:2.0.5
anchore-cli image content docker.io/vulhub/spring-data-commons:2.0.5 java

anchore-cli image add docker.io/vulhub/jenkins:2.138
anchore-cli image wait docker.io/vulhub/jenkins:2.138
anchore-cli image content docker.io/vulhub/jenkins:2.138 java
zhill commented 5 years ago

@stewartadam , thanks for the detailed write-up, we'll take a look. This is very helpful to reproduce and triage.

stewartadam commented 5 years ago

Hi @zhill, any updates on this? Thx