search

vespa-engine / vespa

AI + Data, online. https://vespa.ai
https://vespa.ai
Apache License 2.0
5.8k stars 604 forks source link

Vespa configserver throws an IllegalStateException on startup #7913

Closed paolodedios closed 5 years ago

paolodedios commented 5 years ago

Vespa 6.319.4 open source RPM builds throw an IllegalStateException on startup. Version 6.318.15 and earlier do not exhibit this behavior.

[2018-12-09 21:25:24.663] INFO    : jdisc/configserver flags    BundleEvent STARTED
[2018-12-09 21:25:24.663] INFO    : jdisc/configserver service-monitor  BundleEvent STARTED
[2018-12-09 21:25:24.664] INFO    : jdisc/configserver org.apache.felix.framework       FrameworkEvent PACKAGES REFRESHED
[2018-12-09 21:25:25.075] WARNING : configserver     Container.com.yahoo.container.di.Container Failed to set up first component graph\nexception=\njava.lang.RuntimeException: When resolving dependencies of 'com.yahoo.vespa.service.monitor.internal.ServiceMonitorImpl'\nCaused by: java.lang.IllegalStateException: No global component of class com.yahoo.vespa.service.monitor.application.DuperModel to inject into component 'com.yahoo.vespa.service.monitor.internal.ServiceMonitorImpl'.\n
[2018-12-09 21:25:25.076] WARNING : configserver     Container.com.yahoo.jdisc.core.ApplicationLoader   Exception thrown while activating application.\nexception=\njava.lang.RuntimeException: When resolving dependencies of 'com.yahoo.vespa.service.monitor.internal.ServiceMonitorImpl'\nCaused by: java.lang.IllegalStateException: No global component of class com.yahoo.vespa.service.monitor.application.DuperModel to inject into component 'com.yahoo.vespa.service.monitor.internal.ServiceMonitorImpl'.\n
[2018-12-09 21:25:25.769] WARNING : configserver     stderr     java.lang.RuntimeException: When resolving dependencies of 'com.yahoo.vespa.service.monitor.internal.ServiceMonitorImpl'
[2018-12-09 21:25:25.076] ERROR   : configserver     Container.com.yahoo.jdisc.core.StandaloneMain      JDisc exiting: Throwable caught: \nexception=\njava.lang.RuntimeException: When resolving dependencies of 'com.yahoo.vespa.service.monitor.internal.ServiceMonitorImpl'\nCaused by: java.lang.IllegalStateException: No global component of class com.yahoo.vespa.service.monitor.application.DuperModel to inject into component 'com.yahoo.vespa.service.monitor.internal.ServiceMonitorImpl'.\n
[2018-12-09 21:25:25.770] WARNING : configserver     stderr     Caused by: java.lang.IllegalStateException: No global component of class com.yahoo.vespa.service.monitor.application.DuperModel to inject into component 'com.yahoo.vespa.service.monitor.internal.ServiceMonitorImpl'.
[2018-12-09 21:25:25.201] INFO    : configserver     runserver  will restart in 26 seconds
hakonhall commented 5 years ago

This would be caused by a mismatch between a DuperModel reference in VESPA_HOME/conf/configserver-app/services.xml vs the location of the DuperModel.class in the VESPA_HOME/conf/configserver-app/components/service-monitor.jar bundle. My initial sanity-check on this looked fine, so the next step for me will be to reproduce your case by install RPM etc.

hakonhall commented 5 years ago

The RPM also looks good - how did you start the config server? What's the output of grep DuperModel /opt/vespa/conf/configserver-app/services.xml and md5sum /opt/vespa/conf/configserver-app/components/service-monitor.jar ?

paolodedios commented 5 years ago

The grep command did not return a result.

$grep DuperModel conf/configserver-app/services.xml

Here is the output of md5sum.

$ md5sum /opt/vespa/conf/configserver-app/components/service-monitor.jar
df5b50a006361e37cdd627e1f06b597d  /opt/vespa/conf/configserver-app/components/service-monitor.jar
hakonhall commented 5 years ago

After installation of the 6.319.4 RPM the DuperModel is defined as:

# grep DuperModel /opt/vespa/conf/configserver-app/services.xml 
    <component id="com.yahoo.vespa.service.monitor.application.DuperModel" bundle="service-monitor" />

So somehow your services.xml is not the one from 6.319.4.

paolodedios commented 5 years ago

Thanks for the tip. Turns out my copy of /opt/vespa/conf/configserver-app/services.xml is a couple of versions behind 6.319.4. After updating the file with the DuperModel, the configserver starts normally now.