Open gtt116 opened 8 years ago
william-richard
commented
8 years ago Hi @gtt116 - can you provide your solr configuration? We made some change to the way solr is configured in this version, and it looks like that is where your startup is failing. Take a look at http://tumblr.github.io/collins/configuration.html#solr for collins 2.0.0 docs abotu how to configure solr.
yl3w
commented
8 years ago It's been a while (many months) since I've looked at the code. So, take this fwiw, I'm fairly certain it isn't solr configuration changes. I'm answering also strictly based on the exception.
features.syslogAsset is the asset on which collins writes its own logs, Tattler.scala iirc is the file name for the code. multicollins.thisInstance is equivalent asset for cases where you are running collins at multiple locations, on this I'm unclear because I've no experience doing that.
If you are running a single instance, just make sure features.syslogAsset is defined correctly.
byxorna
commented
8 years ago Remember, the exception handling for syntax errors in the configuration file isnt fantastic, resulting in these red herring style errors.
@gtt116 my guess would be a syntax error in your configuration file. Can you sanitize and post?
yl3w
commented
8 years ago True, it might actually be trying to write a log for an incorrect config from elsewhere and fails because the config didn't load correctly, and I see now, @primer42 saw collins.solr.SolrConfig in the stack trace, so he may be right.
gtt116
commented
8 years ago Thank you guys for these information.
@Primer42 @maddalab @byxorna The config file conf/production.conflook like below:
include "validations.conf"
collins_conf_dir = "/Users/bmatheny/Downloads/collins/conf"
# Secret key
# ~~~~~
# The secret key is used to secure cryptographics functions.
# If you deploy your application to several instances be sure to use the same
# key!
application.secret="AbFgHx0eJx8lalkja812389uasdlkajsdlka98012398uasdlkasdklajsd81298"
parsers.text.maxLength=1048576
# Do not change the crypto key after you have started using it, you will be
# unable to decrypt data
crypto.key="lkkajsdlkajsdlkajsdlkajsdlkajsdlkajsdlkajsdlkajsdlkajsdlkajsdla"
# evolutionplugin should be disabled in a production environment
evolutionplugin=disabled
authentication {
permissionsFile = ${collins_conf_dir}"/permissions.yaml"
}
solr {
useEmbeddedServer = true
embeddedSolrHome = ${collins_conf_dir}"/solr/"
}
db.collins.logStatements=false
db.collins.password="collins_db_password"
db.collins.driver="com.mysql.jdbc.Driver"
db.collins.url="jdbc:mysql://localhost/collins?autoReconnect=true&interactiveClient=true"
db.collins.user="collins_db_uname"Oh, I think I found a hard coding collins_conf_dir here. I think this config item should be automatic modification by scripts/setup ?
But, I changed collins_conf_dir to "/usr/local/collins/current/conf". And changed root logger level to DEBUG in etc/logger.xml I got some thing new:
2016-09-23 16:18:12,802 - [INFO] - from collins.util.config.Registry$ in main Registered tagdecorators
2016-09-23 16:18:12,805 - [INFO] - from collins.util.config.Registry$ in main Initializing collins.intake.IntakeConfig
...skipping...
at play.core.server.NettyServer$$anonfun$main$3.apply(NettyServer.scala:284) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at scala.Option.map(Option.scala:146) [org.scala-lang.scala-library-2.11.7.jar:na]
at play.core.server.NettyServer$.main(NettyServer.scala:284) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.core.server.NettyServer.main(NettyServer.scala) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
2016-09-24 07:01:51,634 - [DEBUG] - from com.jolbox.bonecp.PreparedStatementHandle in main Select
asset9.STATE as asset9_STATE,
asset9.tag as asset9_tag,
asset9.id as asset9_id,
asset9.updated as asset9_updated,
asset9.STATUS as asset9_STATUS,
asset9.ASSET_TYPE as asset9_ASSET_TYPE,
asset9.deleted as asset9_deleted,
asset9.created as asset9_created
From
asset asset9
Where
(asset9.tag = 'none')
2016-09-24 07:01:51,635 - [ERROR] - from application in main Failed to create assetlog
play.api.PlayException: [neither features.syslogAsset or multicollins.thisInstance were specified]
at collins.util.Tattler$$anonfun$syslogAsset$2.apply(Tattler.scala:84) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.Tattler$$anonfun$syslogAsset$2.apply(Tattler.scala:84) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at scala.Option.getOrElse(Option.scala:121) [org.scala-lang.scala-library-2.11.7.jar:na]
at collins.util.Tattler.syslogAsset$lzycompute(Tattler.scala:83) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.Tattler.syslogAsset(Tattler.scala:83) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.Tattler.system(Tattler.scala:75) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Configurable$class.mergeReferenceAndSave(Configurable.scala:125) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.CryptoConfig$.mergeReferenceAndSave(CryptoConfig.scala:6) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPMaybe some thing wrong about the Where (asset9.tag = 'none')?
gtt116
commented
8 years ago So, after I insert some fake record into mysql by insert into asset (tag) values ('none'), I found something new:
2016-09-24 07:17:33,784 - [DEBUG] - from com.jolbox.bonecp.PreparedStatementHandle in main insert into asset_log (format, source, ASSET_ID, MESSAGE_TYPE, CREATED_BY, message, created) values (0,3,3,3,'','Exception in mergeReferenceAndSave: Configuration error[Im sorry, but it looks like you may be missing the unlimited strength Java Cryptography Extension. Unfortunately this is required for operations. You can find more details here http://www.bouncycastle.org/wiki/display/JA1/Frequently+Asked+Questions.]',2016-09-24 07:17:33.779)
2016-09-24 07:17:33,785 - [DEBUG] - from collins.callbacks.Callback$ in main Async Firing asset_log_create
2016-09-24 07:17:33,787 - [ERROR] - from collins.util.config.CryptoConfig$ in main Exception in mergeReferenceAndSave: Configuration error[Im sorry, but it looks like you may be missing the unlimited strength Java Cryptography Extension. Unfortunately this is required for operations. You can find more details here http://www.bouncycastle.org/wiki/display/JA1/Frequently+Asked+Questions.]
play.api.PlayException: Configuration error[Im sorry, but it looks like you may be missing the unlimited strength Java Cryptography Extension. Unfortunately this is required for operations. You can find more details here http://www.bouncycastle.org/wiki/display/JA1/Frequently+Asked+Questions.]
at collins.util.config.ConfigAccessor$$anonfun$globalError$2.apply(ConfigAccessor.scala:28) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.ConfigAccessor$$anonfun$globalError$2.apply(ConfigAccessor.scala:28) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at scala.Option.getOrElse(Option.scala:121) [org.scala-lang.scala-library-2.11.7.jar:na]
at collins.util.config.ConfigAccessor$class.globalError(ConfigAccessor.scala:28) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.CryptoConfig$.globalError(CryptoConfig.scala:6) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.CryptoConfig$.validateConfig(CryptoConfig.scala:20) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Configurable$class.mergeReferenceAndSave(Configurable.scala:110) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.CryptoConfig$.mergeReferenceAndSave(CryptoConfig.scala:6) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Configurable$class.initialize(Configurable.scala:49) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.CryptoConfig$.initialize(CryptoConfig.scala:6) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Registry$$anonfun$validate$1.apply(Registry.scala:56) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Registry$$anonfun$validate$1.apply(Registry.scala:54) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) [org.scala-lang.scala-library-2.11.7.jar:na]
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) [org.scala-lang.scala-library-2.11.7.jar:na]
at collins.util.config.Registry$.validate(Registry.scala:54) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Registry$.setupRegistry(Registry.scala:50) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at Global$.onStart(Global.scala:43) [collins.collins-2.0-SNAPSHOT.jar:na]
at play.api.GlobalPlugin.onStart(GlobalSettings.scala:220) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.api.Play$$anonfun$start$1$$anonfun$apply$mcV$sp$1.apply(Play.scala:91) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.api.Play$$anonfun$start$1$$anonfun$apply$mcV$sp$1.apply(Play.scala:91) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at scala.collection.immutable.List.foreach(List.scala:381) [org.scala-lang.scala-library-2.11.7.jar:na]
at play.api.Play$$anonfun$start$1.apply$mcV$sp(Play.scala:91) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.api.Play$$anonfun$start$1.apply(Play.scala:91) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.api.Play$$anonfun$start$1.apply(Play.scala:91) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]The java -version is
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)It does not seem like collins hate what I insert into MySQL, is it about this jdk does not work for collins 2.0?
william-richard
commented
8 years ago Yep, that does look like you're missing the unlimited strength JCE - have you been able to run previous versions of collins on this same set up? We have a bit of docs about setting it up here: http://tumblr.github.io/collins/index.html#quickstart
I'm also realizing that I may have built collins using JDK 8, which could be causing problems for JD7 and JDK6 - I'll try rebuilding now using JDK6 and we can see if that fixes the problem.
william-richard
commented
8 years ago @gtt116 I have updated https://github.com/tumblr/collins/releases/download/v2.0.0/collins-v2.0.0.zip to be a version of the code built with java 1.7 - can you give it a try and let me know if it works?
gtt116
commented
8 years ago @Primer42 I have tested the new version you provided, but the log says the same:
from /var/log/collins/application.log
2016-09-27 09:46:13,645 - [DEBUG] - from com.jolbox.bonecp.PreparedStatementHandle in main insert into asset_log (format, source, ASSET_ID, MESSAGE_TYPE, CREATED_BY, message, created) values (0,3,3,3,'','Exception in mergeReferenceAndSave: Configuration error[Im sorry, but it looks like you may be missing the unlimited strength Java Cryptography Extension. Unfortunately this is required for operations. You can find more details here http://www.bouncycastle.org/wiki/display/JA1/Frequently+Asked+Questions.]',2016-09-27 09:46:13.641)
2016-09-27 09:46:13,646 - [DEBUG] - from collins.callbacks.Callback$ in main Async Firing asset_log_create
2016-09-27 09:46:13,648 - [ERROR] - from collins.util.config.CryptoConfig$ in main Exception in mergeReferenceAndSave: Configuration error[Im sorry, but it looks like you may be missing the unlimited strength Java Cryptography Extension. Unfortunately this is required for operations. You can find more details here http://www.bouncycastle.org/wiki/display/JA1/Frequently+Asked+Questions.]
play.api.PlayException: Configuration error[Im sorry, but it looks like you may be missing the unlimited strength Java Cryptography Extension. Unfortunately this is required for operations. You can find more details here http://www.bouncycastle.org/wiki/display/JA1/Frequently+Asked+Questions.]
at collins.util.config.ConfigAccessor$$anonfun$globalError$2.apply(ConfigAccessor.scala:28) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.ConfigAccessor$$anonfun$globalError$2.apply(ConfigAccessor.scala:28) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at scala.Option.getOrElse(Option.scala:121) [org.scala-lang.scala-library-2.11.7.jar:na]
at collins.util.config.ConfigAccessor$class.globalError(ConfigAccessor.scala:28) ~[collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.CryptoConfig$.globalError(CryptoConfig.scala:6) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.CryptoConfig$.validateConfig(CryptoConfig.scala:20) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Configurable$class.mergeReferenceAndSave(Configurable.scala:110) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.CryptoConfig$.mergeReferenceAndSave(CryptoConfig.scala:6) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Configurable$class.initialize(Configurable.scala:49) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.CryptoConfig$.initialize(CryptoConfig.scala:6) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Registry$$anonfun$validate$1.apply(Registry.scala:56) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Registry$$anonfun$validate$1.apply(Registry.scala:54) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) [org.scala-lang.scala-library-2.11.7.jar:na]
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) [org.scala-lang.scala-library-2.11.7.jar:na]
at collins.util.config.Registry$.validate(Registry.scala:54) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at collins.util.config.Registry$.setupRegistry(Registry.scala:50) [collins.collins-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
at Global$.onStart(Global.scala:43) [collins.collins-2.0-SNAPSHOT.jar:na]
at play.api.GlobalPlugin.onStart(GlobalSettings.scala:220) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.api.Play$$anonfun$start$1$$anonfun$apply$mcV$sp$1.apply(Play.scala:91) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.api.Play$$anonfun$start$1$$anonfun$apply$mcV$sp$1.apply(Play.scala:91) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at scala.collection.immutable.List.foreach(List.scala:381) [org.scala-lang.scala-library-2.11.7.jar:na]
at play.api.Play$$anonfun$start$1.apply$mcV$sp(Play.scala:91) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.api.Play$$anonfun$start$1.apply(Play.scala:91) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.api.Play$$anonfun$start$1.apply(Play.scala:91) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.utils.Threads$.withContextClassLoader(Threads.scala:21) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.api.Play$.start(Play.scala:90) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.core.StaticApplication.<init>(ApplicationProvider.scala:55) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.core.server.NettyServer$.createServer(NettyServer.scala:253) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.core.server.NettyServer$$anonfun$main$3.apply(NettyServer.scala:289) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at play.core.server.NettyServer$$anonfun$main$3.apply(NettyServer.scala:284) [com.typesafe.play.play_2.11-2.3.9.jar:2.3.9]
at scala.Option.map(Option.scala:146) [org.scala-lang.scala-library-2.11.7.jar:na]
I am using collins-v2.0.0.zip on Centos6.4, jdk1.6.0.
After running scripts/setup, follow all the instructions. But curl failed to connect collins server.
I have confirm the mysql username and password is correct in
/usr/local/collins/current/conf/production.conf.Logs from
/var/log/collins/errorAnd logs from
/var/log/collins/application.logPlease let me know if I can provide more information, thank you very much.