theopenconversationkit / tock

Tock, the open source conversational AI toolkit.
https://doc.tock.ai
Apache License 2.0
496 stars 131 forks source link

NLP_API: Healthcheck ensure MongoDB is available #1245

Closed msansen-esncf closed 3 years ago

msansen-esncf commented 3 years ago

Hi Tock Team,

I can see some troubles with nlp_api start when mongoDB is unavailable that are not took in healthcheck status

it happen when database take longer to start than containers. It can be cool if connection could be tried again or make the container unhealthy?

Regards

Mat

2021-03-28 04:07:14,081 ERROR [main] a.t.n.f.s.ConfigurationRepository: Timed out after 30000 ms while waiting for a server that matches com.mongodb.client.internal.MongoClientDelegate$1@44b18fe4. Client view of cluster state is {type=REPLICA_SET, servers=[{address=mongoHost1:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}, {address=mongohost3:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketException: mongohost3}, caused by {java.net.UnknownHostException: mongohost3}}, {address=mongoHost1:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.NoRouteToHostException: No route to host}}] com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches com.mongodb.client.internal.MongoClientDelegate$1@44b18fe4. Client view of cluster state is {type=REPLICA_SET, servers=[{address=mongoHost1:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}, {address=mongohost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketException: mongoHost}, caused by {java.net.UnknownHostException: mongoHost}}, {address=monoHost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.NoRouteToHostException: No route to host}}] at com.mongodb.internal.connection.BaseCluster.createTimeoutException(BaseCluster.java:413) at com.mongodb.internal.connection.BaseCluster.selectServer(BaseCluster.java:118) at com.mongodb.internal.connection.AbstractMultiServerCluster.selectServer(AbstractMultiServerCluster.java:50) at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:129) at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:87) at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:258) at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:182) at com.mongodb.client.internal.MongoCollectionImpl.executeCreateIndexes(MongoCollectionImpl.java:847) at com.mongodb.client.internal.MongoCollectionImpl.createIndexes(MongoCollectionImpl.java:830) at com.mongodb.client.internal.MongoCollectionImpl.createIndexes(MongoCollectionImpl.java:825) at com.mongodb.client.internal.MongoCollectionImpl.createIndex(MongoCollectionImpl.java:810) at org.litote.kmongo.MongoCollectionsKt.ensureIndex(MongoCollections.kt:636) at org.litote.kmongo.MongoCollectionsKt.ensureIndex(MongoCollections.kt:661) at org.litote.kmongo.MongoCollectionsKt.ensureUniqueIndex(MongoCollections.kt:675) at ai.tock.shared.MongosKt.ensureUniqueIndex(Mongos.kt:242) at ai.tock.shared.MongosKt.ensureUniqueIndex$default(Mongos.kt:239) at ai.tock.nlp.front.storage.mongo.EntityTypeDefinitionMongoDAO$col$2.invoke(EntityTypeDefinitionMongoDAO.kt:55) at ai.tock.nlp.front.storage.mongo.EntityTypeDefinitionMongoDAO$col$2.invoke(EntityTypeDefinitionMongoDAO.kt:49) at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74) at ai.tock.nlp.front.storage.mongo.EntityTypeDefinitionMongoDAO.getCol(EntityTypeDefinitionMongoDAO.kt) at ai.tock.nlp.front.storage.mongo.EntityTypeDefinitionMongoDAO.getEntityTypes(EntityTypeDefinitionMongoDAO.kt:103) at ai.tock.nlp.front.service.ConfigurationRepository.loadEntityTypes(ConfigurationRepository.kt:90)

francoisno commented 3 years ago

Bonjour @msansen-esncf tu peux définir la propriété tock_detailed_healthcheck_enabled=true pour que le healthcheck vérifie la connexion aux bases de données. Je viens d'écrire une documentation beaucoup plus approfondie des lignes de vie Tock, j'espère que cela répondra à tes questions et besoins : http://doc.tock.ai/tock/fr-dark/admin/supervision/#lignes-de-vie-healthchecks