Closed cvdsouza closed 5 years ago
It seems to be a grakn problem.
grakn_1 | Caused by: org.janusgraph.diskstorage.PermanentBackendException: Permanent failure in storage backend
grakn_1 | at org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftKeyColumnValueStore.convertException(CassandraThriftKeyColumnValueStore.java:264)
grakn_1 | at org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftKeyColumnValueStore.getNamesSlice(CassandraThriftKeyColumnValueStore.java:159)
grakn_1 | at org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftKeyColumnValueStore.getNamesSlice(CassandraThriftKeyColumnValueStore.java:108)
grakn_1 | at org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftKeyColumnValueStore.getSlice(CassandraThriftKeyColumnValueStore.java:97)
grakn_1 | at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77)
grakn_1 | at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77)
grakn_1 | at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399)
grakn_1 | at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396)
grakn_1 | at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68)
grakn_1 | at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54)
grakn_1 | at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470)
grakn_1 | at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396)
grakn_1 | at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51)
grakn_1 | at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512)
grakn_1 | at org.janusgraph.graphdb.util.SubqueryIterator.<init>(SubqueryIterator.java:66)
grakn_1 | ... 73 common frames omitted
grakn_1 | Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Cannot assign requested address (connect failed)
grakn_1 | at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
grakn_1 | at org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81)
grakn_1 | at org.janusgraph.diskstorage.cassandra.thrift.thriftpool.CTConnectionFactory.makeRawConnection(CTConnectionFactory.java:110)
grakn_1 | at org.janusgraph.diskstorage.cassandra.thrift.thriftpool.CTConnectionFactory.makeObject(CTConnectionFactory.java:74)
grakn_1 | at org.janusgraph.diskstorage.cassandra.thrift.thriftpool.CTConnectionFactory.makeObject(CTConnectionFactory.java:43)
grakn_1 | at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
grakn_1 | at org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftKeyColumnValueStore.getNamesSlice(CassandraThriftKeyColumnValueStore.java:144)
grakn_1 | ... 86 common frames omitted
grakn_1 | Caused by: java.net.ConnectException: Cannot assign requested address (connect failed)
grakn_1 | at java.net.PlainSocketImpl.socketConnect(Native Method)
grakn_1 | at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
grakn_1 | at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
grakn_1 | at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
grakn_1 | at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
grakn_1 | at java.net.Socket.connect(Socket.java:589)
grakn_1 | at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
grakn_1 | ... 92 common frames omitted
Looking your docker compose do you really have 32Go of memory to run grakn? If yes, can you try to start grakn alone (removing everything else in the docker compose) and see whats happen? You can also try to back to grakn version 1.5.7 and see if the error occurred again?
Thanks.
RE: Memory , yes my VM is : 60CPUs, 120GB RAM , 250GB SSD. I followed your advice: I stood up Grakn alone no issues. I then started adding in my configs into the docker compose. The portion that caused these errors were : worker-import , worker-export replicas which I increased to 10. I did this because I noticed that it's taking over 24 hours for the mitre connector data to be imported into openCTI , so was hoping that by increasing the number of workers I could speed up the process, but it seems grakn starts throwing those errors when I increase the replica numbers.
Thanks for the testing. Seems to be a problem in Grakn cache management. I will try to reproduce and open an issue on grakn side.
@cvdsouza, in the OpenCTI 2.0.0 we will migrate to the new version of Grakn (1.5.9). I let this issue open until the release. I think your problem will be solved with this new release.
@cvdsouza The migration to Grakn 1.5.9 will be in 2.0.1 (release will be done today).
Hi , hoping to get some direction on the error I'm seeing when installing the platform using docker.
Description
{ Please provide a clear and concise description of the bug. } After running
docker-compose --compatibility up
, while the import/exports are running , I get the following error :The docker compose hasn't stopped, but the error occurs periodically.
Environment
Reproducible Steps
Steps to create the smallest reproducible scenario:
Expected Output
In previous versions, the errors were not observed. It was when I used the recently updated docker-compose file that I received these errors.
Attaching the YAML file ( without passwords ). docker-compose (1).txt