DBpedia live sync failing

hpadhuka commented 8 years ago

2016-05-06-03-000017-bash-4.1$ sh bin/liveSync.sh [INFO Global] Options file read successfully [INFO ChangesetExecutor] Patch 2015-08-27-15-000201 CLEARED 160 resources [INFO ChangesetExecutor] Patch 2015-08-27-15-000201 DELETED 977 triples [WARN ChangesetExecutor] Error in query execution: org.dbpedia.extraction.live.mirror.sparul.SPARULException: org.dbpedia.extraction.live.mirror.sparul.SPARULException: virtuoso.jdbc4.VirtuosoException: COL..: Insert stopped because out of seg data here or elsewhere host 0 key RDF_QUAD slice 0 at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQLWrapper(SPARULVosExecutor.java:74) at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.executeSPARUL(SPARULVosExecutor.java:29) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor.executeSparulWrapper(ChangesetExecutor.java:140) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor.executeAction(ChangesetExecutor.java:96) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor.applyChangeset(ChangesetExecutor.java:54) at org.dbpedia.extraction.live.mirror.LiveSync.main(LiveSync.java:181) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293) at java.lang.Thread.run(Thread.java:745) Caused by: org.dbpedia.extraction.live.mirror.sparul.SPARULException: virtuoso.jdbc4.VirtuosoException: COL..: Insert stopped because out of seg data here or elsewhere host 0 key RDF_QUAD slice 0 at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQL(SPARULVosExecutor.java:88) at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQLWrapper(SPARULVosExecutor.java:40) ... 11 more Caused by: virtuoso.jdbc4.VirtuosoException: COL..: Insert stopped because out of seg data here or elsewhere host 0 key RDF_QUAD slice 0 at virtuoso.jdbc4.VirtuosoResultSet.process_result(Unknown Source) at virtuoso.jdbc4.VirtuosoResultSet.(Unknown Source) at virtuoso.jdbc4.VirtuosoStatement.sendQuery(Unknown Source) at virtuoso.jdbc4.VirtuosoStatement.executeQuery(Unknown Source) at com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464) at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQL(SPARULVosExecutor.java:86) ... 12 more [WARN ChangesetExecutor] Tried to ADD 772 but failed, splitting into chunks to spot the error [WARN ChangesetExecutor] Error in query execution:

[ERROR ChangesetExecutor] Cannot ADD triple: http://dbpedia.org/resource/Alexander_Tachie_Mensah http://dbpedia.org/ontology/wikiPageExtracted "2015-08-27T13:24:03Z"^^http://www.w3.org/2001/XMLSchema#dateTime .

jordipala commented 8 years ago

Same problem here and I have no clue about whats happening. I've made a clean restart two times and the error appears again at a different point.

jimkont commented 8 years ago

Hi, sorry @hpadhuka for the late reply, I totally missed this issue looks like the VOS database got corrupted

I think this it the online documentation on how to recover http://docs.openlinksw.com/virtuoso/backup_recovery/

@pkleef can you confirm?

jordipala commented 8 years ago

The backup_recovery procedure is confuse in the Openlink official documentation.

I restored the virtuoso data directory from a previous backup I made using the tar command. After that, the live script continues to advance until the next error occurs. And the point where it occurs is different on every occasion.

I am getting data corruption once or twice for every month of data that the scripts synchronizes.

When you test the dbpedia-live-mirror, are you receiving the same kind of error?
Could you share the recommended Openlink Virtuoso configuration if that helps to avoid this problems?
Any other recommendation to avoid the error?

It's a blocker right now as I need to restore and redo work every day to continue advancing.

ewaszwed commented 8 years ago

Hello,

I've come across the same issue today. After loading live dbpedia dump (very recent from http://live.dbpedia.org/dumps/) I kicked off: sh bin/liveSync.sh that finished without any errors then I run: bin/ontologySync.sh and it gave me many errors:

[WARN OntologyHandler] Could not read cached file, assuming first run... [WARN ChangesetExecutor] Error in query execution: org.dbpedia.extraction.live.mirror.sparul.SPARULException: org.dbpedia.extraction.live.mirror.sparul.SPARULException: virtuoso.jdbc4.VirtuosoException: SQ074: Line 500: at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQLWrapper(SPARULVosExecutor.java:74) at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.executeSPARUL(SPARULVosExecutor.java:29) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor.executeSparulWrapper(ChangesetExecutor.java:140) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor.executeAction(ChangesetExecutor.java:96) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor.applyChangeset(ChangesetExecutor.java:54) at org.dbpedia.extraction.live.mirror.OntologySync.main(OntologySync.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294) at java.lang.Thread.run(Thread.java:745) Caused by: org.dbpedia.extraction.live.mirror.sparul.SPARULException: virtuoso.jdbc4.VirtuosoException: SQ074: Line 500: at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQL(SPARULVosExecutor.java:88) at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQLWrapper(SPARULVosExecutor.java:40) ... 11 more Caused by: virtuoso.jdbc4.VirtuosoException: SQ074: Line 500: at virtuoso.jdbc4.VirtuosoResultSet.process_result(Unknown Source) at virtuoso.jdbc4.VirtuosoResultSet.(Unknown Source) at virtuoso.jdbc4.VirtuosoStatement.sendQuery(Unknown Source) at virtuoso.jdbc4.VirtuosoStatement.executeQuery(Unknown Source) at com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464) at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQL(SPARULVosExecutor.java:86) ... 12 more [WARN ChangesetExecutor] Tried to ADD 30977 but failed, splitting into chunks to spot the error

Currently it is a blocker for me. Was your issue ever fixed? Appreciate any feedback.

jordipala commented 8 years ago

Hi,

Same error here executing bin/ontologySync.sh. I'm using the last development version of Virtuoso (version 07.20.3217)

Best regards,

J.

On 4 October 2016 at 17:46, ewaszwed notifications@github.com wrote:

Hello,

I've come across the same issue today. After loading live dbpedia dump (very recent from http://live.dbpedia.org/dumps/) I kicked off: sh bin/liveSync.sh that finished without any errors then I run: bin/ontologySync.sh and it gave me many errors:

[WARN OntologyHandler] Could not read cached file, assuming first run... [WARN ChangesetExecutor] Error in query execution: org.dbpedia.extraction.live.mirror.sparul.SPARULException: org.dbpedia.extraction.live.mirror.sparul.SPARULException: virtuoso.jdbc4.VirtuosoException: SQ074: Line 500: at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor. execSQLWrapper(SPARULVosExecutor.java:74) at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor. executeSPARUL(SPARULVosExecutor.java:29) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor. executeSparulWrapper(ChangesetExecutor.java:140) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor. executeAction(ChangesetExecutor.java:96) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor. applyChangeset(ChangesetExecutor.java:54) at org.dbpedia.extraction.live.mirror.OntologySync.main( OntologySync.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294) at java.lang.Thread.run(Thread.java:745) Caused by: org.dbpedia.extraction.live.mirror.sparul.SPARULException: virtuoso.jdbc4.VirtuosoException: SQ074: Line 500: at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQL( SPARULVosExecutor.java:88) at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor. execSQLWrapper(SPARULVosExecutor.java:40) ... 11 more Caused by: virtuoso.jdbc4.VirtuosoException: SQ074: Line 500: at virtuoso.jdbc4.VirtuosoResultSet.process_result(Unknown Source) at virtuoso.jdbc4.VirtuosoResultSet.(Unknown Source) at virtuoso.jdbc4.VirtuosoStatement.sendQuery(Unknown Source) at virtuoso.jdbc4.VirtuosoStatement.executeQuery(Unknown Source) at com.jolbox.bonecp.StatementHandle.executeQuery( StatementHandle.java:464) at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQL( SPARULVosExecutor.java:86) ... 12 more [WARN ChangesetExecutor] Tried to ADD 30977 but failed, splitting into chunks to spot the error

Currently it is a blocker for me. Was your issue ever fixed? Appreciate any feedback.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dbpedia/dbpedia-live-mirror/issues/7#issuecomment-251427955, or mute the thread https://github.com/notifications/unsubscribe-auth/AAYlPITBfo7orIUPBlunvPafGC-42QIzks5qwnTFgaJpZM4IYvHK .

jimkont commented 8 years ago

Hi, maybe the VOS JDBC driver is the problem here, the one bundled with maven is a bit old @pkleef what is the best way to get access to the latest JDBC driver the current approach to take snapshots and push to the AKSW maven repo is not so flexible

hpadhuka commented 8 years ago

Hello, Any update regarding this ? I'm still getting the below errors while running the live updates on Virtuoso 7.2.1. I started with a clean install of latest DBpedia live dump. But the liveSync process failed within a couple of days. I did check if the database is corrupt and it is Not.

-bash-4.1$ sh bin/liveSync.sh [INFO Global] Options file read successfully [INFO ChangesetExecutor] Patch 2016-10-24-00-000002 DELETED 745 triples [WARN ChangesetExecutor] Error in query execution: org.dbpedia.extraction.live.mirror.sparul.SPARULException: org.dbpedia.extraction.live.mirror.sparul.SPARULException: virtuoso.jdbc4.VirtuosoException: COL..: Insert stopped because out of seg data here or elsewhere host 0 key RDF_QUAD slice 0 at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.execSQLWrapper(SPARULVosExecutor.java:74) at org.dbpedia.extraction.live.mirror.sparul.SPARULVosExecutor.executeSPARUL(SPARULVosExecutor.java:29) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor.executeSparulWrapper(ChangesetExecutor.java:140) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor.executeAction(ChangesetExecutor.java:96) at org.dbpedia.extraction.live.mirror.changesets.ChangesetExecutor.applyChangeset(ChangesetExecutor.java:54) at org.dbpedia.extraction.live.mirror.LiveSync.main(LiveSync.java:181) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293) at java.lang.Thread.run(Thread.java:745)

dbpedia / dbpedia-live-mirror

DBpedia live sync failing #7