Closed maxpoulain closed 4 years ago
Hi! Sorry for this problem.
I believe that the problem is highlighted by: 63187621 triples loaded, starting compaction
. It seems that the loading process has only loaded 63M Wikidata triples and not >10B. Are you sure that the N-Triples dump you downloaded is complete? You could easily check that by counting the number of triples in the file with wc -l
. If the file is complete, then it's probably a bug in the NTriples parser used by Yago.
Hi,
Thanks for your answer !
So, I think it's the NTriples parser
because I have 13611971478 (~13B) Wikidata triples in the dump I downloaded, not only 63M...
Maxime
@maxpoulain Thank you for having had a look at it? When have you downloaded your dump to allow me to download the same one?
Or, even better, may you give me the area of the file where the parsing stopped?
Something like head --lines 63187700 my_dump.nt | tail --lines 200
should do the work.
I downloaded the file the 29th of June and below you can see the result of your command where the parsing stopped :
<http://www.wikidata.org/reference/16326f9deda836b98c017951fe51ff3b025f9123> <http://www.wikidata.org/prop/reference/P248> <http://www.wikidata.org/entity/Q27230564> .
<http://www.wikidata.org/reference/16326f9deda836b98c017951fe51ff3b025f9123> <http://www.wikidata.org/prop/reference/P6284> "biography/hewes-joseph" .
<http://www.wikidata.org/reference/16326f9deda836b98c017951fe51ff3b025f9123> <http://www.wikidata.org/prop/reference/P1810> "Joseph Hewes" .
<http://www.wikidata.org/reference/16326f9deda836b98c017951fe51ff3b025f9123> <http://www.wikidata.org/prop/reference/P813> "2017-10-09T00:00:00Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> .
<http://www.wikidata.org/reference/16326f9deda836b98c017951fe51ff3b025f9123> <http://www.wikidata.org/prop/reference/value/P813> <http://www.wikidata.org/value/b3795d3425e0bbdd474f3138cad4a069> .
<http://www.wikidata.org/reference/20c28c3c9a8bdb0bd5073442e48271f88bdc5644> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Reference> .
<http://www.wikidata.org/reference/20c28c3c9a8bdb0bd5073442e48271f88bdc5644> <http://www.wikidata.org/prop/reference/P854> <https://finding-aids.lib.unc.edu/01381/> .
<http://www.wikidata.org/reference/d33052f0436d0fd7ad80ee0c55f3ffcdc2ead27d> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Reference> .
<http://www.wikidata.org/reference/d33052f0436d0fd7ad80ee0c55f3ffcdc2ead27d> <http://www.wikidata.org/prop/reference/P214> "21147180" .
<http://www.wikidata.org/reference/d33052f0436d0fd7ad80ee0c55f3ffcdc2ead27d> <http://www.wikidata.org/prop/reference/value-normalized/P214> <http://viaf.org/viaf/21147180> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666388> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/Dataset> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666388> <http://schema.org/about> <http://www.wikidata.org/entity/Q1666388> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666388> <http://schema.org/version> "1115336108"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666388> <http://schema.org/dateModified> "2020-02-14T18:23:23Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666388> <http://wikiba.se/ontology#sitelinks> "1"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666388> <http://wikiba.se/ontology#statements> "9"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666388> <http://wikiba.se/ontology#identifiers> "0"^^<http://www.w3.org/2001/XMLSchema#integer> .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Item> .
<https://de.wikipedia.org/wiki/Thomas_Reindl> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/Article> .
<https://de.wikipedia.org/wiki/Thomas_Reindl> <http://schema.org/about> <http://www.wikidata.org/entity/Q1666388> .
<https://de.wikipedia.org/wiki/Thomas_Reindl> <http://schema.org/inLanguage> "de" .
<https://de.wikipedia.org/wiki/Thomas_Reindl> <http://schema.org/isPartOf> <https://de.wikipedia.org/> .
<https://de.wikipedia.org/wiki/Thomas_Reindl> <http://schema.org/name> "Thomas Reindl"@de .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@de .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@de .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@de .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@nb .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@nb .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@nb .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@da .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@da .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@da .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@nn .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@nn .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@nn .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@sv .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@sv .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@sv .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@it .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@it .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@it .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@fr .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@fr .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@fr .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@en .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@en .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@en .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@nl .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@nl .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@nl .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@sl .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@sl .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@sl .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@ast .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@ast .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@ast .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@es .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@es .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@es .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@ca .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@ca .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@ca .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@sq .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@sq .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@sq .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2000/01/rdf-schema#label> "Thomas Reindl"@ga .
<http://www.wikidata.org/entity/Q1666388> <http://www.w3.org/2004/02/skos/core#prefLabel> "Thomas Reindl"@ga .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/name> "Thomas Reindl"@ga .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "\u00F6sterreichischer Politiker, Landtagsabgeordneter und Gemeinderat"@de .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "politicus uit Oostenrijk"@nl .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "\u0633\u064A\u0627\u0633\u064A \u0646\u0645\u0633\u0627\u0648\u064A"@ar .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "Austrian politician"@en .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "\u0985\u09B8\u09CD\u099F\u09CD\u09B0\u09C0\u09AF\u09BC \u09B0\u09BE\u099C\u09A8\u09C0\u09A4\u09BF\u09AC\u09BF\u09A6"@bn .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "pol\u00EDtic austr\u00EDac"@ca .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "pol\u00EDtico austr\u00EDaco"@es .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "Austria poliitik"@et .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "homme politique autrichien"@fr .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "pol\u00EDtico austr\u00EDaco"@gl .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "\u05E4\u05D5\u05DC\u05D9\u05D8\u05D9\u05E7\u05D0\u05D9 \u05D0\u05D5\u05E1\u05D8\u05E8\u05D9"@he .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "politician austriac"@ro .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "politikan austriak"@sq .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "pol\u00EDticu austriacu"@ast .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "\u0430\u0432\u0441\u0442\u0440\u0456\u0439\u0441\u044C\u043A\u0438\u0439 \u043F\u043E\u043B\u0456\u0442\u0438\u043A"@uk .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "polaiteoir Ostarach"@ga .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "politico austriaco"@it .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "pol\u00EDtico austr\u00EDaco"@pt .
<http://www.wikidata.org/entity/Q1666388> <http://schema.org/description> "politikari austriarra"@eu .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/direct/P21> <http://www.wikidata.org/entity/Q6581097> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/direct/P31> <http://www.wikidata.org/entity/Q5> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/direct/P27> <http://www.wikidata.org/entity/Q40> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/direct/P102> <http://www.wikidata.org/entity/Q179111> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/direct/P569> "1961-09-03T00:00:00Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/direct/P735> <http://www.wikidata.org/entity/Q16428906> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/direct/P39> <http://www.wikidata.org/entity/Q23711521> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/direct/P19> <http://www.wikidata.org/entity/Q1741> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/direct/P106> <http://www.wikidata.org/entity/Q82955> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/P21> <http://www.wikidata.org/entity/statement/q1666388-5FBB23F5-D34F-4388-A305-1C0C8410A48F> .
<http://www.wikidata.org/entity/statement/q1666388-5FBB23F5-D34F-4388-A305-1C0C8410A48F> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Statement> .
<http://www.wikidata.org/entity/statement/q1666388-5FBB23F5-D34F-4388-A305-1C0C8410A48F> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#BestRank> .
<http://www.wikidata.org/entity/statement/q1666388-5FBB23F5-D34F-4388-A305-1C0C8410A48F> <http://wikiba.se/ontology#rank> <http://wikiba.se/ontology#NormalRank> .
<http://www.wikidata.org/entity/statement/q1666388-5FBB23F5-D34F-4388-A305-1C0C8410A48F> <http://www.wikidata.org/prop/statement/P21> <http://www.wikidata.org/entity/Q6581097> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/P31> <http://www.wikidata.org/entity/statement/Q1666388-848A48E8-B600-46DF-B7A8-9774595BE412> .
<http://www.wikidata.org/entity/statement/Q1666388-848A48E8-B600-46DF-B7A8-9774595BE412> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Statement> .
<http://www.wikidata.org/entity/statement/Q1666388-848A48E8-B600-46DF-B7A8-9774595BE412> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#BestRank> .
<http://www.wikidata.org/entity/statement/Q1666388-848A48E8-B600-46DF-B7A8-9774595BE412> <http://wikiba.se/ontology#rank> <http://wikiba.se/ontology#NormalRank> .
<http://www.wikidata.org/entity/statement/Q1666388-848A48E8-B600-46DF-B7A8-9774595BE412> <http://www.wikidata.org/prop/statement/P31> <http://www.wikidata.org/entity/Q5> .
<http://www.wikidata.org/entity/statement/Q1666388-848A48E8-B600-46DF-B7A8-9774595BE412> <http://www.w3.org/ns/prov#wasDerivedFrom> <http://www.wikidata.org/reference/9a24f7c0208b05d6be97077d855671d1dfdbc0dd> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/P27> <http://www.wikidata.org/entity/statement/Q1666388-A572C240-FF36-43A8-ABFB-434BE17F2BBA> .
<http://www.wikidata.org/entity/statement/Q1666388-A572C240-FF36-43A8-ABFB-434BE17F2BBA> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Statement> .
<http://www.wikidata.org/entity/statement/Q1666388-A572C240-FF36-43A8-ABFB-434BE17F2BBA> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#BestRank> .
<http://www.wikidata.org/entity/statement/Q1666388-A572C240-FF36-43A8-ABFB-434BE17F2BBA> <http://wikiba.se/ontology#rank> <http://wikiba.se/ontology#NormalRank> .
<http://www.wikidata.org/entity/statement/Q1666388-A572C240-FF36-43A8-ABFB-434BE17F2BBA> <http://www.wikidata.org/prop/statement/P27> <http://www.wikidata.org/entity/Q40> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/P102> <http://www.wikidata.org/entity/statement/Q1666388-9EDC2522-2A79-47F8-B0A2-9EC3C2EBEC01> .
<http://www.wikidata.org/entity/statement/Q1666388-9EDC2522-2A79-47F8-B0A2-9EC3C2EBEC01> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Statement> .
<http://www.wikidata.org/entity/statement/Q1666388-9EDC2522-2A79-47F8-B0A2-9EC3C2EBEC01> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#BestRank> .
<http://www.wikidata.org/entity/statement/Q1666388-9EDC2522-2A79-47F8-B0A2-9EC3C2EBEC01> <http://wikiba.se/ontology#rank> <http://wikiba.se/ontology#NormalRank> .
<http://www.wikidata.org/entity/statement/Q1666388-9EDC2522-2A79-47F8-B0A2-9EC3C2EBEC01> <http://www.wikidata.org/prop/statement/P102> <http://www.wikidata.org/entity/Q179111> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/P569> <http://www.wikidata.org/entity/statement/Q1666388-FC402E65-5826-42EC-BE0F-C3C75D5187A4> .
<http://www.wikidata.org/entity/statement/Q1666388-FC402E65-5826-42EC-BE0F-C3C75D5187A4> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Statement> .
<http://www.wikidata.org/entity/statement/Q1666388-FC402E65-5826-42EC-BE0F-C3C75D5187A4> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#BestRank> .
<http://www.wikidata.org/entity/statement/Q1666388-FC402E65-5826-42EC-BE0F-C3C75D5187A4> <http://wikiba.se/ontology#rank> <http://wikiba.se/ontology#NormalRank> .
<http://www.wikidata.org/entity/statement/Q1666388-FC402E65-5826-42EC-BE0F-C3C75D5187A4> <http://www.wikidata.org/prop/statement/P569> "1961-09-03T00:00:00Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> .
<http://www.wikidata.org/entity/statement/Q1666388-FC402E65-5826-42EC-BE0F-C3C75D5187A4> <http://www.wikidata.org/prop/statement/value/P569> <http://www.wikidata.org/value/ae748ef3345ef8e1d8c3f1048aa97002> .
<http://www.wikidata.org/entity/statement/Q1666388-FC402E65-5826-42EC-BE0F-C3C75D5187A4> <http://www.w3.org/ns/prov#wasDerivedFrom> <http://www.wikidata.org/reference/9a24f7c0208b05d6be97077d855671d1dfdbc0dd> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/P735> <http://www.wikidata.org/entity/statement/Q1666388-681AB976-1DE7-45A0-8A86-6FE6075C156D> .
<http://www.wikidata.org/entity/statement/Q1666388-681AB976-1DE7-45A0-8A86-6FE6075C156D> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Statement> .
<http://www.wikidata.org/entity/statement/Q1666388-681AB976-1DE7-45A0-8A86-6FE6075C156D> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#BestRank> .
<http://www.wikidata.org/entity/statement/Q1666388-681AB976-1DE7-45A0-8A86-6FE6075C156D> <http://wikiba.se/ontology#rank> <http://wikiba.se/ontology#NormalRank> .
<http://www.wikidata.org/entity/statement/Q1666388-681AB976-1DE7-45A0-8A86-6FE6075C156D> <http://www.wikidata.org/prop/statement/P735> <http://www.wikidata.org/entity/Q16428906> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/P39> <http://www.wikidata.org/entity/statement/Q1666388-D8E929A7-0565-4A13-B78D-10D758F982D6> .
<http://www.wikidata.org/entity/statement/Q1666388-D8E929A7-0565-4A13-B78D-10D758F982D6> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Statement> .
<http://www.wikidata.org/entity/statement/Q1666388-D8E929A7-0565-4A13-B78D-10D758F982D6> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#BestRank> .
<http://www.wikidata.org/entity/statement/Q1666388-D8E929A7-0565-4A13-B78D-10D758F982D6> <http://wikiba.se/ontology#rank> <http://wikiba.se/ontology#NormalRank> .
<http://www.wikidata.org/entity/statement/Q1666388-D8E929A7-0565-4A13-B78D-10D758F982D6> <http://www.wikidata.org/prop/statement/P39> <http://www.wikidata.org/entity/Q23711521> .
<http://www.wikidata.org/entity/statement/Q1666388-D8E929A7-0565-4A13-B78D-10D758F982D6> <http://www.w3.org/ns/prov#wasDerivedFrom> <http://www.wikidata.org/reference/9a24f7c0208b05d6be97077d855671d1dfdbc0dd> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/P19> <http://www.wikidata.org/entity/statement/Q1666388-BD87758D-680C-4225-A755-12A9A1FA884C> .
<http://www.wikidata.org/entity/statement/Q1666388-BD87758D-680C-4225-A755-12A9A1FA884C> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Statement> .
<http://www.wikidata.org/entity/statement/Q1666388-BD87758D-680C-4225-A755-12A9A1FA884C> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#BestRank> .
<http://www.wikidata.org/entity/statement/Q1666388-BD87758D-680C-4225-A755-12A9A1FA884C> <http://wikiba.se/ontology#rank> <http://wikiba.se/ontology#NormalRank> .
<http://www.wikidata.org/entity/statement/Q1666388-BD87758D-680C-4225-A755-12A9A1FA884C> <http://www.wikidata.org/prop/statement/P19> <http://www.wikidata.org/entity/Q1741> .
<http://www.wikidata.org/entity/statement/Q1666388-BD87758D-680C-4225-A755-12A9A1FA884C> <http://www.w3.org/ns/prov#wasDerivedFrom> <http://www.wikidata.org/reference/9a24f7c0208b05d6be97077d855671d1dfdbc0dd> .
<http://www.wikidata.org/entity/Q1666388> <http://www.wikidata.org/prop/P106> <http://www.wikidata.org/entity/statement/Q1666388-57EF0031-E9EC-4AB4-88A3-F86D1475BCA0> .
<http://www.wikidata.org/entity/statement/Q1666388-57EF0031-E9EC-4AB4-88A3-F86D1475BCA0> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Statement> .
<http://www.wikidata.org/entity/statement/Q1666388-57EF0031-E9EC-4AB4-88A3-F86D1475BCA0> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#BestRank> .
<http://www.wikidata.org/entity/statement/Q1666388-57EF0031-E9EC-4AB4-88A3-F86D1475BCA0> <http://wikiba.se/ontology#rank> <http://wikiba.se/ontology#NormalRank> .
<http://www.wikidata.org/entity/statement/Q1666388-57EF0031-E9EC-4AB4-88A3-F86D1475BCA0> <http://www.wikidata.org/prop/statement/P106> <http://www.wikidata.org/entity/Q82955> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666393> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/Dataset> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666393> <http://schema.org/about> <http://www.wikidata.org/entity/Q1666393> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666393> <http://schema.org/version> "1143355195"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666393> <http://schema.org/dateModified> "2020-03-26T09:44:31Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666393> <http://wikiba.se/ontology#sitelinks> "3"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666393> <http://wikiba.se/ontology#statements> "10"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://www.wikidata.org/wiki/Special:EntityData/Q1666393> <http://wikiba.se/ontology#identifiers> "6"^^<http://www.w3.org/2001/XMLSchema#integer> .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Item> .
<https://en.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/Article> .
<https://en.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/about> <http://www.wikidata.org/entity/Q1666393> .
<https://en.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/inLanguage> "en" .
<https://en.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/isPartOf> <https://en.wikipedia.org/> .
<https://en.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/name> "International Brotherhood of Magicians"@en .
<https://de.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/Article> .
<https://de.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/about> <http://www.wikidata.org/entity/Q1666393> .
<https://de.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/inLanguage> "de" .
<https://de.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/isPartOf> <https://de.wikipedia.org/> .
<https://de.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/name> "International Brotherhood of Magicians"@de .
<https://pt.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/Article> .
<https://pt.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/about> <http://www.wikidata.org/entity/Q1666393> .
<https://pt.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/inLanguage> "pt" .
<https://pt.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/isPartOf> <https://pt.wikipedia.org/> .
<https://pt.wikipedia.org/wiki/International_Brotherhood_of_Magicians> <http://schema.org/name> "International Brotherhood of Magicians"@pt .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2000/01/rdf-schema#label> "International Brotherhood of Magicians"@de .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2004/02/skos/core#prefLabel> "International Brotherhood of Magicians"@de .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/name> "International Brotherhood of Magicians"@de .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2000/01/rdf-schema#label> "International Brotherhood of Magicians"@en .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2004/02/skos/core#prefLabel> "International Brotherhood of Magicians"@en .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/name> "International Brotherhood of Magicians"@en .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2000/01/rdf-schema#label> "International Brotherhood of Magicians"@pt .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2004/02/skos/core#prefLabel> "International Brotherhood of Magicians"@pt .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/name> "International Brotherhood of Magicians"@pt .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2000/01/rdf-schema#label> "International Brotherhood of Magicians"@nl .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2004/02/skos/core#prefLabel> "International Brotherhood of Magicians"@nl .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/name> "International Brotherhood of Magicians"@nl .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2000/01/rdf-schema#label> "International Brotherhood of Magicians"@fr .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2004/02/skos/core#prefLabel> "International Brotherhood of Magicians"@fr .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/name> "International Brotherhood of Magicians"@fr .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2000/01/rdf-schema#label> "Internacia Frataro de Iluziistoj"@eo .
<http://www.wikidata.org/entity/Q1666393> <http://www.w3.org/2004/02/skos/core#prefLabel> "Internacia Frataro de Iluziistoj"@eo .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/name> "Internacia Frataro de Iluziistoj"@eo .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/description> "organization"@en .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/description> "organisaatio"@fi .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/description> "organizat\u00EB"@sq .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/description> "organiza\u0135o"@eo .
<http://www.wikidata.org/entity/Q1666393> <http://schema.org/description> "internationaler Verband der Zauberk\u00FCnstler"@de .
<http://www.wikidata.org/entity/Q1666393> <http://www.wikidata.org/prop/direct/P31> <http://www.wikidata.org/entity/Q43229> .
<http://www.wikidata.org/entity/Q1666393> <http://www.wikidata.org/prop/direct/P856> <http://www.magician.org> .
<http://www.wikidata.org/entity/Q1666393> <http://www.wikidata.org/prop/direct/P3348> "67040" .
<http://www.wikidata.org/entity/Q1666393> <http://www.wikidata.org/prop/direct-normalized/P3348> <http://data.nlg.gr/resource/authority/record67040> .
<http://www.wikidata.org/entity/Q1666393> <http://www.wikidata.org/prop/direct/P3417> "International-Brotherhood-of-Magicians" .
<http://www.wikidata.org/entity/Q1666393> <http://www.wikidata.org/prop/direct/P159> <http://www.wikidata.org/entity/Q38022> .
<http://www.wikidata.org/entity/Q1666393> <http://www.wikidata.org/prop/direct/P213> "0000 0001 0080 471X" .
<http://www.wikidata.org/entity/Q1666393> <http://www.wikidata.org/prop/direct/P571> "1922-01-01T00:00:00Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> .
I tried to run the code with only these 200 rows with cargo run --release -- -c wd-preprocessed.db partition -f test.nt.gz
I have no error and 200 triples are loaded.. So, I don't really understand from where the error comes with the big file..
Hi,
Do you have some news about this bug ? Did you manage to reproduce the issue ? Do you need more information ?
Thanks,
Sorry for the delay. I took some vacations.
I have managed to reproduce the issue. I'm currently investigating it.
I have finally found the cause of the error. Wikidata dumps are now using multiple gzip members in the same file. The library the pipeline is using is only decompressing the first member and returning "end of file" after it. I am finishing a fix for it.
Sorry for the long delay.
Nice thank you very much !! Don't worry about the delay ! ;)
Thanks,
Maxime
@Tpt @maxpoulain Was the issue resolved? I guess I am stuck on the same issue.
Yes this issue was resolved, now it correctly parse all ~13B triples
! I ran the code this week without this specific problem, so, I don’t think that your issue is coming from here..
Hi,
I have a question regarding the code of
yago4
.I tried to run the code following the README and it works perfectly BUT I haven't the same execution time ("This process should take around a night if you use an SSD." & "The process should take a few hours.") and mostly I haven't the same output as we can find here : https://yago-knowledge.org/downloads/yago-4. I know that I'm not going to have the exact same output as available on Yago website because I take the latest Wikipedia dump but it's really not the same.
So, first, I downloaded
full Wikidata dump in the N-Triples format
and then I launchedyago4
with the two following commands :and
At the end, I got right output files but they are really really light. As you can see for example,
yago-wd-labels.nt.gz
is only 15M while on Yago website (https://yago-knowledge.org/data/yago4/full/2020-02-24/) is more than 26 Go..So, I just want to understand, if I'm doing something wrong or if there is a bug or an error inside the code.
Thanks for your help.
Maxime