Closed jjkoehorst closed 3 years ago
Hi Jasper, Normally the merge functionality described in the javadoc should still work - that's something people use routinely in OpenRefine. I suspect the problem you have could be due to various things:
Note that more generally I would really like to work on this merging functionality and make it much more configurable: #403 (but that's only relevant to you if the issue is due to point 1 above).
I am indeed using a wikibase instance that runs locally so indeed not wiki data. Is there a way to point all references to the local instance instead of wiki data?
How do you create the data values you are pushing to Wikidata? With which siteIRI? Is it the same as the one you pass to the editing module?
I am pushing nothing to Wikidata the authentication is performed with a local instance.
This is what I use to authenticate:
basicApiConnection = new BasicApiConnection(commandOptions.wiki);
basicApiConnection.login(commandOptions.username,commandOptions.password);
wbde = new WikibaseDataEditor(App.basicApiConnection, App.commandOptions.wiki);
wbdf = new WikibaseDataFetcher(App.basicApiConnection, App.commandOptions.wiki);
wbea = new WbEditingAction(App.basicApiConnection, App.commandOptions.wiki);
wbde.setEditAsBot(true);
wbde.setAverageTimePerEdit(1);
and the wiki is a "-wiki", "http://localhost:8181/w/api.php", path
Ah sorry, I mean how do you create the data values you are pushing to Wikibase? As in, how do you generate the edits?
Ah clear, so I have a item identifier as a string
ItemIdValue itemIdValue = Datamodel.makeWikidataItemIdValue(item);
// Empty list of statements
ArrayList<Statement> statements = new ArrayList<>();
// Statement with creation function
statements.add(Statements.create(itemIdValue, "genome size", BigDecimal.valueOf(genome.getGenomeSize()), wdPageLookup.get("European Nucleotide Archive")));
Statement creation function which returns the statement with a big decimal value and a stated in element
public static Statement create(ItemIdValue itemIdValue, String propertyID, BigDecimal value, String referencePage) {
// Get property value of genome size
PropertyIdValue genomeSizeProperty = Datamodel.makeWikidataPropertyIdValue(propertyLookup.get(propertyID).replaceAll(".*/", ""));
// Set value of genome size to a quantity
QuantityValue genomeSizeValue = Datamodel.makeQuantityValue(value);
// Create the reference
String refPropertyID = propertyLookup.get("stated in").replaceAll(".*/", "");
PropertyIdValue statedIn = Datamodel.makePropertyIdValue(refPropertyID, App.commandOptions.wiki);
Reference reference = ReferenceBuilder.newInstance().withPropertyValue(statedIn, Datamodel.makeItemIdValue(referencePage, App.commandOptions.wiki)).build();
// Create the genome size statement with reference
Statement statement = StatementBuilder.forSubjectAndProperty(itemIdValue, genomeSizeProperty).withValue(genomeSizeValue).withReference(reference).build(); // .withReference(reference)
// Return the statement
return statement;
}
Hope this helps a bit?
I have just synced it to my repository and the statements.add can be found at https://gitlab.com/wurssb/wikigroup/wikistrains/-/blob/master/botjava/src/main/java/nl/munlock/wiki/sync/genetics/GenomeSync.java 72,73
See, the problem is with your first line:
ItemIdValue itemIdValue = Datamodel.makeWikidataItemIdValue(item);
This creates a Wikidata item id value, not one for your Wikibase instance. You should use makeItemIdValue
and supply a siteIRI that identifies your instance (and reuse the same one everywhere else).
We should probably have some checks to detect that and raise an exception properly when you try to push that to your instance.
(and you'll have to check that you are not calling any Wikidata-specific methods elsewhere, for instance Datamodel.makeWikidataPropertyIdValue
)
Thanks a lot this works great! Had multiple occurrences of Wikidata in the code so that is now cleaned up.
When testing on a local Wikibase instance I was adding property information to an item using the
wbde.updateStatements
and later on using thewbde.updateTermsStatements
. Using either method it will add the new statement to the instance even if this statement with this exact value is already stored on this item page.Is there a way to automagically merge duplicate entries and when adding a new property / value when that property already exists with the same value that it will merge the two as in not add the statement as it already exists?
Also according to the functions tested it states that
indicating that this duplication issue should not occur perhaps I am missing an addition argument?