weso / weso-scholia

Collaboration between Scholia and WESO
GNU General Public License v2.0
3 stars 3 forks source link

Research about undeterministic results from Wikidata for some queries #6

Open labra opened 3 years ago

labra commented 3 years ago

we have detected that the same query gives different results at different times at wikidata.

Sometimes, a magical number of 6000 results appears. Do some research to check if there is some limit for that.

thewillyhuman commented 3 years ago

After a little bit of research, we've found that here the 6.000 statements limit is tested. So maybe it is intended. According to this, a batch is formed by 6.000 statements. SO maybe it is related to the number of batches, that only one was returned or something similar.

    public void testOverflowOnCount() {
        // 6000 statements - should go over the limit
        // followed by just one statement - will be split in 2 batches as the first change overflows the batch
        List<Change> changes = asList(MANY_STATEMENTS, SMALL_STATEMENT);
        int count = rdfEnv.getRepo().syncFromChanges(changes, /* verifyResult */ false);
        assertThat(count).isEqualTo(2);
        rdfEnv.verifyUpdates(count);
    }