Combine sameAs retrieval and entityChecking

[MichaelRoeder]

At the moment, the sameAs retrieval as well as the entityChecking are done independently. However, if the sameAs retrieval was able to retrieve data for a given URI, the entity checking can be skipped. Thus, these two steps should be combined into one single step.

[TortugaAttack]

Do i understood it correct? The sameAs retrieves entities which are the same as the resource. And for these entities it will be checked if they exists. So the sameAs retrieval should immediately check if the entity exists instead of doing this after the sameAs retrieval.

So instead of the AbstractDataset the SameAsRetriever should "start" the EntityChecker and the problem should be solved... or did i miss something?

[MichaelRoeder]

It is a little bit more complicated (otherwise it would be too easy :wink: )

If a dataset is loaded it is preprocessed by

1. go through all entities and try to extend their URI set by retrieving owl:sameAs links.
2. go through all URIs of all entities and check whether their URI exists.

However, the first step already checks whether an entity exists since it can not retrieve information about an entity if it is not existing. Thus, I would like to combine both functionalities into one single preprocessing.

There are some additional requirements that have to be taken into account.

• I used "Manager" classes (DomainBasedSameAsRetrieverManager, EntityCheckerManager) that choose SameAsRetriever or EntityChecker instances for a given URI based on the domain/namespace of the URI.
• The sameAs retrieval is done for both - entities from a dataset and entities derived from an annotator - while the entity checking is only done for the datasets
• The sameAs retrieval needs to be able to incorporate information from the dataset or the annotator response. This is done by combining sameAs retrievers if the dataset or annotator response contained sameAs information.
  • datasets: https://github.com/AKSW/gerbil/blob/master/src/main/java/org/aksw/gerbil/dataset/AbstractDatasetConfiguration.java#L64-L81
  • annotators: at the moment this feature is not used but I think it has to be possible just in case that we need it in the future.

You might want to take a look into the sameas and the check packages as well as their JUnit tests to get a better understanding. After that you should think about the structure the preprocessing should have and how it can fulfill the requirements. We can discuss about that if you want.

Cheers, Micha

[MichaelRoeder]

According to a comment in #137 this preprocessing should only be done, if it is needed for the experiment. But this can be added later on, after the refactoring described above is done.