stefandesu
commented
2 years ago As a first step, I change the creator search so that the label fields are only searched for strings that are not a valid URI, and the URI fields are only searched for strings that are a valid URI. This should improve performance for the feature that we recently added to Cocoda (https://github.com/gbv/cocoda/issues/520) and also when searching for owns own mappings in Cocoda.
When searching mappings by creator, it was recently adjusted so that the search is performed via a regular expression that allows partial and case-insensitive search. This was done so that when searching for mappings by a creator in Cocoda, searching for "stefan" will also return results for "Stefan" and "Stefan Peters" etc.
When I tested this feature, I was surprised how performant it was despite using a regular expression that, I assumed, did not use any index. However, in retrospect this is obvious that it was only fast because the development instance doesn't have much data in it. With the release instance of jskos-server, we have almost 400k mappings, and the same search takes about 2-3 seconds.
Since there's no index for the kind of search we want, we probably need some kind of n-gram approach, similar to what we're doing for schemes and concepts in
addKeywords(utils/searchHelper.js). This is a bigger change that should be well tested, thus I'm creating this issue here.For now, this doesn't have a big priority because creator search will likely be performed rarely. But we should do something about it in the long run. We should probably also develop some kind of "big data performance tests" so that we generate a large amount of fake data and test how each kind of request performs.