UKGovLD / ukl-registry-poc

Workspace for design of UKGovLD registry proof-of-concept
Apache License 2.0
17 stars 2 forks source link

No federated search (yet) ... I can't find Spittal! #30

Open 6a6d74 opened 11 years ago

6a6d74 commented 11 years ago

I would expect the web UI search to return (at least one) positive "hit" for the term "Spittal" ... this is part of the delegated register /def/bathingWaters [http://environment.data.gov.uk/doc/bathing-water/ukc2102-03600]

I get zero "hits".

I imagine that this is because the Search component is currently only accessing the (default graph in) the local triple store?

When do you anticipate "federated search" being available?

der commented 11 years ago

Correct, federated search is not implemented. Entity retrieval works over DelegatedRegisters but not text search. This is because the DelegatedRegister has no way to specify a text search endpoint and indeed the Bathing Waters URIset doesn't have one.

Text search is supposed to work over FederatedRegisters but that's not implemented yet.

Having search work over Delegation means either also specifying a text search endpoint or maintaining a harvest of the delegation endpoint and duplicating its data in the registry index. I think the latter will be needed and have this on a list but its a pretty big job and not clear how soon can address that one. I think it ties in with the whole notion of processing triggers that will be needed for analysis of registered datasets.

6a6d74 commented 11 years ago

Noted. OpenSearch API looks best for federating queries to delegated registers.

A worry I have when the query is distributed in this way is performance; although (far!) out of scope for this iteration at some point in time "we" should compare this to the performance benefits provided by crawling the delegated / federated resources and building a local index ... albeit at significantly greater implementation complexity :-).

Basically, our options are (i) distribute the query, or (ii) crawl the delegate registers to build a local index.