Search engine

[david972]

Preflight checklist

• [X] I could not find a solution in the existing issues, docs, nor discussions.
• [X] I agree to follow this project's Code of Conduct.
• [X] I have read and am following this repository's Contribution Guidelines.
• [ ] This issue affects my Ory Cloud project.
• [X] I have joined the Ory Community Slack.
• [X] I am signed up to the Ory Security Patch Newsletter.

Context and scope

The goal is to design the search engine feature to retrieve and filtering the results

Goals and non-goals

Goals

• Add a search field
• Review the paging

The design

TO DO

APIs

No response

Data storage

No response

Code and pseudo-code

No response

Degree of constraint

No response

Alternatives considered

https://github.com/ory/kratos/pull/2442#issuecomment-1190867502

[david972]

@ofairfoul https://github.com/ory/kratos/pull/2442#issuecomment-1190867502 Would it be possible to simply expose FindByCredentialsIdentifier on the Admin Identity API? I think this would meet the needs of the majority of people who just want to find usernames(emails) which are already registered. e.g. https://developer.okta.com/docs/reference/api/users/#get-user-with-login

[kmherrmann]

Yes, I believe that makes sense!

[Windfarer]

It would be better if we can query by the fields from public_metadata and admin_metadata

[aeneasr]

@ofairfoul #2442 (comment) Would it be possible to simply expose FindByCredentialsIdentifier on the Admin Identity API? I think this would meet the needs of the majority of people who just want to find usernames(emails) which are already registered. e.g. https://developer.okta.com/docs/reference/api/users/#get-user-with-login

That's a good idea IMO! @zepatrik probably also needs this!

[david972]

let me know I can spend time on it

[kmherrmann]

@david972 a PR would be much appreciated! :)

[viters]

@david972 @kmherrmann Hows it going? Seems like a really nice feature to have with not a lot of work required to deliver, I could try to pick it up if it got stuck. I see only one closed PR https://github.com/ory/kratos/pull/2442 related to this, with different approach to implementation.

[david972]

Very well and you ? The other one was to "translate" the call into sql query this one is to expose a method on admin side

[jonas-jonas]

FYI, we now shipped the credentials_identifier query parameter in the listIdentities endpoint:

• Changelog: https://changelog.ory.sh/announcements/an-api-to-fetch-identities-by-email-or-username
• API Documentation: https://www.ory.sh/docs/reference/api#tag/identity/operation/listIdentities

[artsiommiksiuk]

@jonas-jonas, by any chance, idk if you aware, but what about "preview_credentials_identifier_similar" field in listIdentities? It states that it will be removed, but some fuzzy search functionality is also needed for my use cases. I can't find any info about this field.

Also, I don't see any resolution nor discussion about lookup in metadata fields as well. Every ticket and discussion leading here, and it was just closed with only exact match filtering. Exact match lookup is barely can be called a search engine functionality IMHO.

Would be nice to understand if there's any plan, or discussion which I just overlooked about identities search .

[jonas-jonas]

The problem with that parameter is, that the underlying database needs to support it well, which hasn't always been the case, yielding inconsistent results and database issues.

Adding support for search engines is a huge topic, that is not on our immediate roadmap, unfortunately. That includes metadata lookup, which is similarly complex, because of the schema-less nature of metadata.

[artsiommiksiuk]

That's a bummer. So, if we need it, the only option for us is to implement own GDPR compliant (because we are in EU) index for that?

[jonas-jonas]

preview_credentials_identifier_similar is based on such an index, so for now it will keep working.

[aeneasr]

We removed it, it’s not scalable