search

cambialens / lens-api-doc

10 stars 5 forks source link

Question: patent search mappings used at Lens.org #67

Open markharwood opened 9 months ago

markharwood commented 9 months ago

Is it possible to share the elasticsearch index mappings used for Lens.org? I fully appreciate if this can't be done because of commercial sensitivity or you not wanting to publicise something you are likely to want to change. The reason for asking is I'm working on an elasticsearch environment where queries can be edited outside of Lens.org and I then want users to be able to take the refined queries and apply them back on the lens.org data store with a high degree of reproducibility in the results. To achieve this it would be useful to know a couple of things: 1) The index mappings used at Lens 2) The default fields used for query_string queries.

I plan to populate my index with data exported from Lens and I have tried to second-guess what might be used as a mapping based on the documentation for searchable fields and the API response format but they differ considerably. I expect the JSON schema used in the elasticsearch index differs from the type of JSON returned via the API (Is this because the API serves elastic-matched results from JSON held in a MongoDB store?).

Thanks for any insight.

AaronBallagh commented 8 months ago

Hi Mark,

Thank you for this question. Can you please explain the purpose for your use of The Lens and clarify the entity that you are doing this for?

It is against The Lens Terms of Use to replicate the existing Lens services, so it would be great if you could clarify your intent.

Thank you,

Aaron

markharwood commented 8 months ago

Hi Aaron. I’m a private individual and previously a member of the core elasticsearch development team. I love what you’ve done with elasticsearch but there’s a number of features I put into elasticsearch that aren’t being used in Lens.org and were designed to help users refine queries. I am making use of these in my front-end query refinement tool similar to 2Dsearch but which benefits greatly from using these elasticsearch features on a local cache of data exported from Lens.org. My ultimate goal would be to bring all these features to the lens.org website and I would be happy to discuss further. In the short term the mappings would improve interoperability of queries developed in my tool when exporting for use on Lens.org. I am currently the only person with access to this tool/environment.

AaronBallagh commented 8 months ago

Hi Mark,

Thanks for providing these details. It would be helpful if we could have call to help us understand further and provide details of the elastic index.

We also collaborate with the community on similar apps through our Lens Labs initiative, you can see the community apps and tools the Lens supports here. So your app might be a good fit for a supported community app.

I have sent you an email, please let me know if you did not get it.

Aaron