benbusby / whoogle-search

A self-hosted, ad-free, privacy-respecting metasearch engine
https://pypi.org/project/whoogle-search/
MIT License
9.61k stars 948 forks source link

[FEATURE] Add a parsable (JSON or XML) endpoint for results #607

Open throwaway34241 opened 2 years ago

throwaway34241 commented 2 years ago

Describe the feature you'd like to see added

Current the /search endpoint returns a HTML file to display the results. This works well for UI users, which might be the first use case of Whoogle. But it's hard for programmatic use cases, as it requires parsing HTML.

I propose to add a new /search.{json|xml} endpoint to return the same results in a structured format: JSON or XML

Idea 1: JSON

A /search.json?q=%s endpoint which would search results as JSON.

I don't think there's a standard JSON format to use, but we could take inspiration for https://zenserp.com/ or https://serpapi.com/.

Idea 2: OpenSearch XML format

This project already uses opensearch, so we could leverage the same standard's response XML: https://github.com/dewitt/opensearch/blob/master/opensearch-1-1-draft-6.md#opensearch-response-elements

Additional context

I have a personal preference for JSON, as it seems to be today's web standard for APIs.

Adding parsable results would allow integration with various projects, via API (e.g. Zapier, Integromat), RSS or simple scripts. Some use cases include:

vacom13 commented 2 years ago

@benbusby I will take it up

sutidor commented 2 years ago

@vacom13 is there any beta to test?

vacom13 commented 2 years ago

@sutidor unfortunately there isn't yet 😢.

benbusby commented 2 years ago

Closing in favor of #814, which includes an example JSON response that could be used as a model for Whoogle JSON results.

throwaway34241 commented 1 year ago

814 is a dead link for me, any new updated link?

benbusby commented 1 year ago

Huh, not sure what happened. I'll reopen this one.

Anduin2017 commented 8 months ago

Also expecting this feature :D

zenoize commented 1 month ago

Also expecting this feature

emengweb commented 1 month ago

need json feature