docs(pse): adding custom search engine

[Kristinita]

1. Summary

It would be nice, if documentation of this repository will contain information, how users can create Surfingkeys inline query for custom search engines, that created on Programmable Search Engine (formerly Google Custom Search Engine) platform.

2. Argumentation

2.1. Common cause

Not all Surfingkeys users has great JavaScript skills to add a search engine yourself 😿.

3. Example

3.1. Data

For example, I want to add “Kristinita's Search” search engine.

1. Short “Kristinita's Search” description on English
2. Link to search engine
3. cse.google.com direct link
4. my Surfingkeys configuration

3.2. Expected behavior

Inline Surfingkeys SERP similar as real SERP:

4. Not helped

Yes, I have seen, that part of completions.js contains “Helper functions for Google Custom Search Engine completions”, but I couldn't understand, how I can use it for Kristinita's Search.

Thanks.

[b0o]

Hey @Kristinita, thanks for the question.

I see from your configuration that you're not using my conf setup. That's fine, it just means you will need to copy over the necessary code to make this work.

I was able to use your Google CSE as a SurfingKeys completion engine using the following:

const googleCseKey = "PUT_YOUR_GOOGLE_CSE_KEY_HERE"

const createSuggestionItem = (html, props = {}) => {
  const li = document.createElement("li")
  li.innerHTML = html
  return { html: li.outerHTML, props }
}

addSearchAliasX("k",
  "kristinita",
  "https://cse.google.com/cse/publicurl?cx=013024336414733191742:sps98skj394&q=",
  "o",
  `https://www.googleapis.com/customsearch/v1?key=${googleCseKey}&cx=013024336414733191742:sps98skj394&q=`,
  (response) => {
    const res = JSON.parse(response.text).items
    return res.map((s) => createSuggestionItem(`
        <div>
          <div class="title"><strong>${s.htmlTitle}</strong></div>
          <div>${s.htmlSnippet}</div>
        </div>`, { url: s.link }))
  })

mapkey("ok", "#8Search kristinita", () => Front.openOmnibar({ type: "SearchEngine", extra: "k" }))

You need to put your Google CSE API key in the variable googleCseKey. If you don't have a Google Custom Search API key, you can get one here by clicking the 'Get a Key' button.

Please let me know if this helps!

[Kristinita]

Status: Problems 😿

1. Main

1.1. Public API key

I'm Firefox user.

• Surfingkeys official repository:

You can load settings from URLs in scheme http/https, for example, https://gist.githubusercontent.com/brookhong/755982e0014004207849f6e357af56f3/raw/777f62deb54098d82070e1e05dc86f9a798a29a1/gistfile1.txt. If you want to load settings from local file, please check Allow access to file URLs in chrome://extensions/ for this extension, and specify the full path of your settings file, for example, /home/brook/.surfingkeys.js. Unfortunately there is no API support to read local file for Firefox, so you could not load settings from local file in Firefox.

• Your conf.priv.example.js:

// WARNING: Don't git commit your actual conf.priv.js file if you // add real API keys to it! Malicious parties frequently scan GitHub for // these keys and use them for bad stuff. The conf.priv.js file is already // ignored by .gitignore, but just be safe.

1.2. Limits

• Custom Search JSON API Pricing:

Custom Search JSON API provides 100 search queries per day for free.

For example, I need Kira search query. K — 1st query, i — 2nd, r — 3rd and so on, isn't it? The limit will end quickly. Like mine yesterday:

{
  "error": {
    "code": 429,
    "message": "Quota exceeded for quota group 'default' and limit 'Queries per day' of service 'customsearch.googleapis.com' for consumer 'project_number:831504421415'.",
    "errors": [
      {
        "message": "Quota exceeded for quota group 'default' and limit 'Queries per day' of service 'customsearch.googleapis.com' for consumer 'project_number:831504421415'.",
        "domain": "global",
        "reason": "rateLimitExceeded"
      }
    ],
    "status": "RESOURCE_EXHAUSTED"
  }
}

2. Side

2.1. Comments

I should have put your code at the beginning of Surfingkeys configuration file, before any comments.

Is it necessary? Or I can add comments before code?

2.2. Images

• Example JSON API answer

For example, this works for me:

const googleCseKey = "My real code";

const createSuggestionItem = (html, props = {}) => {
  const li = document.createElement("li");
  li.innerHTML = html;
  return { html: li.outerHTML, props };
};

addSearchAliasX("k",
  "Kristinita's Search",
  "https://cse.google.com/cse/publicurl?cx=013024336414733191742:sps98skj394&q=",
  "o",
  `https://www.googleapis.com/customsearch/v1?key=${googleCseKey}&cx=013024336414733191742:sps98skj394&q=`,
  (response) => {
    const res = JSON.parse(response.text).items;
    return res.map((s) => createSuggestionItem(`
        <div class="KiraSERPResult">
          <div class="KiraTitle"><strong>${s.htmlTitle}</strong></div>
          <div class="KiraFormattedURL">${s.htmlFormattedUrl}</div>
          <div class="KiraSnippet">${s.htmlSnippet}</div>
        </div>`, { url: s.link }));
  });

mapkey("ok", "#8Open Search with alias k", function() {
    Front.openOmnibar({type: "SearchEngine", extra: "k"});
});

But if I add the line:

<div class="KiraThumbnail">${s.pagemap.cse_thumbnail[0].src}</div>

Inline query will not work for me.

Is it possible get images for SERP items?

3. Main questions

1. Is any ideas, how it is possibleuse PRE API key without publish it to public places?

2. Is it possible parse HTML instead of using JSON API? Reason — 100 queries limit.

   1. If no, is it possible set timeout to reduce the number of queries of PRE API? For example, in Surfingkeys: ok → I printed Kira → solely if I don't print any symbol in 1 second, Surfingkeys sends a request to JSON API.

Thanks.

[b0o]

@Kristinita

1.1: Don't publish the API key to a public git repo. You can use various methods to prevent this from happening. The approach used in my conf is to keep the private keys in a file conf.priv.js, which is added to my .gitignore file to ensure it is never committed to the repo, and then using Parcel bundler or similar to bundle that file along with my other configuration files into a final configuration file which I install into SurfingKeys (this final, bundled configuration file is never made public).

I use Firefox as well, and to get around the lack of local file access, I serve the file on a local HTTP server and configure SurfingKeys to read it from there. See the instructions in my README for a bit of an overview on this (Step 5 -> Option A -> Local Web Server).

1.2: See SurfingKey's settings.omnibarSuggestionTimeout - set it to a higher value like 500 or 750 to increase the amount of time between when the last keystroke is pressed and the HTTP request will be sent, i.e. place the line settings.omnibarSuggestionTimeout = 500 in your config.

2.1: Comments can be placed anywhere, they shouldn't affect anything.

2.2: This is generally possible as long as that data is sent with the JSON response from the API. You'll need to inspect the response to see how you can extract the image URL.

Remember to handle cases where an image is not present in a result item; for example, the object at line 349 of your example JSON response doesn't have a cse_thumbnail key. So your code s.pagemap.cse_thumbnail[0].src will try to access an item at index 0 of undefined, which will throw a JS error.

I can't guide you fully on how to do this; you can look at the many examples I have in completions.js to get an idea of how you can parse the JSON response from various APIs. If you have JS questions, try StackOverflow.

3.1: see 1.1 3.2: see 1.2

[b0o]

@Kristinita I'm going to close issue this as it really doesn't pertain to this repo. Feel free to continue posting here if you have any other questions.