Custom search results layout

[scorchingoyster]

May use Google's CSE, TBD

[scorchingoyster]

managed to use their web tool to spit out some test results, here is a chunk of sample output for result items:

  {
   "kind": "customsearch#result",
   "title": "Man-in-the-browser attack - OWASP",
   "htmlTitle": "Man-in-the-browser attack - OWASP",
   "link": "https://www.owasp.org/index.php/Man-in-the-browser_attack",
   "displayLink": "www.owasp.org",
   "snippet": "The Man-in-the-Browser attack is the same approach as Man-in-the-middle \nattack, but in this case a Trojan Horse is used to intercept and manipulate calls ...",
   "htmlSnippet": "The Man-in-the-Browser attack is the same approach as Man-in-the-<b>middle</b> <br>\nattack, but in this case a Trojan Horse is used to intercept and manipulate calls&nbsp;...",
   "cacheId": "49qAsp30rlUJ",
   "formattedUrl": "https://www.owasp.org/index.php/Man-in-the-browser_attack",
   "htmlFormattedUrl": "https://www.owasp.org/index.php/Man-in-the-browser_attack"
  },
  {
   "kind": "customsearch#result",
   "title": "Securing the SSL/TLS channel against man-in-the-middle attacks ...",
   "htmlTitle": "Securing the SSL/TLS channel against man-in-the-<b>middle</b> attacks ...",
   "link": "https://www.owasp.org/images/4/4b/OWASP_defending-MITMA_APAC2012.pdf",
   "displayLink": "www.owasp.org",
   "snippet": "Mar 15, 2011 ... Securing the SSL/TLS channel against man-in-the-middle attacks: Future \ntechnologies - HTTP Strict. Transport Security and Pinning of Certs.",
   "htmlSnippet": "Mar 15, 2011 <b>...</b> Securing the SSL/TLS channel against man-in-the-<b>middle</b> attacks: Future <br>\ntechnologies - HTTP Strict. Transport Security and Pinning of Certs.",
   "cacheId": "3NhuNmcumXEJ",
   "mime": "application/pdf",
   "fileFormat": "PDF/Adobe Acrobat",
   "formattedUrl": "https://www.owasp.org/.../OWASP_defending-MITMA_APAC2012.pdf",
   "htmlFormattedUrl": "https://www.owasp.org/.../OWASP_defending-MITMA_APAC2012.pdf",
   "pagemap": {
    "cse_thumbnail": [
     {
      "width": "225",
      "height": "225",
      "src": "https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcSuNENyBCLinq9phBvq16BWlG6TGRjW8GVUhuphGbznXpMPJJRuHSDYUDC1"
     }
    ],
    "metatags": [
     {
      "title": "Securing the SSL/TLS channel against man-in-the-middle attacks: Future Technologies",
      "author": "Tobias Gondrom (tobias.gondrom@gondrom.org)",
      "subject": "Securing the SSL/TLS channel against man-in-the-middle attacks: Future Technologies",
      "creationdate": "D:20120528051836+01'00'",
      "moddate": "D:20120528051836+01'00'",
      "producer": "Microsoft® PowerPoint® 2010",
      "creator": "Microsoft® PowerPoint® 2010"
     }
    ],
    "cse_image": [
     {
      "src": "x-raw-image:///fdbe75b231a161d729d64d81f864fefaa80a67089cd09a1a9aa519bef5254c84"
     }
    ]
   }
  }

thinking of showing these data: document title, URL, relevant text snippet, thumbnail (optional)

with the ability to sort by relevance or date.

[mikemccamon]

Sort by relevance (but if you could add a button to re-search and return result by date even better). I don't think we need thumbnail - goal is to have all our content someday as html.

== Search Results Page ==

Search Results for {search-term} from OWASP.org

 <ul>
  <li><a href={link}>{title}</a><br>
  {link}<br>
  {snippet}</li>
   ...
 </ul>

=== Example ===

Man-in-the-browser attack - OWASP https://www.owasp.org/index.php/Man-in-the-browser_attack The Man-in-the-Browser attack is the same approach as Man-in-the-middle \nattack, but in this case a Trojan Horse is used to intercept and manipulate calls ...

Securing the SSL/TLS channel against man-in-the-middle attacks ... https://www.owasp.org/images/4/4b/OWASP_defending-MITMA_APAC2012.pdf Mar 15, 2011 ... Securing the SSL/TLS channel against man-in-the-middle attacks: Future \ntechnologies - HTTP Strict. Transport Security and Pinning of Certs.

I think we should follow Google formatting of having title larger, hyperlink displayed (is that important since all results will be on owasp.org?) is small greenish, and then snippet as our regular paragraph text. Confirmed that Google, Bing, and Duckduckgo use similar layout. With our font treatment we'll be different enough.

The template should have the left column that we can customize. I could see us having the generic "about us" but we could also put "ads" in that space using md.

One big issue with lists like this is pagination. Not sure how Custom search handles that but maybe for v1.0 we just list top 25 results(?). I do like how duckduckgo handles it with a "More Results" button.Also can't tell from JSON object if the search result set include meta data about search (like total results)

[scorchingoyster]

I think what I will do is go through the specs once and then produce a static HTML version styled with CSS for you to review. Once you approve the layout, then Harold can take it from there to implement it the way he thinks that would work best. This seems to be a good approach so far.

[mikemccamon]

Sounds great!

Mike McCamon | m: +1.913.940.2737 | Sent from iPhone

On Aug 24, 2019, at 10:45 AM, Scorching Oyster notifications@github.com wrote:

I think what I will do is go through the specs once and then produce a static HTML version styled with CSS for you to review. O nce you approve the layout, then Harold can take it from there to implement it the way he thinks that would work best. This seems to be a good approach so far.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[scorchingoyster]

I skimmed through the API specs a bit and I think your requirements should be doable, although would require some heavy-lifting on Harold's side for the customization:

1. The UI can provide a dropdown to re-search by date, but that would also mean needing to add an optional date label in the result box, otherwise there is going to be a loss of context for the search results (although DuckDuckGo doesn't do this). See screenshot.

2. Going with the approach that DuckDuckGo takes, the extra results can be hidden when they are fetched initially (even that they are already loaded with the page), and only display every X number of items when the "See more results" button is clicked. This can be achieved through client-side JS.

This also reminds me about pricing, which I think you mentioned at one point:

Custom Search JSON API provides 100 search queries per day for free. If you need more, you may sign up for billing in the API Console. Additional requests cost $5 per 1000 queries, up to 10k queries per day.

[mikemccamon]

Thanks for taking a look at the API.

1) Given the complexities and me thinking about it more, our content isn't so volatile to merit date search.

2) I'm going to assume we can tell Google how many results we want back. So perhaps we ask for 100, show 25 and then each time a user clicks they get 25 more up to 100(?) Harold is at beach with family this wekeend so I don't expect him to respond. Let's go with a DDG approach which means we just need template and 100 results.

Pricing) Yes I remember this conversation. I just looked our site metrics and for last 13 month averages 26,264 per month; 875 per day. Seems like we'd be under $2K per year which is tolerable. There might be a nonprofit pricing path (or often since we are OWASP we can get a deal). I think we're still good and I'm 100% convinced our new site design will make it easier for people to discover things anyway!

[mikemccamon]

This works.

[scorchingoyster]

25 a page might be a bit heavy. WIP page shown below is 10. I checked and Google and DuckDuckGo both show 10 results per page by default as well.

Of course, Harold can adjust this later on as he sees fit; what the template does is that it will just fill the body section height as more results are loaded.

[mikemccamon]

Oh yeah... let's go with grabbing 40 results and display 10 at a time.

[scorchingoyster]

Just pushed it to live, please take a look here: https://scorchingoyster.github.io/owasp-layout/search-results

[mikemccamon]

Looking great. I’m on mobile, any chance the green text can have more narrow line spacing? Otherwise looks awesome. Not love correct? Just the design from a static search, correct?

Mike McCamon | m: +1.913.940.2737 | Sent from iPhone

On Aug 24, 2019, at 6:19 PM, Scorching Oyster notifications@github.com wrote:

Just pushed it to live, please take a look here: https://scorchingoyster.github.io/owasp-layout/search-results

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[scorchingoyster]

I can change the line spacing, no problem.

It's not live; just a copy/paste HTML file using the content sample I fetched from above.