Websites where the Add Custom Search fails

[ssborbis]

If you come across any sites / forms that don't work with the Add Custom Search function, list them here. I have some workarounds in mind, but I need more examples.

news.google.com - main search form

[DenB10]

Other examples: cnil.fr jesuismort.com

Note: The "Find" button in advanced mode (search on MycroftProject) doesn't work (version 1.6.3).

[ssborbis]

Note: The "Find" button in advanced mode (search on MycroftProject) doesn't work (version 1.6.3).

Thanks. 1.6.3.1 going up. Dunno how some of this stuff gets by me and the linter

[ssborbis]

Ok, new code works for those websites. Should work with all non-standard search forms using GET

[DenB10]

Works fine :) But tooltip "PerformASearchTooltip" doesn't appear!

[ssborbis]

But tooltip "PerformASearchTooltip" doesn't appear!

This is driving me nuts. I can't find the problem.

[ssborbis]

nevermind, got it

[sojusnik]

http://context.reverso.net/translation/

[ssborbis]

http://context.reverso.net/translation/

Fix ready

[sojusnik]

https://www.similarweb.com/

[ssborbis]

Same fix seems to work for that one too. I'll get a new release in the next couple days

[Tubelerczyk]

After upgrading to version 1.7.0, it stopped working:

http://chomikuj.pl/action/SearchFiles

But it works in version 1.6.9.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

This, for example, does not work either:

https://www.gry-online.pl/szukaj.asp

And there are more such search engines.

[sojusnik]

Can confirm that with ContextSearch 1.7.0 on Firefox 61.0.1.

While all OpenSearch engine work fine, f.i. when searching from the address bar, some of the ContextSearch ones aren't. Here's a screenshot of one of them.

[Tubelerczyk]

I have Firefox version 61.0.2 (64bit). The 1.6.9 version of the plugin works well on this Firefox. Version 1.7.0 does not support some search engines. For now I am using version 1.6.9.

[ssborbis]

hotfix going up 1.7.0.1

[GlasWolf]

https://who-called.co.uk/

[ssborbis]

https://who-called.co.uk/

That site adds ok for me. It uses the alternative 'do a search' method, but it added. What happens when you try?

[GlasWolf]

That's odd - it worked this time. When I tried earlier it threw up an error, very roughly "ContextSearch was unable to create a search entry for that website".

Not wanting to hijack this thread but is there any way to simulate a search done from the search box, rather than simply insert the search term into the URL? Phone numbers are usually shown formatted with spaces, so that method doesn't work.

Oh and thanks for getting back to me!

[ssborbis]

is there any way to simulate a search done from the search box, rather than simply insert the search term into the URL? Phone numbers are usually shown formatted with spaces, so that method doesn't work.

I'm not sure I know what you mean.

For that particular website, I see it will remove spaces and hyphens, effective defeating my fallback code which checks the url for the search terms. So if you entered 1-234-567-8910 the search results url would be https://who-called.co.uk/Number/12345678910 and my code wouldn't be able to pick out the search terms. I can't account for that kind of processing, be it client-side via javascript or server-side.

[GlasWolf]

Basically I want to be able to highlight and ContextSearch phone numbers which include hyphens and spaces. The {searchTerms} method gives a 404 but, as you say, the site's own search function automatically strips them. Can that be replicated somehow, maybe using POST?

[ssborbis]

Ah, I see. I just tried a highlight search for 1-234-567-8910 and got the 404 error. I'd have to write some site-specific code or add an option for each engine to modify the search terms with a regex rule. Doable, but a bit time-consuming for a rare issue.

You might want to enable 'Manage With Bookmarks' and add a bookmarklet to the ContextSearch Menu folder with the following code as the URL. It won't be 100% what you want, but you'll be able to access it via ContextSearch

javascript:q = "" + (window.getSelection ? window.getSelection() : document.getSelection ? document.getSelection() : document.selection.createRange().text); if (q!=null) location="https://who-called.co.uk/Number/" + escape(q).replace(/[^0-9]/g, ""); void 0

or

javascript:q = "" + (window.getSelection ? window.getSelection() : document.getSelection ? document.getSelection() : document.selection.createRange().text); if (q!=null) window.open("https://who-called.co.uk/Number/" + escape(q).replace(/[^0-9]/g, ""), "_blank"); void 0

for opening in a new tab (popup blocker risk)

[GlasWolf]

Great idea to use bookmarklets. Edited the replace term to replace(/%25\d\d|[^\d]/g, "") to remove character codes, and it works perfectly - thanks so much for your help!

[ssborbis]

@GlasWolf I've added a regex option to modify search terms for each engine. Look for it in the next release.

[GlasWolf]

Even easier! I'll check it out when it appears, thanks again.

[ssborbis]

New regex option is live in 1.8.1. I tested the search engine for https://who-called.co.uk/ using the search regex "%25\\d\\d|[^\\d]", "" and it seems to work. I didn't actually get a proper result due to my ip being blocked (VPN) but the url looked good

searched for 1-234-567-8910 resulting url https://who-called.co.uk/Number/12345678910

[ernestobarrera]

This professional medicines search engine uses the POST method. Would it be possible to add it to your complement? Thank you https://cima.aemps.es/cima/publico/buscadoravanzado.html

[ssborbis]

@ebluser

I looked at https://cima.aemps.es/cima/publico/buscadoravanzado.html and I don't think it can work with ContextSearch. It seems to use javascript to fetch .json using a REST API.

You could, for example, use a template like this:

https://cima.aemps.es/cima/rest/medicamentos?multiple2={searchTerms}&cargaprincipiosactivos=true&nomostrarip=1&comerc=1

... and search for ABFENTIQ to receive this result: https://cima.aemps.es/cima/rest/medicamentos?multiple2=ABFENTIQ&cargaprincipiosactivos=true&nomostrarip=1&comerc=1

... but that .json may not be helpful to you.

[ernestobarrera]

@ebluser

I looked at https://cima.aemps.es/cima/publico/buscadoravanzado.html and I don't think it can work with ContextSearch. It seems to use javascript to fetch .json using a REST API.

You could, for example, use a template like this:

https://cima.aemps.es/cima/rest/medicamentos?multiple2={searchTerms}&cargaprincipiosactivos=true&nomostrarip=1&comerc=1

... and search for ABFENTIQ to receive this result: https://cima.aemps.es/cima/rest/medicamentos?multiple2=ABFENTIQ&cargaprincipiosactivos=true&nomostrarip=1&comerc=1

... but that .json may not be helpful to you. Indeed, it is not. However, ContestSearch has been able to include other search engines with post method. If you could define technical specifications or prerequisites for ContestSearch, it might be useful if the search engine developers could include improvements. Thanks!

[ssborbis]

@ebluser

Indeed, it is not. However, ContestSearch has been able to include other search engines with post method. If you could define technical specifications or prerequisites for ContestSearch, it might be useful if the search engine developers could include improvements. Thanks!

I'm not seeing where that engine uses POST. I'm looking at the network logs and only see GET requests to the REST API. Do you have a POST request I could look at?

[ernestobarrera]

I'm not seeing where that engine uses POST. I'm looking at the network logs and only see GET requests to the REST API. Do you have a POST request I could look at?

Sorry, I misinterpreted the behaviour of this search engine. Thanks again for the analysis and for the firefox add-on.

[githubeing]

If you come across any sites / forms that don't work with the Add Custom Search function, list them here.

yandex.translate:

1. entered text
2. web page

[githubeing]

the ocr feature also doesn't work with contextsearch, but not sure if it is possible at all to make it work together. because yandex requires to upload the image from local computer, doesn't allow to paste url of image: https://translate.yandex.com/ocr

[ssborbis]

@githubeing

Looks like yandex is using some javascript-based forms I can't account for. Making a custom engine would be trivial though, based on the results url.

Template: https://translate.yandex.com/?lang=en-fr&text={seachTerms}

The OCR feature looks difficult to handle. There might be a bookmarlet-based solution though.

[githubeing]

If you come across any sites / forms that don't work with the Add Custom Search function, list them here.

another example: https://yandex.ru/images/ - search by image url (see screenshot)

i'm not asking you for help, i now that the correct template is: https://yandex.ru/images/search?url={searchTerms}&rpt=imageview but the "add custom search" feature detects it wrong: https://yandex.ru/images/?cbir-url={searchTerms}

i post this only because you asked it, maybe it'll be useful to development of the project

do you need such examples further?

[ssborbis]

@githubeing

Hmm, when I try adding a custom engine from https://yandex.ru/images/ I get https://yandex.ru/images/search?text={searchTerms} and everything works fine. Maybe a region / locale / language difference in the form url?

[githubeing]

you've just added the "find image by text" template, and i was talking about "find image by url"

notice the screenshot above

you have to click the camera icon first, then try to add the form "Введите адрес картинки"

[ssborbis]

@githubeing K I'll take a look

[githubeing]

see screencast with the wrong detection and how i fix it below: screencast-2019-03-14_05.32.45.zip

[ssborbis]

@githubeing

A bit off the original issue... I'm running into a bug where I can only get the context menu option Add Custom Search for the image url search if I first bring up the context menu on the main search bar. If I reload the page, click the camera, and try to go straight to the image url search, the Add Custom Search menu entry fails to load. It's very bizarre. Are you seeing that too?

[ssborbis]

@githubeing

I posted before I watched your video. Yep, that's what I'm seeing

[ssborbis]

@githubeing

Ok, I've got 3 issues going on.

1. My MutationObserver for dynamically adding listeners to INPUT elements wasn't seeing children of added nodes, and missed seeing the image url search bar added to the page when clicking the camera. Fixed

2. The Add Custom Search menu entry was not being removed from the context menu properly, meaning it could be added by the first search bar and hang around to be used on the second image url search bar. This was also causing a 2-tiered menu in text boxes when trying to search from the context menu in some cases. Fixed

3. Yandex using some irregular javascript + form to build the search params. I might not be able to find a generic workaround for this, but getting the first 2 bugs fixed was worth it.

(edit) I also noticed the Test button on the custom engine form wasn't working Fixed

[githubeing]

on emojipedia.org the add custom search item doesn't show up at all in the context menu almost all the time. but sometimes it does. very rarely, so rarely that it becomes really hard to add the search engine because of this (not using manual adding of course). see screenshots

[ssborbis]

@githubeing

I'm showing the menu every time, chrome and FF. What's your hardware?

[ssborbis]

@githubeing

Also, do you get better results if you left-click to focus on the input first, then right-click?

[githubeing]

not sure what you mean by "what's your hardware". what steps do you expect me to do to answer this your question? my hardware is a desktop pc.

no, i don't get better results when focusing by left-clicking first - i get same results

[ssborbis]

I mean what's your processor, ram. I'm trying to figure out if I'm getting different results because of a difference in processing speeds. There could be a timing issue with code execution.

[ssborbis]

@githubeing

I'm seeing it now. This might be related to the Search with ... issue as well. I'll keep you posted

[githubeing]

cpu isn't very slow, ram is enough imo do you think this could be the cause?

[ssborbis]

cpu isn't very slow, ram is enough imo, take a look: do you think this could be the cause?

There's a bit of tricky timing I'm having to deal with to make the correct menus show up in the context menu, but I should be able to find a suitable workaround. I'm rewriting that bit of code and you can try it out when I release the next patch. I'm still not getting the menus to work 100% on my machine, but it's much better than before.

[githubeing]

any sites / forms that don't work with the Add Custom Search function

another site: https://context.reverso.net the SE would be e.g. [GET] https://context.reverso.net/translation/english-spanish/{searchTerms}

[ssborbis]

@githubeing Thanks. Got it working with a simple fix. I'm sure it was causing failure elsewhere