Prevent empty UI in Spotlight, Top Sites

[pdehaan]

Currently it is too easy to get empty Spotlight/Featured/Highlight boxes if you delete/hide items or in other cases.

This will be partially mitigated somewhat on a new profile via #200, but we should come up with an alternate solution like re-querying Firefox/Embedly on item delete, or in the case of not enough data, merging it with the default data set from #200.

[nchapman]

It could be really interesting for us to use this tool from embedly to create a query that would show content results rather than "recent top sites".

[nchapman]

Here's a rather epic (and obviously unoptimized) query based on the embedly tool with some additional news sites added for good measure:

SELECT DISTINCT
  p.id, p.url
FROM
  moz_historyvisits hv
INNER JOIN
  moz_places p ON p.id = hv.place_id
WHERE
  -- http://embed.ly/tools/generator -- all minus google maps
  p.url LIKE '%yfrog.%/%' OR p.url LIKE '%www.flickr.com/photos/%' OR p.url LIKE '%flic.kr/%' OR p.url LIKE '%twitpic.com/%' OR p.url LIKE '%www.twitpic.com/%' OR p.url LIKE '%twitpic.com/photos/%' OR p.url LIKE '%www.twitpic.com/photos/%' OR p.url LIKE '%imgur.com/%' OR p.url LIKE '%twitgoo.com/%' OR p.url LIKE '%i%.photobucket.com/albums/%' OR p.url LIKE '%s%.photobucket.com/albums/%' OR p.url LIKE '%media.photobucket.com/image/%' OR p.url LIKE '%www.mobypicture.com/user/%/view/%' OR p.url LIKE '%moby.to/%' OR p.url LIKE '%xkcd.com/%' OR p.url LIKE '%www.xkcd.com/%' OR p.url LIKE '%imgs.xkcd.com/%' OR p.url LIKE '%www.asofterworld.com/index.php?id=%' OR p.url LIKE '%www.asofterworld.com/%.jpg' OR p.url LIKE '%asofterworld.com/%.jpg' OR p.url LIKE '%www.qwantz.com/index.php?comic=%' OR p.url LIKE '%23hq.com/%/photo/%' OR p.url LIKE '%www.23hq.com/%/photo/%' OR p.url LIKE '%dribbble.com/shots/%' OR p.url LIKE '%drbl.in/%' OR p.url LIKE '%.smugmug.com/%' OR p.url LIKE '%.smugmug.com/%#%' OR p.url LIKE '%img.ly/%' OR p.url LIKE '%www.tinypic.com/view.php%' OR p.url LIKE '%tinypic.com/view.php%' OR p.url LIKE '%www.tinypic.com/player.php%' OR p.url LIKE '%tinypic.com/player.php%' OR p.url LIKE '%www.tinypic.com/r/%/%' OR p.url LIKE '%tinypic.com/r/%/%' OR p.url LIKE '%.tinypic.com/%.jpg' OR p.url LIKE '%.tinypic.com/%.png' OR p.url LIKE '%meadd.com/%/%' OR p.url LIKE '%meadd.com/%' OR p.url LIKE '%.deviantart.com/art/%' OR p.url LIKE '%.deviantart.com/gallery/%' OR p.url LIKE '%.deviantart.com/#/%' OR p.url LIKE '%fav.me/%' OR p.url LIKE '%.deviantart.com' OR p.url LIKE '%.deviantart.com/gallery' OR p.url LIKE '%.deviantart.com/%/%.jpg' OR p.url LIKE '%.deviantart.com/%/%.gif' OR p.url LIKE '%.deviantart.net/%/%.jpg' OR p.url LIKE '%.deviantart.net/%/%.gif' OR p.url LIKE '%www.fotopedia.com/%/%' OR p.url LIKE '%fotopedia.com/%/%' OR p.url LIKE '%photozou.jp/photo/show/%/%' OR p.url LIKE '%photozou.jp/photo/photo_only/%/%' OR p.url LIKE '%instagr.am/p/%' OR p.url LIKE '%instagram.com/p/%' OR p.url LIKE '%www.instagram.com/p/%' OR p.url LIKE '%skitch.com/%/%/%' OR p.url LIKE '%img.skitch.com/%' OR p.url LIKE '%www.questionablecontent.net/' OR p.url LIKE '%questionablecontent.net/' OR p.url LIKE '%www.questionablecontent.net/view.php%' OR p.url LIKE '%questionablecontent.net/view.php%' OR p.url LIKE '%questionablecontent.net/comics/%.png' OR p.url LIKE '%www.questionablecontent.net/comics/%.png' OR p.url LIKE '%twitrpix.com/%' OR p.url LIKE '%.twitrpix.com/%' OR p.url LIKE '%www.someecards.com/%/%' OR p.url LIKE '%someecards.com/%/%' OR p.url LIKE '%some.ly/%' OR p.url LIKE '%www.some.ly/%' OR p.url LIKE '%pikchur.com/%' OR p.url LIKE '%achewood.com/%' OR p.url LIKE '%www.achewood.com/%' OR p.url LIKE '%achewood.com/index.php%' OR p.url LIKE '%www.achewood.com/index.php%' OR p.url LIKE '%www.whosay.com/%/content/%' OR p.url LIKE '%www.whosay.com/%/photos/%' OR p.url LIKE '%www.whosay.com/%/videos/%' OR p.url LIKE '%say.ly/%' OR p.url LIKE '%ow.ly/i/%' OR p.url LIKE '%mlkshk.com/p/%' OR p.url LIKE '%lockerz.com/s/%' OR p.url LIKE '%pics.lockerz.com/s/%' OR p.url LIKE '%d.pr/i/%' OR p.url LIKE '%www.eyeem.com/p/%' OR p.url LIKE '%www.eyeem.com/a/%' OR p.url LIKE '%www.eyeem.com/u/%' OR p.url LIKE '%giphy.com/gifs/%' OR p.url LIKE '%gph.is/%' OR p.url LIKE '%frontback.me/p/%' OR p.url LIKE '%www.fotokritik.com/%/%' OR p.url LIKE '%fotokritik.com/%/%' OR p.url LIKE '%vid.me/%' OR p.url LIKE '%galeri.uludagsozluk.com/%' OR p.url LIKE '%gfycat.com/%' OR p.url LIKE '%tochka.net/%' OR p.url LIKE '%.tochka.net/%' OR p.url LIKE '%4cook.net/recipe/%' OR p.url LIKE '%www.alphahat.com/view/%' OR p.url LIKE '%alphahat.com/view/%' OR p.url LIKE '%gist.github.com/%' OR p.url LIKE '%www.slideshare.net/%/%' OR p.url LIKE '%www.slideshare.net/mobile/%/%' OR p.url LIKE '%.slideshare.net/%/%' OR p.url LIKE '%slidesha.re/%' OR p.url LIKE '%scribd.com/doc/%' OR p.url LIKE '%www.scribd.com/doc/%' OR p.url LIKE '%scribd.com/mobile/documents/%' OR p.url LIKE '%www.scribd.com/mobile/documents/%' OR p.url LIKE '%screenr.com/%' OR p.url LIKE '%polldaddy.com/community/poll/%' OR p.url LIKE '%polldaddy.com/poll/%' OR p.url LIKE '%answers.polldaddy.com/poll/%' OR p.url LIKE '%www.howcast.com/videos/%' OR p.url LIKE '%www.screencast.com/%/media/%' OR p.url LIKE '%screencast.com/%/media/%' OR p.url LIKE '%www.screencast.com/t/%' OR p.url LIKE '%screencast.com/t/%' OR p.url LIKE '%issuu.com/%/docs/%' OR p.url LIKE '%www.kickstarter.com/projects/%/%' OR p.url LIKE '%www.scrapblog.com/viewer/viewer.aspx%' OR p.url LIKE '%foursquare.com/%' OR p.url LIKE '%www.foursquare.com/%' OR p.url LIKE '%4sq.com/%' OR p.url LIKE '%linkedin.com/in/%' OR p.url LIKE '%linkedin.com/pub/%' OR p.url LIKE '%.linkedin.com/in/%' OR p.url LIKE '%.linkedin.com/pub/%' OR p.url LIKE '%linkedin.com/in/%' OR p.url LIKE '%linkedin.com/company/%' OR p.url LIKE '%.linkedin.com/company/%' OR p.url LIKE '%www.sliderocket.com/%' OR p.url LIKE '%sliderocket.com/%' OR p.url LIKE '%app.sliderocket.com/%' OR p.url LIKE '%portal.sliderocket.com/%' OR p.url LIKE '%beta-sliderocket.com/%' OR p.url LIKE '%my.opera.com/%/albums/show.dml?id=%' OR p.url LIKE '%my.opera.com/%/albums/showpic.dml?album=%&picture=%' OR p.url LIKE '%tumblr.com/%' OR p.url LIKE '%.tumblr.com/post/%' OR p.url LIKE '%www.polleverywhere.com/polls/%' OR p.url LIKE '%www.polleverywhere.com/multiple_choice_polls/%' OR p.url LIKE '%www.polleverywhere.com/free_text_polls/%' OR p.url LIKE '%www.quantcast.com/wd:%' OR p.url LIKE '%www.quantcast.com/%' OR p.url LIKE '%siteanalytics.compete.com/%' OR p.url LIKE '%.status.net/notice/%' OR p.url LIKE '%identi.ca/notice/%' OR p.url LIKE '%www.studivz.net/Profile/%' OR p.url LIKE '%www.studivz.net/l/%' OR p.url LIKE '%www.studivz.net/Groups/Overview/%' OR p.url LIKE '%www.studivz.net/Gadgets/Info/%' OR p.url LIKE '%www.studivz.net/Gadgets/Install/%' OR p.url LIKE '%www.studivz.net/%' OR p.url LIKE '%www.meinvz.net/Profile/%' OR p.url LIKE '%www.meinvz.net/l/%' OR p.url LIKE '%www.meinvz.net/Groups/Overview/%' OR p.url LIKE '%www.meinvz.net/Gadgets/Info/%' OR p.url LIKE '%www.meinvz.net/Gadgets/Install/%' OR p.url LIKE '%www.meinvz.net/%' OR p.url LIKE '%www.schuelervz.net/Profile/%' OR p.url LIKE '%www.schuelervz.net/l/%' OR p.url LIKE '%www.schuelervz.net/Groups/Overview/%' OR p.url LIKE '%www.schuelervz.net/Gadgets/Info/%' OR p.url LIKE '%www.schuelervz.net/Gadgets/Install/%' OR p.url LIKE '%www.schuelervz.net/%' OR p.url LIKE '%myloc.me/%' OR p.url LIKE '%pastebin.com/%' OR p.url LIKE '%pastie.org/%' OR p.url LIKE '%www.pastie.org/%' OR p.url LIKE '%redux.com/stream/item/%/%' OR p.url LIKE '%redux.com/f/%/%' OR p.url LIKE '%www.redux.com/stream/item/%/%' OR p.url LIKE '%www.redux.com/f/%/%' OR p.url LIKE '%cl.ly/%' OR p.url LIKE '%cl.ly/%/content' OR p.url LIKE '%speakerdeck.com/%/%' OR p.url LIKE '%www.kiva.org/lend/%' OR p.url LIKE '%www.timetoast.com/timelines/%' OR p.url LIKE '%storify.com/%/%' OR p.url LIKE '%meetup.com/%' OR p.url LIKE '%meetu.ps/%' OR p.url LIKE '%www.dailymile.com/people/%/entries/%' OR p.url LIKE '%.kinomap.com/%' OR p.url LIKE '%www.metacdn.com/r/c/%/%' OR p.url LIKE '%www.metacdn.com/r/m/%/%' OR p.url LIKE '%prezi.com/%/%' OR p.url LIKE '%.uservoice.com/%/suggestions/%' OR p.url LIKE '%formspring.me/%' OR p.url LIKE '%www.formspring.me/%' OR p.url LIKE '%formspring.me/%/q/%' OR p.url LIKE '%www.formspring.me/%/q/%' OR p.url LIKE '%twitlonger.com/show/%' OR p.url LIKE '%www.twitlonger.com/show/%' OR p.url LIKE '%tl.gd/%' OR p.url LIKE '%www.qwiki.com/q/%' OR p.url LIKE '%crocodoc.com/%' OR p.url LIKE '%.crocodoc.com/%' OR p.url LIKE '%www.wikipedia.org/wiki/%' OR p.url LIKE '%.wikipedia.org/wiki/%' OR p.url LIKE '%www.wikimedia.org/wiki/File%' OR p.url LIKE '%360.io/%' OR p.url LIKE '%www.behance.net/gallery/%' OR p.url LIKE '%behance.net/gallery/%' OR p.url LIKE '%www.jdsupra.com/legalnews/%' OR p.url LIKE '%jdsupra.com/legalnews/%' OR p.url LIKE '%minilogs.com/%' OR p.url LIKE '%www.minilogs.com/%' OR p.url LIKE '%jsfiddle.net/%' OR p.url LIKE '%ponga.com/%' OR p.url LIKE '%list.ly/list/%' OR p.url LIKE '%crowdmap.com/post/%' OR p.url LIKE '%.crowdmap.com/post/%' OR p.url LIKE '%crowdmap.com/map/%' OR p.url LIKE '%.crowdmap.com/map/%' OR p.url LIKE '%ifttt.com/recipes/%' OR p.url LIKE '%weavly.com/watch/%' OR p.url LIKE '%www.weavly.com/watch/%' OR p.url LIKE '%tagmotion.com/tree/%' OR p.url LIKE '%www.tagmotion.com/tree/%' OR p.url LIKE '%public.talely.com/%/%' OR p.url LIKE '%polarb.com/%' OR p.url LIKE '%.polarb.com/%' OR p.url LIKE '%on.bubb.li/%' OR p.url LIKE '%bubb.li/%' OR p.url LIKE '%.bubb.li/%' OR p.url LIKE '%embed.imajize.com/%' OR p.url LIKE '%giflike.com/a/%' OR p.url LIKE '%www.giflike.com/a/%' OR p.url LIKE '%i.giflike.com/%' OR p.url LIKE '%rapidengage.com/s/%' OR p.url LIKE '%infomous.com/node/%' OR p.url LIKE '%stepic.org/%' OR p.url LIKE '%chirb.it/%' OR p.url LIKE '%beta.polstir.com/%/%' OR p.url LIKE '%polstir.com/%/%' OR p.url LIKE '%www.gettyimages.com/detail/photo/%' OR p.url LIKE '%gty.im/%' OR p.url LIKE '%isnare.com/%' OR p.url LIKE '%www.isnare.com/%' OR p.url LIKE '%www.branchtrack.com/projects/%' OR p.url LIKE '%jsbin.com/%/%' OR p.url LIKE '%jsbin.com/%' OR p.url LIKE '%wedgi.es/%' OR p.url LIKE '%public.chartblocks.com/c/%' OR p.url LIKE '%radd.it/r/%' OR p.url LIKE '%radd.it/comments/%' OR p.url LIKE '%radd.it/user/%' OR p.url LIKE '%radd.it/playlists/%' OR p.url LIKE '%radd.it/magic/%' OR p.url LIKE '%vibi.com/videocard/%' OR p.url LIKE '%kastio.com/webcasts/%' OR p.url LIKE '%.kastio.com/webcasts/%' OR p.url LIKE '%megavisor.com/view/%' OR p.url LIKE '%megavisor.com/en/view/%' OR p.url LIKE '%bunkrapp.com/%/%' OR p.url LIKE '%.cartodb.com/%/%' OR p.url LIKE '%flowvella.com/s/%' OR p.url LIKE '%fr.peoplbrain.com/tutoriaux/%' OR p.url LIKE '%codepicnic.com/bites/%' OR p.url LIKE '%codepicnic.com/consoles/%' OR p.url LIKE '%tr.instela.com/%' OR p.url LIKE '%codepen.io/%/pen/%' OR p.url LIKE '%codepen.io/%/pen/%' OR p.url LIKE '%www.vtility.net/virtualtour/%' OR p.url LIKE '%quora.com/%/answer/%' OR p.url LIKE '%www.quora.com/%/answer/%' OR p.url LIKE '%tunein.com/%' OR p.url LIKE '%tun.in/%' OR p.url LIKE '%scribblemaps.com/maps/view/%/%' OR p.url LIKE '%www.scribblemaps.com/maps/view/%/%' OR p.url LIKE '%www.codeply.com/view/%' OR p.url LIKE '%codeply.com/view/%' OR p.url LIKE '%www.candybank.com/%' OR p.url LIKE '%flat.io/score/%' OR p.url LIKE '%shorti.com/%' OR p.url LIKE '%www.shorti.com/%' OR p.url LIKE '%blab.im/%' OR p.url LIKE '%pollplug.com/poll/%' OR p.url LIKE '%alpha.vrchive.com/%' OR p.url LIKE '%www.globalgiving.org/projects/%' OR p.url LIKE '%www.globalgiving.org/funds/%' OR p.url LIKE '%www.globalgiving.org/microprojects/%' OR p.url LIKE '%www.newhive.com/%/%' OR p.url LIKE '%slidr.io/%/%' OR p.url LIKE '%publons.com/author/%' OR p.url LIKE '%www.publons.com/author/%' OR p.url LIKE '%calameo.com/%' OR p.url LIKE '%www.calameo.com/%' OR p.url LIKE '%relayto.com/%' OR p.url LIKE '%www.relayto.com/%' OR p.url LIKE '%w.graphiq.com/w/%' OR p.url LIKE '%view.stacker.cc/%' OR p.url LIKE '%content.newsbound.com/%/%' OR p.url LIKE '%.silk.co/explore/%' OR p.url LIKE '%docs.com/%' OR p.url LIKE '%rocketium.com/%' OR p.url LIKE '%cdn.knightlab.com/libs/timeline3/%' OR p.url LIKE '%cdn.knightlab.com/libs/juxtapose/%' OR p.url LIKE '%uploads.knightlab.com/storymapjs/%/index.html' OR p.url LIKE '%www.thelastgraph.com/lg.php?a=%' OR p.url LIKE '%thelastgraph.com/lg.php?a=%' OR p.url LIKE '%amazon.%/gp/product/%' OR p.url LIKE '%amazon.%/%/dp/%' OR p.url LIKE '%amazon.%/dp/%' OR p.url LIKE '%amazon.%/o/ASIN/%' OR p.url LIKE '%amazon.%/gp/offer-listing/%' OR p.url LIKE '%amazon.%/%/ASIN/%' OR p.url LIKE '%amazon.%/gp/product/images/%' OR p.url LIKE '%amazon.%/gp/aw/d/%' OR p.url LIKE '%www.amzn.com/%' OR p.url LIKE '%amzn.com/%' OR p.url LIKE '%www.shopstyle.com/browse%' OR p.url LIKE '%www.shopstyle.com/action/apiVisitRetailer%' OR p.url LIKE '%api.shopstyle.com/action/apiVisitRetailer%' OR p.url LIKE '%www.shopstyle.com/action/viewLook%' OR p.url LIKE '%itunes.apple.com/%' OR p.url LIKE '%shoplocket.com/products/%' OR p.url LIKE '%etsy.com/%' OR p.url LIKE '%www.etsy.com/%' OR p.url LIKE '%fiverr.com/%/%' OR p.url LIKE '%www.fiverr.com/%/%' OR p.url LIKE '%youtube.com/watch%' OR p.url LIKE '%.youtube.com/v/%' OR p.url LIKE '%youtu.be/%' OR p.url LIKE '%.youtube.com/user/%' OR p.url LIKE '%.youtube.com/%#%/%' OR p.url LIKE '%m.youtube.com/watch%' OR p.url LIKE '%m.youtube.com/index%' OR p.url LIKE '%.youtube.com/profile%' OR p.url LIKE '%.youtube.com/view_play_list%' OR p.url LIKE '%.youtube.com/playlist%' OR p.url LIKE '%www.youtube.com/embed/%' OR p.url LIKE '%youtube.com/gif%' OR p.url LIKE '%www.youtube.com/gif%' OR p.url LIKE '%www.youtube.com/attribution_link%' OR p.url LIKE '%youtube.com/attribution_link%' OR p.url LIKE '%youtube.ca/%' OR p.url LIKE '%youtube.jp/%' OR p.url LIKE '%youtube.com.br/%' OR p.url LIKE '%youtube.co.uk/%' OR p.url LIKE '%youtube.nl/%' OR p.url LIKE '%youtube.pl/%' OR p.url LIKE '%youtube.es/%' OR p.url LIKE '%youtube.ie/%' OR p.url LIKE '%it.youtube.com/%' OR p.url LIKE '%youtube.fr/%' OR p.url LIKE '%twitch.tv/%' OR p.url LIKE '%twitch.tv/%/b/%' OR p.url LIKE '%www.ustream.tv/recorded/%' OR p.url LIKE '%www.ustream.tv/channel/%' OR p.url LIKE '%www.ustream.tv/%' OR p.url LIKE '%ustre.am/%' OR p.url LIKE '%qik.com/video/%' OR p.url LIKE '%qik.com/%' OR p.url LIKE '%qik.ly/%' OR p.url LIKE '%revision3.com/%' OR p.url LIKE '%.dailymotion.com/video/%' OR p.url LIKE '%.dailymotion.com/%/video/%' OR p.url LIKE '%collegehumor.com/video:%' OR p.url LIKE '%collegehumor.com/video/%' OR p.url LIKE '%www.collegehumor.com/video:%' OR p.url LIKE '%www.collegehumor.com/video/%' OR p.url LIKE '%telly.com/%' OR p.url LIKE '%www.telly.com/%' OR p.url LIKE '%break.com/%/%' OR p.url LIKE '%www.break.com/%/%' OR p.url LIKE '%vids.myspace.com/index.cfm?fuseaction=vids.individual&videoid%' OR p.url LIKE '%www.myspace.com/index.cfm?fuseaction=%&videoid%' OR p.url LIKE '%www.metacafe.com/watch/%' OR p.url LIKE '%www.metacafe.com/w/%' OR p.url LIKE '%blip.tv/%/%' OR p.url LIKE '%.blip.tv/%/%' OR p.url LIKE '%video.google.com/videoplay?%' OR p.url LIKE '%viddler.com/v/%' OR p.url LIKE '%liveleak.com/view?%' OR p.url LIKE '%www.liveleak.com/view?%' OR p.url LIKE '%animoto.com/play/%' OR p.url LIKE '%video214.com/play/%' OR p.url LIKE '%dotsub.com/view/%' OR p.url LIKE '%www.overstream.net/view.php?oid=%' OR p.url LIKE '%www.livestream.com/%' OR p.url LIKE '%new.livestream.com/%' OR p.url LIKE '%www.worldstarhiphop.com/videos/video%.php?v=%' OR p.url LIKE '%worldstarhiphop.com/videos/video%.php?v=%' OR p.url LIKE '%bambuser.com/v/%' OR p.url LIKE '%bambuser.com/channel/%' OR p.url LIKE '%bambuser.com/channel/%/broadcast/%' OR p.url LIKE '%www.schooltube.com/video/%/%' OR p.url LIKE '%sendables.jibjab.com/view/%' OR p.url LIKE '%sendables.jibjab.com/originals/%' OR p.url LIKE '%jibjab.com/view/%' OR p.url LIKE '%www.xtranormal.com/watch/%' OR p.url LIKE '%socialcam.com/v/%' OR p.url LIKE '%www.socialcam.com/v/%' OR p.url LIKE '%v.youku.com/v_show/%' OR p.url LIKE '%v.youku.com/v_playlist/%' OR p.url LIKE '%www.snotr.com/video/%' OR p.url LIKE '%snotr.com/video/%' OR p.url LIKE '%www.clipfish.de/%/%/video/%' OR p.url LIKE '%www.myvideo.de/watch/%' OR p.url LIKE '%www.vzaar.com/videos/%' OR p.url LIKE '%vzaar.com/videos/%' OR p.url LIKE '%www.vzaar.tv/%' OR p.url LIKE '%vzaar.tv/%' OR p.url LIKE '%vzaar.me/%' OR p.url LIKE '%.vzaar.me/%' OR p.url LIKE '%coub.com/view/%' OR p.url LIKE '%coub.com/embed/%' OR p.url LIKE '%www.streamio.com/api/v1/%' OR p.url LIKE '%streamio.com/api/v1/%' OR p.url LIKE '%vine.co/v/%' OR p.url LIKE '%www.vine.co/v/%' OR p.url LIKE '%www.viddy.com/video/%' OR p.url LIKE '%www.viddy.com/%/v/%' OR p.url LIKE '%www.tudou.com/programs/view/%' OR p.url LIKE '%tudou.com/programs/view/%' OR p.url LIKE '%sproutvideo.com/videos/%' OR p.url LIKE '%embed.minoto-video.com/%' OR p.url LIKE '%iframe.minoto-video.com/%' OR p.url LIKE '%play.minoto-video.com/%' OR p.url LIKE '%dashboard.minoto-video.com/main/video/details/%' OR p.url LIKE '%api.minoto-video.com/publishers/%/videos/%' OR p.url LIKE '%www.brainshark.com/%/%' OR p.url LIKE '%brainshark.com/%/%' OR p.url LIKE '%23video.com/%' OR p.url LIKE '%.23video.com/%' OR p.url LIKE '%goanimate.com/videos/%' OR p.url LIKE '%brainsonic.com/%' OR p.url LIKE '%.brainsonic.com/%' OR p.url LIKE '%lustich.de/videos/%' OR p.url LIKE '%web.tv/%' OR p.url LIKE '%.web.tv/%' OR p.url LIKE '%mynet.com/video/%' OR p.url LIKE '%www.mynet.com/video/' OR p.url LIKE '%izlesene.com/video/%' OR p.url LIKE '%www.izlesene.com/video/' OR p.url LIKE '%alkislarlayasiyorum.com/%' OR p.url LIKE '%www.alkislarlayasiyorum.com/%' OR p.url LIKE '%59saniye.com/%' OR p.url LIKE '%www.59saniye.com/%' OR p.url LIKE '%zie.nl/video/%' OR p.url LIKE '%www.zie.nl/video/%' OR p.url LIKE '%app.ustudio.com/embed/%/%' OR p.url LIKE '%.kaltura.com/%' OR p.url LIKE '%bale.io/%' OR p.url LIKE '%www.allego.com/%' OR p.url LIKE '%clipter.com/c/%' OR p.url LIKE '%app.stghv.com/%' OR p.url LIKE '%sendvid.com/%' OR p.url LIKE '%www.snappytv.com/%' OR p.url LIKE '%snappytv.com/%' OR p.url LIKE '%frankly.me/%' OR p.url LIKE '%streamable.com/%' OR p.url LIKE '%ticker.tv/v/%' OR p.url LIKE '%videobio.com/playerjs/%' OR p.url LIKE '%clippituser.tv/%' OR p.url LIKE '%www.clippituser.tv/%' OR p.url LIKE '%my.storygami.com/video/%' OR p.url LIKE '%player.videopath.com/%' OR p.url LIKE '%.hosted.panopto.com/%' OR p.url LIKE '%www.dreambroker.com/channel/%' OR p.url LIKE '%dreambroker.com/channel/%' OR p.url LIKE '%www.oumy.com/v/%' OR p.url LIKE '%oumy.com/v/%' OR p.url LIKE '%www.touchcast.com/%/%' OR p.url LIKE '%content.streamonecloud.net/embed/%' OR p.url LIKE '%skip.st/one/%' OR p.url LIKE '%www.whitehouse.gov/photos-and-video/video/%' OR p.url LIKE '%www.whitehouse.gov/video/%' OR p.url LIKE '%wh.gov/photos-and-video/video/%' OR p.url LIKE '%wh.gov/video/%' OR p.url LIKE '%www.hulu.com/watch%' OR p.url LIKE '%www.hulu.com/w/%' OR p.url LIKE '%www.hulu.com/embed/%' OR p.url LIKE '%hulu.com/watch%' OR p.url LIKE '%hulu.com/w/%' OR p.url LIKE '%hulu.tv/%' OR p.url LIKE '%crackle.com/c/%' OR p.url LIKE '%www.funnyordie.com/videos/%' OR p.url LIKE '%www.funnyordie.com/m/%' OR p.url LIKE '%funnyordie.com/videos/%' OR p.url LIKE '%funnyordie.com/m/%' OR p.url LIKE '%www.vimeo.com/groups/%/videos/%' OR p.url LIKE '%www.vimeo.com/%' OR p.url LIKE '%vimeo.com/groups/%/videos/%' OR p.url LIKE '%vimeo.com/%' OR p.url LIKE '%vimeo.com/m/#/%' OR p.url LIKE '%player.vimeo.com/%' OR p.url LIKE '%www.ted.com/talks/%.html%' OR p.url LIKE '%www.ted.com/talks/lang/%/%.html%' OR p.url LIKE '%www.ted.com/index.php/talks/%.html%' OR p.url LIKE '%www.ted.com/index.php/talks/lang/%/%.html%' OR p.url LIKE '%www.ted.com/talks/' OR p.url LIKE '%nfb.ca/film/%' OR p.url LIKE '%thedailyshow.cc.com/videos/%' OR p.url LIKE '%www.thedailyshow.com/watch/%' OR p.url LIKE '%www.thedailyshow.com/full-episodes/%' OR p.url LIKE '%www.thedailyshow.com/collection/%/%/%' OR p.url LIKE '%yahoo.com/movies/%' OR p.url LIKE '%.yahoo.com/movies/%' OR p.url LIKE '%thecolbertreport.cc.com/videos/%' OR p.url LIKE '%www.colbertnation.com/the-colbert-report-collections/%' OR p.url LIKE '%www.colbertnation.com/full-episodes/%' OR p.url LIKE '%www.colbertnation.com/the-colbert-report-videos/%' OR p.url LIKE '%www.comedycentral.com/videos/index.jhtml?%' OR p.url LIKE '%www.theonion.com/video/%' OR p.url LIKE '%theonion.com/video/%' OR p.url LIKE '%wordpress.tv/%/%/%/%/' OR p.url LIKE '%www.traileraddict.com/trailer/%' OR p.url LIKE '%www.traileraddict.com/clip/%' OR p.url LIKE '%www.traileraddict.com/poster/%' OR p.url LIKE '%www.trailerspy.com/trailer/%/%' OR p.url LIKE '%www.trailerspy.com/trailer/%' OR p.url LIKE '%www.trailerspy.com/view_video.php%' OR p.url LIKE '%fora.tv/%/%/%/%' OR p.url LIKE '%www.spike.com/video/%' OR p.url LIKE '%www.gametrailers.com/video%' OR p.url LIKE '%gametrailers.com/video%' OR p.url LIKE '%www.koldcast.tv/video/%' OR p.url LIKE '%www.koldcast.tv/#video:%' OR p.url LIKE '%mixergy.com/%' OR p.url LIKE '%video.pbs.org/video/%' OR p.url LIKE '%www.zapiks.com/%' OR p.url LIKE '%www.trutv.com/video/%' OR p.url LIKE '%www.nzonscreen.com/title/%' OR p.url LIKE '%nzonscreen.com/title/%' OR p.url LIKE '%app.wistia.com/embed/medias/%' OR p.url LIKE '%wistia.com/%' OR p.url LIKE '%.wistia.com/%' OR p.url LIKE '%.wi.st/%' OR p.url LIKE '%wi.st/%' OR p.url LIKE '%confreaks.net/videos/%' OR p.url LIKE '%www.confreaks.net/videos/%' OR p.url LIKE '%confreaks.com/videos/%' OR p.url LIKE '%www.confreaks.com/videos/%' OR p.url LIKE '%video.allthingsd.com/video/%' OR p.url LIKE '%videos.nymag.com/%' OR p.url LIKE '%aniboom.com/animation-video/%' OR p.url LIKE '%www.aniboom.com/animation-video/%' OR p.url LIKE '%grindtv.com/%/video/%' OR p.url LIKE '%www.grindtv.com/%/video/%' OR p.url LIKE '%ifood.tv/recipe/%' OR p.url LIKE '%ifood.tv/video/%' OR p.url LIKE '%ifood.tv/channel/user/%' OR p.url LIKE '%www.ifood.tv/recipe/%' OR p.url LIKE '%www.ifood.tv/video/%' OR p.url LIKE '%www.ifood.tv/channel/user/%' OR p.url LIKE '%logotv.com/video/%' OR p.url LIKE '%www.logotv.com/video/%' OR p.url LIKE '%lonelyplanet.com/Clip.aspx?%' OR p.url LIKE '%www.lonelyplanet.com/Clip.aspx?%' OR p.url LIKE '%streetfire.net/video/%.htm%' OR p.url LIKE '%www.streetfire.net/video/%.htm%' OR p.url LIKE '%sciencestage.com/v/%.html' OR p.url LIKE '%sciencestage.com/a/%.html' OR p.url LIKE '%www.sciencestage.com/v/%.html' OR p.url LIKE '%www.sciencestage.com/a/%.html' OR p.url LIKE '%link.brightcove.com/services/player/bcpid%' OR p.url LIKE '%bcove.me/%' OR p.url LIKE '%wirewax.com/%' OR p.url LIKE '%www.wirewax.com/%' OR p.url LIKE '%canalplus.fr/%' OR p.url LIKE '%www.canalplus.fr/%' OR p.url LIKE '%www.vevo.com/watch/%' OR p.url LIKE '%www.vevo.com/video/%' OR p.url LIKE '%pixorial.com/watch/%' OR p.url LIKE '%www.pixorial.com/watch/%' OR p.url LIKE '%spreecast.com/events/%' OR p.url LIKE '%www.spreecast.com/events/%' OR p.url LIKE '%showme.com/sh/%' OR p.url LIKE '%www.showme.com/sh/%' OR p.url LIKE '%.looplogic.com/%' OR p.url LIKE '%on.aol.com/video/%' OR p.url LIKE '%on.aol.com/playlist/%' OR p.url LIKE '%videodetective.com/%/%' OR p.url LIKE '%www.videodetective.com/%/%' OR p.url LIKE '%khanacademy.org/%' OR p.url LIKE '%www.khanacademy.org/%' OR p.url LIKE '%vidyard.com/%' OR p.url LIKE '%www.veoh.com/watch/%' OR p.url LIKE '%veoh.com/watch/%' OR p.url LIKE '%.univision.com/%/video/%' OR p.url LIKE '%.vidcaster.com/%' OR p.url LIKE '%muzu.tv/%' OR p.url LIKE '%www.muzu.tv/%' OR p.url LIKE '%vube.com/%/%' OR p.url LIKE '%www.vube.com/%/%' OR p.url LIKE '%boxofficebuz.com/video/%' OR p.url LIKE '%www.godtube.com/featured/video/%' OR p.url LIKE '%godtube.com/featured/video/%' OR p.url LIKE '%www.godtube.com/watch/%' OR p.url LIKE '%godtube.com/watch/%' OR p.url LIKE '%mediamatters.org/mmtv/%' OR p.url LIKE '%www.clikthrough.com/theater/video/%' OR p.url LIKE '%www.clipsyndicate.com/video/playlist/%/%' OR p.url LIKE '%www.srf.ch/play/%/%/%/%?id=%' OR p.url LIKE '%www.mpora.com/videos/%' OR p.url LIKE '%mpora.com/videos/%' OR p.url LIKE '%vice.com/%' OR p.url LIKE '%www.vice.com/%' OR p.url LIKE '%videodonor.com/video/%' OR p.url LIKE '%api.lovelive.tv/v1/%' OR p.url LIKE '%www.hurriyettv.com/%' OR p.url LIKE '%www.hurriyettv.com/%' OR p.url LIKE '%video.uludagsozluk.com/%' OR p.url LIKE '%www.ign.com/videos/%' OR p.url LIKE '%ign.com/videos/%' OR p.url LIKE '%www.askmen.com/video/%' OR p.url LIKE '%askmen.com/video/%' OR p.url LIKE '%video.esri.com/%' OR p.url LIKE '%arcg.is/%' OR p.url LIKE '%www.zapkolik.com/video/%' OR p.url LIKE '%.iplayerhd.com/playerframe/%' OR p.url LIKE '%.iplayerhd.com/player/video/%' OR p.url LIKE '%plays.tv/video/%' OR p.url LIKE '%app.videocheckout.com/embed/%' OR p.url LIKE '%www.lynda.com/%' OR p.url LIKE '%lynda.com/%' OR p.url LIKE '%v.embedly.com/%' OR p.url LIKE '%espn.go.com/video/clip%' OR p.url LIKE '%espn.go.com/%/story%' OR p.url LIKE '%abcnews.com/%/video/%' OR p.url LIKE '%abcnews.com/video/playerIndex%' OR p.url LIKE '%abcnews.go.com/%/video/%' OR p.url LIKE '%abcnews.go.com/video/playerIndex%' OR p.url LIKE '%washingtonpost.com/wp-dyn/%/video/%/%/%/%' OR p.url LIKE '%www.washingtonpost.com/wp-dyn/%/video/%/%/%/%' OR p.url LIKE '%www.boston.com/video%' OR p.url LIKE '%boston.com/video%' OR p.url LIKE '%www.boston.com/%video%' OR p.url LIKE '%boston.com/%video%' OR p.url LIKE '%www.facebook.com/photo.php%' OR p.url LIKE '%www.facebook.com/video.php%' OR p.url LIKE '%www.facebook.com/%/posts/%' OR p.url LIKE '%fb.me/%' OR p.url LIKE '%www.facebook.com/%/photos/%' OR p.url LIKE '%www.facebook.com/%/videos/%' OR p.url LIKE '%fb.com' OR p.url LIKE '%cnbc.com/id/%?%video%' OR p.url LIKE '%www.cnbc.com/id/%?%video%' OR p.url LIKE '%cnbc.com/id/%/play/1/video/%' OR p.url LIKE '%www.cnbc.com/id/%/play/1/video/%' OR p.url LIKE '%cnb.cx/%' OR p.url LIKE '%cbsnews.com/video/watch/%' OR p.url LIKE '%plus.google.com/%' OR p.url LIKE '%www.google.com/profiles/%' OR p.url LIKE '%google.com/profiles/%' OR p.url LIKE '%www.cnn.com/video/%' OR p.url LIKE '%www.cnn.com/videos/%' OR p.url LIKE '%www.cnn.com/videos/%' OR p.url LIKE '%cnn.it/%' OR p.url LIKE '%edition.cnn.com/video/%' OR p.url LIKE '%edition.cnn.com/videos/%' OR p.url LIKE '%money.cnn.com/video/%' OR p.url LIKE '%money.cnn.com/videos/%' OR p.url LIKE '%today.msnbc.msn.com/id/%/vp/%' OR p.url LIKE '%www.msnbc.msn.com/id/%/vp/%' OR p.url LIKE '%www.msnbc.msn.com/id/%/ns/%' OR p.url LIKE '%today.msnbc.msn.com/id/%/ns/%' OR p.url LIKE '%msnbc.msn.com/%/watch/%' OR p.url LIKE '%www.msnbc.msn.com/%/watch/%' OR p.url LIKE '%nbcnews.com/%' OR p.url LIKE '%www.nbcnews.com/%' OR p.url LIKE '%multimedia.foxsports.com/m/video/%/%' OR p.url LIKE '%msn.foxsports.com/video%' OR p.url LIKE '%www.globalpost.com/video/%' OR p.url LIKE '%www.globalpost.com/dispatch/%' OR p.url LIKE '%theguardian.com/%/video/%/%/%/%' OR p.url LIKE '%www.theguardian.com/%/video/%/%/%/%' OR p.url LIKE '%guardian.co.uk/%/video/%/%/%/%' OR p.url LIKE '%www.guardian.co.uk/%/video/%/%/%/%' OR p.url LIKE '%bravotv.com/%/%/videos/%' OR p.url LIKE '%www.bravotv.com/%/%/videos/%' OR p.url LIKE '%dsc.discovery.com/videos/%' OR p.url LIKE '%animal.discovery.com/videos/%' OR p.url LIKE '%health.discovery.com/videos/%' OR p.url LIKE '%investigation.discovery.com/videos/%' OR p.url LIKE '%military.discovery.com/videos/%' OR p.url LIKE '%planetgreen.discovery.com/videos/%' OR p.url LIKE '%science.discovery.com/videos/%' OR p.url LIKE '%tlc.discovery.com/videos/%' OR p.url LIKE '%video.forbes.com/fvn/%' OR p.url LIKE '%distrify.com/film/%' OR p.url LIKE '%muvi.es/%' OR p.url LIKE '%video.foxnews.com/v/%' OR p.url LIKE '%video.foxbusiness.com/v/%' OR p.url LIKE '%www.reuters.com/video/%' OR p.url LIKE '%reuters.com/video/%' OR p.url LIKE '%live.huffingtonpost.com/r/segment/%/%' OR p.url LIKE '%video.nytimes.com/video/%' OR p.url LIKE '%www.nytimes.com/video/%/%' OR p.url LIKE '%nyti.ms/%' OR p.url LIKE '%www.vol.at/video/%' OR p.url LIKE '%vol.at/video/%' OR p.url LIKE '%www.spiegel.de/video/%' OR p.url LIKE '%spiegel.de/video/%' OR p.url LIKE '%www.zeit.de/video/%' OR p.url LIKE '%zeit.de/video/%' OR p.url LIKE '%www.rts.ch/play/tv/%' OR p.url LIKE '%soundcloud.com/%' OR p.url LIKE '%soundcloud.com/%/%' OR p.url LIKE '%soundcloud.com/%/sets/%' OR p.url LIKE '%soundcloud.com/groups/%' OR p.url LIKE '%snd.sc/%' OR p.url LIKE '%open.spotify.com/%' OR p.url LIKE '%spoti.fi/%' OR p.url LIKE '%play.spotify.com/%' OR p.url LIKE '%www.last.fm/music/%' OR p.url LIKE '%www.last.fm/music/+videos/%' OR p.url LIKE '%www.last.fm/music/+images/%' OR p.url LIKE '%www.last.fm/music/%/_/%' OR p.url LIKE '%www.last.fm/music/%/%' OR p.url LIKE '%www.mixcloud.com/%/%/' OR p.url LIKE '%www.hark.com/clips/%' OR p.url LIKE '%www.rdio.com/#/artist/%/album/%' OR p.url LIKE '%www.rdio.com/artist/%/album/%' OR p.url LIKE '%www.zero-inch.com/%' OR p.url LIKE '%.bandcamp.com/' OR p.url LIKE '%.bandcamp.com/track/%' OR p.url LIKE '%.bandcamp.com/album/%' OR p.url LIKE '%freemusicarchive.org/music/%' OR p.url LIKE '%www.freemusicarchive.org/music/%' OR p.url LIKE '%freemusicarchive.org/curator/%' OR p.url LIKE '%www.freemusicarchive.org/curator/%' OR p.url LIKE '%www.npr.org/%/%/%/%/%' OR p.url LIKE '%www.npr.org/%/%/%/%/%/%' OR p.url LIKE '%www.npr.org/%/%/%/%/%/%/%' OR p.url LIKE '%www.npr.org/templates/story/story.php%' OR p.url LIKE '%huffduffer.com/%/%' OR p.url LIKE '%www.audioboom.com/boos/%' OR p.url LIKE '%audioboom.com/boos/%' OR p.url LIKE '%boo.fm/b%' OR p.url LIKE '%www.xiami.com/song/%' OR p.url LIKE '%xiami.com/song/%' OR p.url LIKE '%www.saynow.com/playMsg.html%' OR p.url LIKE '%www.saynow.com/playMsg.html%' OR p.url LIKE '%grooveshark.com/%' OR p.url LIKE '%radioreddit.com/songs%' OR p.url LIKE '%www.radioreddit.com/songs%' OR p.url LIKE '%radioreddit.com/?q=songs%' OR p.url LIKE '%www.radioreddit.com/?q=songs%' OR p.url LIKE '%www.gogoyoko.com/song/%' OR p.url LIKE '%hypem.com/premiere/%' OR p.url LIKE '%bop.fm/s/%/%' OR p.url LIKE '%clyp.it/%' OR p.url LIKE '%www.dnbradio.com/%' OR p.url LIKE '%dnbradio.com/%' OR p.url LIKE '%anchor.fm/%' OR p.url LIKE '%bumpers.fm/e/%' OR p.url LIKE '%buzzsprout.com/%' OR p.url LIKE '%.buzzsprout.com/%))' OR p.url LIKE '%(https://(www.instagram.com/p/%' OR p.url LIKE '%skitch.com/%/%/%' OR p.url LIKE '%img.skitch.com/%' OR p.url LIKE '%vidd.me/%' OR p.url LIKE '%vid.me/%' OR p.url LIKE '%gfycat.com/%' OR p.url LIKE '%.accredible.com/%' OR p.url LIKE '%accredible.com/%' OR p.url LIKE '%gist.github.com/%' OR p.url LIKE '%issuu.com/%/docs/%' OR p.url LIKE '%foursquare.com/%' OR p.url LIKE '%www.foursquare.com/%' OR p.url LIKE '%speakerdeck.com/%/%' OR p.url LIKE '%storify.com/%/%' OR p.url LIKE '%crocodoc.com/%' OR p.url LIKE '%.crocodoc.com/%' OR p.url LIKE '%urtak.com/u/%' OR p.url LIKE '%urtak.com/clr/%' OR p.url LIKE '%ganxy.com/%' OR p.url LIKE '%www.ganxy.com/%' OR p.url LIKE '%sketchfab.com/models/%' OR p.url LIKE '%sketchfab.com/show/%' OR p.url LIKE '%ifttt.com/recipes/%' OR p.url LIKE '%cloudup.com/%' OR p.url LIKE '%hackpad.com/%' OR p.url LIKE '%rapidengage.com/s/%' OR p.url LIKE '%stepic.org/%' OR p.url LIKE '%readtapestry.com/s/%/' OR p.url LIKE '%chirb.it/%' OR p.url LIKE '%medium.com/%' OR p.url LIKE '%medium.com/%/%' OR p.url LIKE '%www.branchtrack.com/projects/%' OR p.url LIKE '%www.wedgies.com/question/%' OR p.url LIKE '%public.chartblocks.com/c/%' OR p.url LIKE '%megavisor.com/view/%' OR p.url LIKE '%megavisor.com/en/view/%' OR p.url LIKE '%bunkrapp.com/%/%' OR p.url LIKE '%portfolium.com/entry/%' OR p.url LIKE '%.cartodb.com/%/%' OR p.url LIKE '%flowvella.com/s/%' OR p.url LIKE '%fr.peoplbrain.com/tutoriaux/%' OR p.url LIKE '%codepicnic.com/bites/%' OR p.url LIKE '%codepicnic.com/consoles/%' OR p.url LIKE '%tr.instela.com/%' OR p.url LIKE '%quora.com/%/answer/%' OR p.url LIKE '%www.quora.com/%/answer/%' OR p.url LIKE '%tunein.com/%' OR p.url LIKE '%tun.in/%' OR p.url LIKE '%scribblemaps.com/maps/view/%/%' OR p.url LIKE '%www.scribblemaps.com/maps/view/%/%' OR p.url LIKE '%marvelapp.com/%' OR p.url LIKE '%www.flat.io/score/%' OR p.url LIKE '%blab.im/%' OR p.url LIKE '%glitter.club/%' OR p.url LIKE '%pollplug.com/poll/%' OR p.url LIKE '%alpha.vrchive.com/%' OR p.url LIKE '%www.globalgiving.org/projects/%' OR p.url LIKE '%www.globalgiving.org/funds/%' OR p.url LIKE '%www.globalgiving.org/microprojects/%' OR p.url LIKE '%www.newhive.com/%/%' OR p.url LIKE '%newhive.com/%/%' OR p.url LIKE '%newhive.com/%/%' OR p.url LIKE '%slidr.io/%/%' OR p.url LIKE '%publons.com/author/%' OR p.url LIKE '%www.publons.com/author/%' OR p.url LIKE '%calameo.com/%' OR p.url LIKE '%www.calameo.com/%' OR p.url LIKE '%relayto.com/%' OR p.url LIKE '%www.relayto.com/%' OR p.url LIKE '%w.graphiq.com/w/%' OR p.url LIKE '%view.stacker.cc/%' OR p.url LIKE '%content.newsbound.com/%/%' OR p.url LIKE '%projects.invisionapp.com/share/%' OR p.url LIKE '%invis.io/%' OR p.url LIKE '%.silk.co/explore/%' OR p.url LIKE '%docs.com/%' OR p.url LIKE '%sway.com/%' OR p.url LIKE '%publicgood.com/campaign/%' OR p.url LIKE '%airtable.com/shr%' OR p.url LIKE '%rocketium.com/%' OR p.url LIKE '%cdn.knightlab.com/libs/timeline3/%' OR p.url LIKE '%cdn.knightlab.com/libs/juxtapose/%' OR p.url LIKE '%itunes.apple.com/%' OR p.url LIKE '%etsy.com/%' OR p.url LIKE '%www.etsy.com/%' OR p.url LIKE '%youtube.com/watch%' OR p.url LIKE '%.youtube.com/v/%' OR p.url LIKE '%youtu.be/%' OR p.url LIKE '%.youtube.com/playlist%' OR p.url LIKE '%www.youtube.com/embed/%' OR p.url LIKE '%youtube.com/gif%' OR p.url LIKE '%www.youtube.com/gif%' OR p.url LIKE '%www.youtube.com/attribution_link%' OR p.url LIKE '%youtube.com/attribution_link%' OR p.url LIKE '%youtube.ca/%' OR p.url LIKE '%youtube.jp/%' OR p.url LIKE '%youtube.com.br/%' OR p.url LIKE '%youtube.co.uk/%' OR p.url LIKE '%youtube.nl/%' OR p.url LIKE '%youtube.pl/%' OR p.url LIKE '%youtube.es/%' OR p.url LIKE '%youtube.ie/%' OR p.url LIKE '%it.youtube.com/%' OR p.url LIKE '%youtube.fr/%' OR p.url LIKE '%screen.yahoo.com/%/%' OR p.url LIKE '%animoto.com/play/%' OR p.url LIKE '%video214.com/play/%' OR p.url LIKE '%coub.com/view/%' OR p.url LIKE '%coub.com/embed/%' OR p.url LIKE '%www.streamio.com/api/v1/%' OR p.url LIKE '%streamio.com/api/v1/%' OR p.url LIKE '%vine.co/v/%' OR p.url LIKE '%www.vine.co/v/%' OR p.url LIKE '%mixbit.com/v/%' OR p.url LIKE '%www.brainshark.com/%/%' OR p.url LIKE '%brainshark.com/%/%' OR p.url LIKE '%23video.com/%' OR p.url LIKE '%.23video.com/%' OR p.url LIKE '%brainsonic.com/%' OR p.url LIKE '%.brainsonic.com/%' OR p.url LIKE '%www.reelhouse.org/%' OR p.url LIKE '%reelhouse.org/%' OR p.url LIKE '%.kaltura.com/%' OR p.url LIKE '%www.allego.com/%' OR p.url LIKE '%clipter.com/c/%' OR p.url LIKE '%app.devhv.com/oembed/%' OR p.url LIKE '%app.stghv.com/%' OR p.url LIKE '%sendvid.com/%' OR p.url LIKE '%clipmine.com/video/%' OR p.url LIKE '%clipmine.com/embed/%' OR p.url LIKE '%streamable.com/%' OR p.url LIKE '%clippituser.tv/%' OR p.url LIKE '%www.clippituser.tv/%' OR p.url LIKE '%my.storygami.com/video/%' OR p.url LIKE '%gifs.com/gif/%' OR p.url LIKE '%www.gifs.com/gif/%' OR p.url LIKE '%gifs.com/%' OR p.url LIKE '%www.gifs.com/%' OR p.url LIKE '%player.videopath.com/%' OR p.url LIKE '%.hosted.panopto.com/%' OR p.url LIKE '%www.dreambroker.com/channel/%' OR p.url LIKE '%dreambroker.com/channel/%' OR p.url LIKE '%www.oumy.com/v/%' OR p.url LIKE '%oumy.com/v/%' OR p.url LIKE '%iloopit.net/%/%' OR p.url LIKE '%uploadly.com/%' OR p.url LIKE '%www.vibby.com/watch%' OR p.url LIKE '%app.ilosvideos.com/view/%' OR p.url LIKE '%qwip.it/watch/%' OR p.url LIKE '%content.streamonecloud.net/embed/%' OR p.url LIKE '%platform.vixyvideo.com/%' OR p.url LIKE '%platform.vixyvideo.com/%' OR p.url LIKE '%skip.st/one/%' OR p.url LIKE '%www.vimeo.com/%' OR p.url LIKE '%vimeo.com/%' OR p.url LIKE '%player.vimeo.com/%' OR p.url LIKE '%yahoo.com/movies/%' OR p.url LIKE '%.yahoo.com/movies/%' OR p.url LIKE '%app.wistia.com/embed/medias/%' OR p.url LIKE '%wistia.com/%' OR p.url LIKE '%.wistia.com/%' OR p.url LIKE '%.wi.st/%' OR p.url LIKE '%wi.st/%' OR p.url LIKE '%.looplogic.com/%' OR p.url LIKE '%khanacademy.org/%' OR p.url LIKE '%www.khanacademy.org/%' OR p.url LIKE '%vidyard.com/%' OR p.url LIKE '%.stream.co.jp/apiservice/%' OR p.url LIKE '%.stream.ne.jp/apiservice/%' OR p.url LIKE '%api.lovelive.tv/v1/%' OR p.url LIKE '%video.esri.com/%' OR p.url LIKE '%mix.office.com/watch/%' OR p.url LIKE '%mix.office.com/mix/%' OR p.url LIKE '%mix.office.com/embed/%' OR p.url LIKE '%mix.office.com/MyMixes/Details/%' OR p.url LIKE '%.iplayerhd.com/playerframe/%' OR p.url LIKE '%.iplayerhd.com/player/video/%' OR p.url LIKE '%plays.tv/video/%' OR p.url LIKE '%app.videocheckout.com/embed/%' OR p.url LIKE '%lynda.com/%' OR p.url LIKE '%lynda.com/%' OR p.url LIKE '%v.embedly.com/%' OR p.url LIKE '%www.facebook.com/photo.php%' OR p.url LIKE '%www.facebook.com/video.php%' OR p.url LIKE '%www.facebook.com/%/posts/%' OR p.url LIKE '%fb.me/%' OR p.url LIKE '%www.facebook.com/%/photos/%' OR p.url LIKE '%www.facebook.com/%/videos/%' OR p.url LIKE '%plus.google.com/%' OR p.url LIKE '%soundcloud.com/%' OR p.url LIKE '%soundcloud.com/%/%' OR p.url LIKE '%soundcloud.com/%/sets/%' OR p.url LIKE '%soundcloud.com/groups/%' OR p.url LIKE '%open.spotify.com/%' OR p.url LIKE '%play.spotify.com/%' OR p.url LIKE '%bop.fm/s/%/%' OR p.url LIKE '%bop.fm/p/%' OR p.url LIKE '%bop.fm/a/%' OR p.url LIKE '%clyp.it/%' OR p.url LIKE '%sfx.io/%' OR p.url LIKE '%anchor.fm/%' OR p.url LIKE '%bumpers.fm/e/%' OR p.url LIKE '%buzzsprout.com/%' OR p.url LIKE '%.buzzsprout.com/%'
  -- example news sites  
  OR p.url LIKE '%reddit.com/%/%' OR p.url LIKE '%cnn.com/%/%' OR p.url LIKE '%nytimes.com/%/%' OR p.url LIKE '%huffingtonpost.com/%/%' OR p.url LIKE '%theguardian.com/%/%' OR p.url LIKE '%foxnews.com/%/%' OR p.url LIKE '%forbes.com/%/%' OR p.url LIKE '%bbc.co.uk/%/%' OR p.url LIKE '%usatoday.com/%/%' OR p.url LIKE '%drudgereport.com/%/%' OR p.url LIKE '%abcnews.go.com/%/%' OR p.url LIKE '%latimes.com/%/%' OR p.url LIKE '%time.com/%/%' OR p.url LIKE '%nypost.com/%/%' OR p.url LIKE '%cnbc.com/%/%' OR p.url LIKE '%sfgate.com/%/%' OR p.url LIKE '%theatlantic.com/%/%' OR p.url LIKE '%fortune.com/%/%' OR p.url LIKE '%chicagotribune.com/%/%' OR p.url LIKE '%theverge.com/%/%' OR p.url LIKE '%engadget.com/%/%' OR p.url LIKE '%techcrunch.com/%/%' OR p.url LIKE '%wired.com/%/%' OR p.url LIKE '%pcmag.com/%/%' OR p.url LIKE '%arstechnica.com/%/%' OR p.url LIKE '%economist.com/%/%' OR p.url LIKE '%theonion.com/%/%' OR p.url LIKE '%vox.com/%/%'
ORDER BY 
  hv.id DESC
LIMIT 50;

The results are interesting. Feel free to run that on your own places database to see what sorts of results you get. I think doing something like this is probably the right route for spotlight (for now).

[nchapman]

I experimented with a bunch of other queries, but we don't really have enough data to be very clever. I found this query to be nearly as good as anything fancier that I did:

SELECT
  p.url
FROM
  moz_historyvisits hv
INNER JOIN
  moz_places p ON p.id = hv.place_id
WHERE
  -- only specific visit types
  hv.visit_type = 1 AND
  -- visited minimal times
  p.visit_count <= 3 AND
  -- has a title (leaves out some good things but also leaves out redirects)
  p.title NOT NULL AND
  -- exclude search results and "apps"
  p.rev_host NOT IN ('moc.elgoog.www.', 'moc.elgoog.radnelac.', 'tsohlacol.', 'oc.t.', '.')
GROUP BY
  -- limits duplicate entries from the same host
  p.rev_host
ORDER BY
  hv.visit_date DESC
LIMIT 25;

[k88hudson]

One thing that could possibly cause issues for Spotlight is that we are only searching for items in the last 72 hours, but if the browser hasn't been used in the last 72 hours, there will be 0 results

[nchapman]

Okay here's my final-ish answer:

You should see a bookmark in the first position if it was created in the last 3 days. The rest of the URLs should be from unique hosts, be visited fewer than 3 times, and should have titles. There's also a list of excluded domains that we can expand. The intention is that those domains would be domains that we don't expect to get good results from embedly (search engines, app-y sites that are authenticated, etc)

updated: this was missing the exclusion of very recent history visits

-- this insanity is because `union` resorts the results so it's a `union all` + this `distinct query`
SELECT DISTINCT * FROM (
  -- subquery so that we can order and limit before union
  SELECT * FROM (
    SELECT
      hv.id, b.id bookmark_id, p.url, hv.visit_date, hv.visit_type
    FROM
      moz_historyvisits hv
    INNER JOIN
      moz_places p ON p.id = hv.place_id
    INNER JOIN
      moz_bookmarks b ON b.fk = p.id
    WHERE
      -- created in the last 3 days
      date(b.dateAdded / 1000 / 1000, 'unixepoch') > datetime('now', '-3 day') AND
      -- visited minimal times
      p.visit_count <= 3
    ORDER BY
      b.dateAdded DESC
    LIMIT 1
  )

  UNION ALL

  -- subquery so that we can order and limit before union
  SELECT * FROM (
    SELECT
      hv.id, b.id bookmark_id, p.url, hv.visit_date, hv.visit_type
    FROM
      moz_historyvisits hv
    INNER JOIN
      moz_places p ON p.id = hv.place_id
    LEFT JOIN
      moz_bookmarks b ON b.fk = p.id
    WHERE
      -- visited more than 30 minutes ago
      datetime(hv.visit_date / 1000 / 1000, 'unixepoch') < datetime('now', '-30 minutes') AND
      -- visited minimal times
      p.visit_count <= 3 AND
      -- has a title (leaves out some good things but also leaves out redirects)
      p.title NOT NULL AND
      -- exclude search results and "apps"
      p.rev_host NOT IN ('moc.elgoog.www.', 'moc.elgoog.radnelac.', 'tsohlacol.', 'oc.t.', '.')
    GROUP BY
      -- limits duplicate entries from the same host
      p.rev_host
    ORDER BY
      hv.visit_date DESC
    LIMIT 25
  )
)

[nchapman]

BTW, please feel free to not use this query at all. I went to the trouble of writing a SQL query so that I could test this approach on my history and have others run it on their places.db as well. Happy to hear any feedback people have from testing this with their own data.

[k88hudson]

This is partially resolved by https://github.com/mozilla/activity-streams/commit/3bff3389a307081306fe17ba5a29edf34a412191, which allows for fallbacks instead of complete removal of items in Spotlight

[oyiptong]

@nchapman I have the query implemented and can make a special build. However, we might need to rethink the de-duplication a rethink what shows in this rubric.

[k88hudson]

Since both the fallback UI and query change are implemented, I'm going to close this. Further work is being done in https://github.com/mozilla/activity-streams/issues/533