search

oscarotero / Embed

Get info from any web service or page
MIT License
2.09k stars 310 forks source link

Too much bugs for V3 not ready #191

Closed onigetoc closed 7 years ago

onigetoc commented 7 years ago

Multiple URLs do not work

http://www.allotoi.com/sports/colonnes-de-lumiere-dans-la-nuit-canadienne http://reveilletoi.com/spiritualite/les-crises-de-lego http://codesniff.com/plugins/rss-feeds-news-blocks-wordpress-plugins-for-rss http://softplug.com

V3 Do not work https://oscarotero.com/embed3/demo/index.php?url=http://www.allotoi.com/sports/colonnes-de-lumiere-dans-la-nuit-canadienne V2 still working https://oscarotero.com/embed2/demo/index.php?url=http://www.allotoi.com/sports/colonnes-de-lumiere-dans-la-nuit-canadienne

V3 Do not work https://oscarotero.com/embed3/demo/index.php?url=http://softplug.com/fr/ V2 still working https://oscarotero.com/embed2/demo/index.php?url=http://softplug.com/fr/

oscarotero commented 7 years ago

It seems that there's serveral sites returning different content depending of the user-agent request header. The default user-agent used was "Embed PHP Library", but i changed it to fix #186, and this change makes fail other sites, so I've changed it again to fix the other cases and this makes fail these other sites.

Finally I've found a user-agent that works fine in all cases (or so I think).

If you find more failing sites, please try to change the user agent.

onigetoc commented 7 years ago

For my own website parser, i also find that this User agent work every time.

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0