2019-08-18 12:41:35
Graby is ready to fetch
2019-08-18 12:41:35
. looking for site config for weibo.com in primary folder
weibo.com
2019-08-18 12:41:35
Appending site config settings from global.txt
2019-08-18 12:41:35
. looking for site config for global in primary folder
global
2019-08-18 12:41:35
... found site config global.txt
global.txt
2019-08-18 12:41:35
Cached site config with key: weibo.com
weibo.com
2019-08-18 12:41:35
. looking for site config for global in primary folder
global
2019-08-18 12:41:35
... found site config global.txt
global.txt
2019-08-18 12:41:35
Appending site config settings from global.txt
2019-08-18 12:41:35
Cached site config with key: global
global
2019-08-18 12:41:35
Cached site config with key: weibo.com.merged
weibo.com.merged
2019-08-18 12:41:35
Fetching url: https://weibo.com/5210843684/I2M8O5eCs
https://weibo.com/5210843684/I2M8O5eCs
2019-08-18 12:41:35
Trying using method "get" on url "https://weibo.cn/5210843684/I2M8O5eCs"
get
https://weibo.cn/5210843684/I2M8O5eCs
2019-08-18 12:41:35
Use default user-agent "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" for url "https://weibo.cn/5210843684/I2M8O5eCs"
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
https://weibo.cn/5210843684/I2M8O5eCs
2019-08-18 12:41:35
Use default referer "http://www.google.co.uk/url?sa=t&source=web&cd=1" for url "https://weibo.cn/5210843684/I2M8O5eCs"
http://www.google.co.uk/url?sa=t&source=web&cd=1
https://weibo.cn/5210843684/I2M8O5eCs
2019-08-18 12:41:35
Data fetched: [array]
........
Graby used the default UA (Googlebot) instead of the one specified in the weibo.cn.txt site config.
Graby configuration: (in
services.yml
on a Wallabag instance)Note I use Googlebot UA and add a redirect rule from "weibo.com" to "weibo.cn".
In site config
weibo.cn.txt
:Note I specify a normal browser UA.
Graby logs when extracting https://weibo.com/5210843684/I2M8O5eCs :
Graby used the default UA (Googlebot) instead of the one specified in the
weibo.cn.txt
site config.