Closed Star-Song closed 5 years ago
Nature的rss是Feedburner生成的, 可以试一下在其链接之后添加一个查询字符串'?format=xml'或'?fmt=xml' 比如:http://feeds.nature.com/nature/rss/current?format=xml.
嗯,Nature好了,不过现在推送的话,第一篇文章的内容是错的。。。
漫画是“外包”的:) 我没有研究过漫画相关的抓取,因为我本人从来没有接触过漫画。
我们都要感谢 @insert0003 ,漫画相关代码都是他的贡献。
好的,谢谢,我再琢磨琢磨吧,感谢大神们的付出。
你是指的直接打开看漫画网站吗,美国不行,日本可以,我本人在日本。gae项目如果选择是us的话,就无法获取了,需要设置东亚。
重新创建Google账号和GAE项目位置选择东亚2,推送看漫画仍然为nonews。
可以把gcloud里面的输出log发一下吗,看一下有什么错误信息没有。 还有你的漫画网址是什么,我试试看。
没有任何报错,点击推送的前后一段时间的log如下 { "protoPayload": { "@type": "type.googleapis.com/google.appengine.logging.v1.RequestLog", "appId": "n~star-kindlepush", "versionId": "1", "requestId": "5c0bbc9e00ff01c13841ea89ce00016e7e737461722d6b696e646c6570757368000131000100", "ip": "128.199.158.179", "startTime": "2018-12-08T12:44:14.115Z", "endTime": "2018-12-08T12:44:14.239962Z", "latency": "0.124962s", "megaCycles": "61", "method": "GET", "resource": "/deliver?u=admin", "httpVersion": "HTTP/1.1", "status": 200, "responseSize": "7617", "referrer": "http://star-kindlepush.appspot.com/advdelivernow", "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36", "urlMapEntry": "apps.module_front.app", "host": "star-kindlepush.appspot.com", "cost": 8.51266e-7, "instanceIndex": -1, "finished": true, "instanceId": "00c61b117c647dff11c65f53795ed00cdaf13f548d905506cd273cd3fa77d33a968e", "appEngineRelease": "1.9.65", "traceId": "66bd55f3beae5822163a214191c95542", "first": true, "traceSampled": true }, "insertId": "5c0bbc9e0003a9f0b91023b6", "httpRequest": { "status": 200 }, "resource": { "type": "gae_app", "labels": { "zone": "asia-east2-3", "project_id": "star-kindlepush", "version_id": "1", "module_id": "default" } }, "timestamp": "2018-12-08T12:44:14.115Z", "labels": { "clone_id": "00c61b117c647dff11c65f53795ed00cdaf13f548d905506cd273cd3fa77d33a968e" }, "logName": "projects/star-kindlepush/logs/appengine.googleapis.com%2Frequest_log", "operation": { "id": "5c0bbc9e00ff01c13841ea89ce00016e7e737461722d6b696e646c6570757368000131000100", "producer": "appengine.googleapis.com/request_id", "first": true, "last": true }, "trace": "projects/star-kindlepush/traces/66bd55f3beae5822163a214191c95542", "receiveTimestamp": "2018-12-08T12:44:14.256490110Z", "traceSampled": true }, 漫画网站为 https://www.manhuagui.com/comic/22843/
改为手机版页面后,即 https://m.manhuagui.com/comic/22843/ 添加订阅并推送,worker的log有报错, logMessage": "fetch comic page failed: https://m.manhuagui.com/comic/22843/ 完整log如下 [ { "protoPayload": { "@type": "type.googleapis.com/google.appengine.logging.v1.RequestLog", "appId": "n~star-kindlepush", "versionId": "1", "requestId": "5c0bb96500ff07c53cc8e656d700016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "ip": "0.1.0.2", "startTime": "2018-12-08T12:30:29.509244Z", "endTime": "2018-12-08T12:32:11.803724Z", "latency": "102.294480s", "megaCycles": "2222", "method": "GET", "resource": "/worker?u=admin&id=5066549580791808", "httpVersion": "HTTP/1.1", "status": 200, "responseSize": "102", "referrer": "http://star-kindlepush.appspot.com/deliver?u=admin", "userAgent": "AppEngine-Google; (+http://code.google.com/appengine)", "urlMapEntry": "apps.module_backend.app", "host": "worker.star-kindlepush.appspot.com", "cost": 1.16005e-7, "taskQueueName": "deliverqueue1", "taskName": "0794032091171792004", "finished": true, "instanceId": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50", "line": [ { "time": "2018-12-08T12:32:11.639667Z", "severity": "WARNING", "logMessage": "fetch comic page failed: https://m.manhuagui.com/comic/22843/", "sourceLocation": { "file": "/base/data/home/apps/n~star-kindlepush/worker:1.414387191098339176/books/comic/manhuaguibase.py", "line": "75", "functionName": "getChapterList" } } ], "moduleId": "worker", "appEngineRelease": "1.9.65", "traceId": "449babf56a534585d9891809855efe05", "first": true, "traceSampled": true }, "insertId": "5c0bb9cb000c443b2dbe025a", "httpRequest": { "status": 200 }, "resource": { "type": "gae_app", "labels": { "zone": "asia-east2-3", "project_id": "star-kindlepush", "version_id": "1", "module_id": "worker" } }, "timestamp": "2018-12-08T12:30:29.509244Z", "severity": "WARNING", "labels": { "clone_id": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50" }, "logName": "projects/star-kindlepush/logs/appengine.googleapis.com%2Frequest_log", "operation": { "id": "5c0bb96500ff07c53cc8e656d700016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "producer": "appengine.googleapis.com/request_id", "first": true, "last": true }, "trace": "projects/star-kindlepush/traces/449babf56a534585d9891809855efe05", "receiveTimestamp": "2018-12-08T12:32:11.818610058Z", "traceSampled": true }, { "protoPayload": { "@type": "type.googleapis.com/google.appengine.logging.v1.RequestLog", "appId": "n~star-kindlepush", "versionId": "1", "requestId": "5c0bb8ec00ff0e19bd556b9be800016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "ip": "0.1.0.2", "startTime": "2018-12-08T12:28:28.924093Z", "endTime": "2018-12-08T12:30:11.361828Z", "latency": "102.437735s", "megaCycles": "2203", "method": "GET", "resource": "/worker?u=admin&id=5066549580791808&feedsId=5674823384563712", "httpVersion": "HTTP/1.1", "status": 200, "responseSize": "102", "referrer": "http://star-kindlepush.appspot.com/deliver?u=admin&id=5066549580791808&feedsId=5674823384563712", "userAgent": "AppEngine-Google; (+http://code.google.com/appengine)", "urlMapEntry": "apps.module_backend.app", "host": "worker.star-kindlepush.appspot.com", "cost": 1.16005e-7, "taskQueueName": "deliverqueue1", "taskName": "5654032091171792004", "finished": true, "instanceId": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50", "line": [ { "time": "2018-12-08T12:30:11.195233Z", "severity": "WARNING", "logMessage": "fetch comic page failed: https://m.manhuagui.com/comic/22843/", "sourceLocation": { "file": "/base/data/home/apps/n~star-kindlepush/worker:1.414387191098339176/books/comic/manhuaguibase.py", "line": "75", "functionName": "getChapterList" } } ], "moduleId": "worker", "appEngineRelease": "1.9.65", "traceId": "3ffee37f9192a784c88c73eef5876fc3", "first": true, "traceSampled": true }, "insertId": "5c0bb953000585f806134588", "httpRequest": { "status": 200 }, "resource": { "type": "gae_app", "labels": { "zone": "asia-east2-3", "project_id": "star-kindlepush", "version_id": "1", "module_id": "worker" } }, "timestamp": "2018-12-08T12:28:28.924093Z", "severity": "WARNING", "labels": { "clone_id": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50" }, "logName": "projects/star-kindlepush/logs/appengine.googleapis.com%2Frequest_log", "operation": { "id": "5c0bb8ec00ff0e19bd556b9be800016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "producer": "appengine.googleapis.com/request_id", "first": true, "last": true }, "trace": "projects/star-kindlepush/traces/3ffee37f9192a784c88c73eef5876fc3", "receiveTimestamp": "2018-12-08T12:30:11.377420293Z", "traceSampled": true }, { "protoPayload": { "@type": "type.googleapis.com/google.appengine.logging.v1.RequestLog", "appId": "n~star-kindlepush", "versionId": "1", "requestId": "5c0bb64500ff052ac9951af26f00016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "ip": "0.1.0.3", "startTime": "2018-12-08T12:17:09.338633Z", "endTime": "2018-12-08T12:17:10.936504Z", "latency": "1.597871s", "megaCycles": "1536", "method": "GET", "resource": "/_ah/start", "httpVersion": "HTTP/1.1", "status": 404, "responseSize": "123", "urlMapEntry": "apps.module_backend.app", "host": "0.1.worker.star-kindlepush.appspot.com", "cost": 1.3746e-8, "wasLoadingRequest": true, "finished": true, "instanceId": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50", "line": [ { "time": "2018-12-08T12:17:10.936372Z", "severity": "INFO", "logMessage": "This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application." } ], "moduleId": "worker", "appEngineRelease": "1.9.65", "traceId": "9088a11a9b9adb1cf8654f44dfe09cba", "first": true }, "insertId": "5c0bb646000e4ac788538993", "httpRequest": { "status": 404 }, "resource": { "type": "gae_app", "labels": { "project_id": "star-kindlepush", "version_id": "1", "module_id": "worker", "zone": "asia-east2-3" } }, "timestamp": "2018-12-08T12:17:09.338633Z", "severity": "INFO", "labels": { "clone_id": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50" }, "logName": "projects/star-kindlepush/logs/appengine.googleapis.com%2Frequest_log", "operation": { "id": "5c0bb64500ff052ac9951af26f00016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "producer": "appengine.googleapis.com/request_id", "first": true, "last": true }, "trace": "projects/star-kindlepush/traces/9088a11a9b9adb1cf8654f44dfe09cba", "receiveTimestamp": "2018-12-08T12:17:10.961764452Z" }, { "protoPayload": { "@type": "type.googleapis.com/google.appengine.logging.v1.RequestLog", "appId": "n~star-kindlepush", "versionId": "1", "requestId": "5c0bb64500ff0501b8e27c0abc00016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "ip": "0.1.0.2", "startTime": "2018-12-08T12:17:09.328120Z", "endTime": "2018-12-08T12:18:53.427163Z", "latency": "104.099043s", "megaCycles": "2201", "method": "GET", "resource": "/worker?u=admin&id=5066549580791808", "httpVersion": "HTTP/1.1", "status": 200, "responseSize": "102", "referrer": "http://star-kindlepush.appspot.com/deliver?u=admin", "userAgent": "AppEngine-Google; (+http://code.google.com/appengine)", "urlMapEntry": "apps.module_backend.app", "host": "worker.star-kindlepush.appspot.com", "cost": 1.16117e-7, "taskQueueName": "deliverqueue1", "taskName": "9520326159401376106", "pendingTime": "1.608644724s", "finished": true, "instanceId": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50", "line": [ { "time": "2018-12-08T12:18:53.273511Z", "severity": "WARNING", "logMessage": "fetch comic page failed: https://m.manhuagui.com/comic/22843/", "sourceLocation": { "file": "/base/data/home/apps/n~star-kindlepush/worker:1.414387191098339176/books/comic/manhuaguibase.py", "line": "75", "functionName": "getChapterList" } } ], "moduleId": "worker", "appEngineRelease": "1.9.65", "traceId": "506a5a6183f81128b07ccb0f91a74ea2", "first": true, "traceSampled": true }, "insertId": "5c0bb6ad0006854c53e9c7b4", "httpRequest": { "status": 200 }, "resource": { "type": "gae_app", "labels": { "zone": "asia-east2-3", "project_id": "star-kindlepush", "version_id": "1", "module_id": "worker" } }, "timestamp": "2018-12-08T12:17:09.328120Z", "severity": "WARNING", "labels": { "clone_id": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50" }, "logName": "projects/star-kindlepush/logs/appengine.googleapis.com%2Frequest_log", "operation": { "id": "5c0bb64500ff0501b8e27c0abc00016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "producer": "appengine.googleapis.com/request_id", "first": true, "last": true }, "trace": "projects/star-kindlepush/traces/506a5a6183f81128b07ccb0f91a74ea2", "receiveTimestamp": "2018-12-08T12:18:53.455333743Z", "traceSampled": true } ]
从你的log里面,暂时看不出来什么错误。 我在自己测试了那个漫画,应该可以正常推送的。
所以在#534的pull request里面,加了部分log输出,添加733.so的支持, 等这个合并了以后,你可以更新一下试试看新的log,或者使用cartoonmad的网址。 https://www.cartoonmad.com/comic/5893.html
至今漫画柜还是不能用国内/香港代理访问。前面说的日本节点倒是没问题。8说了,我去补少女终末旅行了~
好像看漫画网站在国外ip打不开?使用了两个代理,日本和新加坡节点均无法正常访问看漫画。还是其他什么原因?