cdhigh / KindleEar

Aggregates RSS and web content(Calibre recipe), sends to Kindle, and includes an e-ink optimized online reader.
http://cdhigh.github.io/KindleEar/
MIT License
2.73k stars 630 forks source link

看漫画(漫画柜)订阅的漫画始终nonews #532

Closed Star-Song closed 5 years ago

Star-Song commented 5 years ago

好像看漫画网站在国外ip打不开?使用了两个代理,日本和新加坡节点均无法正常访问看漫画。还是其他什么原因?

linux40 commented 5 years ago

不是这个问题,Nature也始终显示nonews,链接在

cdhigh commented 5 years ago

Nature的rss是Feedburner生成的, 可以试一下在其链接之后添加一个查询字符串'?format=xml'或'?fmt=xml' 比如:http://feeds.nature.com/nature/rss/current?format=xml.

linux40 commented 5 years ago

嗯,Nature好了,不过现在推送的话,第一篇文章的内容是错的。。。

cdhigh commented 5 years ago

漫画是“外包”的:) 我没有研究过漫画相关的抓取,因为我本人从来没有接触过漫画。

我们都要感谢 @insert0003 ,漫画相关代码都是他的贡献。

Star-Song commented 5 years ago

好的,谢谢,我再琢磨琢磨吧,感谢大神们的付出。

insert0003 commented 5 years ago

你是指的直接打开看漫画网站吗,美国不行,日本可以,我本人在日本。gae项目如果选择是us的话,就无法获取了,需要设置东亚。

Star-Song commented 5 years ago

重新创建Google账号和GAE项目位置选择东亚2,推送看漫画仍然为nonews。

insert0003 commented 5 years ago

可以把gcloud里面的输出log发一下吗,看一下有什么错误信息没有。 还有你的漫画网址是什么,我试试看。

Star-Song commented 5 years ago

没有任何报错,点击推送的前后一段时间的log如下 { "protoPayload": { "@type": "type.googleapis.com/google.appengine.logging.v1.RequestLog", "appId": "n~star-kindlepush", "versionId": "1", "requestId": "5c0bbc9e00ff01c13841ea89ce00016e7e737461722d6b696e646c6570757368000131000100", "ip": "128.199.158.179", "startTime": "2018-12-08T12:44:14.115Z", "endTime": "2018-12-08T12:44:14.239962Z", "latency": "0.124962s", "megaCycles": "61", "method": "GET", "resource": "/deliver?u=admin", "httpVersion": "HTTP/1.1", "status": 200, "responseSize": "7617", "referrer": "http://star-kindlepush.appspot.com/advdelivernow", "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36", "urlMapEntry": "apps.module_front.app", "host": "star-kindlepush.appspot.com", "cost": 8.51266e-7, "instanceIndex": -1, "finished": true, "instanceId": "00c61b117c647dff11c65f53795ed00cdaf13f548d905506cd273cd3fa77d33a968e", "appEngineRelease": "1.9.65", "traceId": "66bd55f3beae5822163a214191c95542", "first": true, "traceSampled": true }, "insertId": "5c0bbc9e0003a9f0b91023b6", "httpRequest": { "status": 200 }, "resource": { "type": "gae_app", "labels": { "zone": "asia-east2-3", "project_id": "star-kindlepush", "version_id": "1", "module_id": "default" } }, "timestamp": "2018-12-08T12:44:14.115Z", "labels": { "clone_id": "00c61b117c647dff11c65f53795ed00cdaf13f548d905506cd273cd3fa77d33a968e" }, "logName": "projects/star-kindlepush/logs/appengine.googleapis.com%2Frequest_log", "operation": { "id": "5c0bbc9e00ff01c13841ea89ce00016e7e737461722d6b696e646c6570757368000131000100", "producer": "appengine.googleapis.com/request_id", "first": true, "last": true }, "trace": "projects/star-kindlepush/traces/66bd55f3beae5822163a214191c95542", "receiveTimestamp": "2018-12-08T12:44:14.256490110Z", "traceSampled": true }, 漫画网站为 https://www.manhuagui.com/comic/22843/

Star-Song commented 5 years ago

改为手机版页面后,即 https://m.manhuagui.com/comic/22843/ 添加订阅并推送,worker的log有报错, logMessage": "fetch comic page failed: https://m.manhuagui.com/comic/22843/ 完整log如下 [ { "protoPayload": { "@type": "type.googleapis.com/google.appengine.logging.v1.RequestLog", "appId": "n~star-kindlepush", "versionId": "1", "requestId": "5c0bb96500ff07c53cc8e656d700016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "ip": "0.1.0.2", "startTime": "2018-12-08T12:30:29.509244Z", "endTime": "2018-12-08T12:32:11.803724Z", "latency": "102.294480s", "megaCycles": "2222", "method": "GET", "resource": "/worker?u=admin&id=5066549580791808", "httpVersion": "HTTP/1.1", "status": 200, "responseSize": "102", "referrer": "http://star-kindlepush.appspot.com/deliver?u=admin", "userAgent": "AppEngine-Google; (+http://code.google.com/appengine)", "urlMapEntry": "apps.module_backend.app", "host": "worker.star-kindlepush.appspot.com", "cost": 1.16005e-7, "taskQueueName": "deliverqueue1", "taskName": "0794032091171792004", "finished": true, "instanceId": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50", "line": [ { "time": "2018-12-08T12:32:11.639667Z", "severity": "WARNING", "logMessage": "fetch comic page failed: https://m.manhuagui.com/comic/22843/", "sourceLocation": { "file": "/base/data/home/apps/n~star-kindlepush/worker:1.414387191098339176/books/comic/manhuaguibase.py", "line": "75", "functionName": "getChapterList" } } ], "moduleId": "worker", "appEngineRelease": "1.9.65", "traceId": "449babf56a534585d9891809855efe05", "first": true, "traceSampled": true }, "insertId": "5c0bb9cb000c443b2dbe025a", "httpRequest": { "status": 200 }, "resource": { "type": "gae_app", "labels": { "zone": "asia-east2-3", "project_id": "star-kindlepush", "version_id": "1", "module_id": "worker" } }, "timestamp": "2018-12-08T12:30:29.509244Z", "severity": "WARNING", "labels": { "clone_id": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50" }, "logName": "projects/star-kindlepush/logs/appengine.googleapis.com%2Frequest_log", "operation": { "id": "5c0bb96500ff07c53cc8e656d700016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "producer": "appengine.googleapis.com/request_id", "first": true, "last": true }, "trace": "projects/star-kindlepush/traces/449babf56a534585d9891809855efe05", "receiveTimestamp": "2018-12-08T12:32:11.818610058Z", "traceSampled": true }, { "protoPayload": { "@type": "type.googleapis.com/google.appengine.logging.v1.RequestLog", "appId": "n~star-kindlepush", "versionId": "1", "requestId": "5c0bb8ec00ff0e19bd556b9be800016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "ip": "0.1.0.2", "startTime": "2018-12-08T12:28:28.924093Z", "endTime": "2018-12-08T12:30:11.361828Z", "latency": "102.437735s", "megaCycles": "2203", "method": "GET", "resource": "/worker?u=admin&id=5066549580791808&feedsId=5674823384563712", "httpVersion": "HTTP/1.1", "status": 200, "responseSize": "102", "referrer": "http://star-kindlepush.appspot.com/deliver?u=admin&id=5066549580791808&feedsId=5674823384563712", "userAgent": "AppEngine-Google; (+http://code.google.com/appengine)", "urlMapEntry": "apps.module_backend.app", "host": "worker.star-kindlepush.appspot.com", "cost": 1.16005e-7, "taskQueueName": "deliverqueue1", "taskName": "5654032091171792004", "finished": true, "instanceId": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50", "line": [ { "time": "2018-12-08T12:30:11.195233Z", "severity": "WARNING", "logMessage": "fetch comic page failed: https://m.manhuagui.com/comic/22843/", "sourceLocation": { "file": "/base/data/home/apps/n~star-kindlepush/worker:1.414387191098339176/books/comic/manhuaguibase.py", "line": "75", "functionName": "getChapterList" } } ], "moduleId": "worker", "appEngineRelease": "1.9.65", "traceId": "3ffee37f9192a784c88c73eef5876fc3", "first": true, "traceSampled": true }, "insertId": "5c0bb953000585f806134588", "httpRequest": { "status": 200 }, "resource": { "type": "gae_app", "labels": { "zone": "asia-east2-3", "project_id": "star-kindlepush", "version_id": "1", "module_id": "worker" } }, "timestamp": "2018-12-08T12:28:28.924093Z", "severity": "WARNING", "labels": { "clone_id": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50" }, "logName": "projects/star-kindlepush/logs/appengine.googleapis.com%2Frequest_log", "operation": { "id": "5c0bb8ec00ff0e19bd556b9be800016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "producer": "appengine.googleapis.com/request_id", "first": true, "last": true }, "trace": "projects/star-kindlepush/traces/3ffee37f9192a784c88c73eef5876fc3", "receiveTimestamp": "2018-12-08T12:30:11.377420293Z", "traceSampled": true }, { "protoPayload": { "@type": "type.googleapis.com/google.appengine.logging.v1.RequestLog", "appId": "n~star-kindlepush", "versionId": "1", "requestId": "5c0bb64500ff052ac9951af26f00016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "ip": "0.1.0.3", "startTime": "2018-12-08T12:17:09.338633Z", "endTime": "2018-12-08T12:17:10.936504Z", "latency": "1.597871s", "megaCycles": "1536", "method": "GET", "resource": "/_ah/start", "httpVersion": "HTTP/1.1", "status": 404, "responseSize": "123", "urlMapEntry": "apps.module_backend.app", "host": "0.1.worker.star-kindlepush.appspot.com", "cost": 1.3746e-8, "wasLoadingRequest": true, "finished": true, "instanceId": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50", "line": [ { "time": "2018-12-08T12:17:10.936372Z", "severity": "INFO", "logMessage": "This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application." } ], "moduleId": "worker", "appEngineRelease": "1.9.65", "traceId": "9088a11a9b9adb1cf8654f44dfe09cba", "first": true }, "insertId": "5c0bb646000e4ac788538993", "httpRequest": { "status": 404 }, "resource": { "type": "gae_app", "labels": { "project_id": "star-kindlepush", "version_id": "1", "module_id": "worker", "zone": "asia-east2-3" } }, "timestamp": "2018-12-08T12:17:09.338633Z", "severity": "INFO", "labels": { "clone_id": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50" }, "logName": "projects/star-kindlepush/logs/appengine.googleapis.com%2Frequest_log", "operation": { "id": "5c0bb64500ff052ac9951af26f00016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "producer": "appengine.googleapis.com/request_id", "first": true, "last": true }, "trace": "projects/star-kindlepush/traces/9088a11a9b9adb1cf8654f44dfe09cba", "receiveTimestamp": "2018-12-08T12:17:10.961764452Z" }, { "protoPayload": { "@type": "type.googleapis.com/google.appengine.logging.v1.RequestLog", "appId": "n~star-kindlepush", "versionId": "1", "requestId": "5c0bb64500ff0501b8e27c0abc00016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "ip": "0.1.0.2", "startTime": "2018-12-08T12:17:09.328120Z", "endTime": "2018-12-08T12:18:53.427163Z", "latency": "104.099043s", "megaCycles": "2201", "method": "GET", "resource": "/worker?u=admin&id=5066549580791808", "httpVersion": "HTTP/1.1", "status": 200, "responseSize": "102", "referrer": "http://star-kindlepush.appspot.com/deliver?u=admin", "userAgent": "AppEngine-Google; (+http://code.google.com/appengine)", "urlMapEntry": "apps.module_backend.app", "host": "worker.star-kindlepush.appspot.com", "cost": 1.16117e-7, "taskQueueName": "deliverqueue1", "taskName": "9520326159401376106", "pendingTime": "1.608644724s", "finished": true, "instanceId": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50", "line": [ { "time": "2018-12-08T12:18:53.273511Z", "severity": "WARNING", "logMessage": "fetch comic page failed: https://m.manhuagui.com/comic/22843/", "sourceLocation": { "file": "/base/data/home/apps/n~star-kindlepush/worker:1.414387191098339176/books/comic/manhuaguibase.py", "line": "75", "functionName": "getChapterList" } } ], "moduleId": "worker", "appEngineRelease": "1.9.65", "traceId": "506a5a6183f81128b07ccb0f91a74ea2", "first": true, "traceSampled": true }, "insertId": "5c0bb6ad0006854c53e9c7b4", "httpRequest": { "status": 200 }, "resource": { "type": "gae_app", "labels": { "zone": "asia-east2-3", "project_id": "star-kindlepush", "version_id": "1", "module_id": "worker" } }, "timestamp": "2018-12-08T12:17:09.328120Z", "severity": "WARNING", "labels": { "clone_id": "00c61b117ca1fe1730620c07256e5223911d5aae6375fa45ecda6c2ca1af4cb2fefee0ea50" }, "logName": "projects/star-kindlepush/logs/appengine.googleapis.com%2Frequest_log", "operation": { "id": "5c0bb64500ff0501b8e27c0abc00016e7e737461722d6b696e646c65707573680001776f726b65723a31000100", "producer": "appengine.googleapis.com/request_id", "first": true, "last": true }, "trace": "projects/star-kindlepush/traces/506a5a6183f81128b07ccb0f91a74ea2", "receiveTimestamp": "2018-12-08T12:18:53.455333743Z", "traceSampled": true } ]

insert0003 commented 5 years ago

从你的log里面,暂时看不出来什么错误。 我在自己测试了那个漫画,应该可以正常推送的。

所以在#534的pull request里面,加了部分log输出,添加733.so的支持, 等这个合并了以后,你可以更新一下试试看新的log,或者使用cartoonmad的网址。 https://www.cartoonmad.com/comic/5893.html

escape0707 commented 3 years ago

至今漫画柜还是不能用国内/香港代理访问。前面说的日本节点倒是没问题。8说了,我去补少女终末旅行了~