New article doesn't show up on planet

[ionelmc]

I have published a new entry in https://blog.ionelmc.ro/feeds/python.atom.xml (7 may) and it doesn't seem to show up on the planet. Is there a problem on my end?

[ionelmc]

I've checked it at https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fblog.ionelmc.ro%2Ffeeds%2Fpython.atom.xml - it's definitely valid.

[tjguk]

Thanks for checking the most obvious stuff; hopefully one of us will have a chance to look at the feed before too long.

[ionelmc]

It still doesn't publish my new entries. @tjguk or anyone can take a look and tell me what's wrong? My feed is still valid, eg:

[ionelmc]

Or maybe @tseaver? Sorry for pinging but then this was open for a long time.

I also have a rss feed (https://blog.ionelmc.ro/feeds/python.rss.xml), maybe that works?

[rochacbruno]

@ionelmc I pushed a change from atom to .rss (lets see if that will work in a couple hours)

[ionelmc]

Well I don't see anything appearing. Is there any time limit in action? (now the entry would be two days old)

Also, are there any logs to check? I've tried running the planet.py script locally and it seems to work with both atom and rss feeds. Actually, is that script up to date with what's deployed?

[rochacbruno]

@ionelmc I dont have access to the server where it is running, maybe @tseaver has access to it to see the logs.

This repo is the same which is deployed automatically to the server every ~2 hours.

[tseaver]

I don't have access to the server / logs.

I have tried debugging by copying the INI file, deleting all feeds except the one for @ionelmc, and running locally: the one current feed item ("Rehashing the src layout", 2017-09-24T17:00:00Z) does appear (the others are too old for our conifguration).

Checking now: the last item on the Planet is newer than that: it may be scrolled off due to a max-items configuration?

[ionelmc]

Oh well cat's out of the bag now. I think this should be left open in case someone can check those logs.

Anyway, I've been trying some things, it turns out the planet won't fetch my feed on ubuntu 10.04 (old ssl problem). Could that be it?

If so, you could try to change my feed to http (it should be fine, there's no redirect to https)?

[tjguk]

Fine with leaving it open.

The slightly wider issue here is -- as elsewhere within the python.org ecosystem -- that we have various decoupled sets of volunteers, each holding a piece of the puzzle. Like several of those volunteers, I have a finger in a few pies but not all. Specifically I'm not involved with the infrastructure team.

I pinged the infra team a month or so ago to ask this very question about access to logs (or to the server) but haven't heard back. Obviously they're volunteers too and I know perfectly well how long I can leave issues hanging because of the aggro they'll cause.

I'll ping again (or any of the other planet@ crew is welcome to do so; I think I cc-ed you all last time). And let's see what we can do.

TJG

[pydanny]

@ionelmc I had a similar problem with my blog (https://github.com/python/planet/issues/233#issuecomment-307997084).

Planet Python uses the W3C validator for feeds, which is both strict to the point that it blocks you and me, yet allows broken posts through. The solution I think is to remove or fix the broken validation. Alas, I don't have time for it. If there was a way to pay for the fix I would happily pay someone to do the work.

[ionelmc]

@pydanny But my feed is technically valid. Are we talking about a different validator than https://validator.w3.org/feed/?

[pybites]

@ionelmc did you make any progress on this?

I want to give it another go, last time it seemed (Open)SSL related, see:

https://github.com/python/planet/issues/240 and https://github.com/python/planet/issues/114

[pydanny]

I just modernized my blog and also have a valid feed. However, just like with @ionelmc, my latest article never showed up.

Maybe the answer is to delete our blog feeds and re-add them?

[hobojoe1848]

Hey Dan, worth a shot but I based on the previous issues @pybites linked, I think it's the swap to OpenSSL. If we move the site back to HTTP then the feed is happily pulled by the Planet Python scripts. The second we swapped to HTTPS on PyBit.es, it died.

[bbelderbos]

It seems related to SSL/feedparser/old Python 2 version, I can get around it using requests but I tested it on 2.7: https://github.com/bbelderbos/planet/commit/39044605053ca124213b8ca7dba4dee1f50dfed8

One weird thing with this small sample set is that realpython also throws an error yet they are visiable in the feed: https://web.archive.org/web/20180327170023/http://planetpython.org

I was going to PR this but I see in planet.py it "Requires Python 2.1, recommends 2.3” and requests is supported on >= 2.6 (https://pypi.python.org/pypi/requests)

Also my error is different as the one @tseaver reported in https://github.com/python/planet/pull/239 for our feed: SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

@rochacbruno @tseaver @tjguk could you please advice? It would be nice if we can fix this. It affects multiple feeds!

Thanks

[bbelderbos]

We probably need to debug with the exact Python version used on the server (2.1?)

Only a slight change in version already shows different results in feedparser.parse

$ python2.7 Python 2.7.13 (default, Jul 19 2017, 15:02:16)

info = feedparser.parse('https://pybit.es/feeds/all.atom.xml', etag=None, modified=None, agent='Planet Python +http://planetpython.org/ Planet/2.0 +http://www.planetplanet.org UniversalFeedParser/4.1 +http://feedparser.org/')

(using feedparser from the code repo)

import feedparser feedparser.version ‘4.1'

info = feedparser.parse('https://pybit.es/feeds/all.atom.xml', etag=None, modified=None, agent='Planet Python +http://planetpython.org/ Planet/2.0 +http://www.planetplanet.org UniversalFeedParser/4.1 +http://feedparser.org/') info {'feed': {}, 'encoding': 'utf-8', 'bozo': 1, 'version': None, 'entries': [], 'bozo_exception': URLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)'),)}

Yet:

[bbelderb@macbook planet (issue135)]$ python2 Python 2.7.10 (default, Jul 15 2017, 17:16:57)

import feedparser feedparser.version '4.1' info = feedparser.parse('https://pybit.es/feeds/all.atom.xml', etag=None, modified=None, agent='Planet Python +http://planetpython.org/ Planet/2.0 +http://www.planetplanet.org UniversalFeedParser/4.1 +http://feedparser.org/') info.bozo 0 => works

[holdenweb]

The Planet uses Python 2.1?

Steve Holden

On Sun, Apr 1, 2018 at 10:29 AM, Bob Belderbos notifications@github.com wrote:

We probably need to debug with the exact Python version used on the server (2.1?)

Only a slight change in version already shows different results in feedparser.parse

$ python2.7 Python 2.7.13 (default, Jul 19 2017, 15:02:16)

info = feedparser.parse('https://pybit.es/feeds/all.atom.xml', etag=None, modified=None, agent='Planet Python +http://planetpython.org/ Planet/2.0 + http://www.planetplanet.org UniversalFeedParser/4.1 + http://feedparser.org/')

(using feedparser from the code repo)

import feedparser feedparser.version ‘4.1'

info = feedparser.parse('https://pybit.es/feeds/all.atom.xml', etag=None, modified=None, agent='Planet Python +http://planetpython.org/ Planet/2.0 + http://www.planetplanet.org UniversalFeedParser/4.1 + http://feedparser.org/') info {'feed': {}, 'encoding': 'utf-8', 'bozo': 1, 'version': None, 'entries': [], 'bozo_exception': URLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)'),)}

Yet:

[bbelderb@macbook planet (issue135)]$ python2 Python 2.7.10 (default, Jul 15 2017, 17:16:57)

import feedparser feedparser.version '4.1' info = feedparser.parse('https://pybit.es/feeds/all.atom.xml', etag=None, modified=None, agent='Planet Python +http://planetpython.org/ Planet/2.0 + http://www.planetplanet.org UniversalFeedParser/4.1 + http://feedparser.org/') info.bozo 0 => works

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/python/planet/issues/135#issuecomment-377774534, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGbYCCNoKjb5EschJwtR8czn-sF9wXcks5tkJ4QgaJpZM4IZnXN .

[pybites]

Added this in the bonus section of our new Planet Python challenge: https://pybit.es/codechallenge49.html

[ewdurbin]

Confirmed this issue is also related to the TLS issues in fetching feeds.

Server was running Python 2.7.6 (Ubuntu 14.04) but has been updated to latest Python 2.7.15 release and is fetching modern TLS'd feeds appropriately now.