search

sympy / planet.sympy.org

Legacy repository. Use https://github.com/sympy/planet-sympy and https://github.com/planet-sympy/planet.sympy.org instead
Other
9 stars 27 forks source link

Don't push a change if there haven't been any updates #5

Open asmeurer opened 12 years ago

asmeurer commented 12 years ago

This is spamming with unnecessary updates. The result is that this repo is almost always at the top of https://github.com/sympy/, and I even once noticed it at https://github.com/timeline, which I think would normally be pretty rare. The bot should push only when there is something to push.

certik commented 12 years ago

I absolutely agree. However I don't have time currently to implement this. It shouldn't be that difficult --- pretty much this line:

https://github.com/sympy/planet.sympy.org/blob/master/server_update#L40

has to be modified. If anyone would like to give it a shot, that'd be great.

certik commented 12 years ago

For now, since the summer is over, I have decreased the bot frequency from every 20 minutes to every 2 hours. Let's fix it though, that'd be much better.

asmeurer commented 12 years ago

Do you know how to do it? It looks to me like it recreates the whole planet from scratch, then replaces the old one, so it's not obvious to me how to tell if something significant has changed or not.

I did notice that the rss10.xml and rss11.xml files are only updated when something changes (btw, I found 2f78760189bc0306059089bc2442bdc197aa9276 as a random example of a commit that actually changed something, since this is non-trivial to do). Do you know if this is always the case? If so, then we can make the rule to only commit if one of those files was updated (or maybe just if any file other than the three that are always updated is updated). Will that always work? I don't know too much about planet.

certik commented 12 years ago

Good catch. That could work. I don't know much myself, but at least we have things tracked by git, so that we can experiment and improve.

Btw, I made a mistake in the cronjob, and it was executing every minute. Oops. I fixed it finally, now it should run every two hours.

certik commented 12 years ago

Ok, I think I have fixed this in 87a03cfe5d5632b1ebd6510b0bf14bfb5b1eefbc. I did a few tests and things seems to be working well. I am now running the bot every 20 minutes, and no change is pushed unless rss10.xml changes, see the commit. I am closing this one.

certik commented 12 years ago

I have to reopen it. Aaron, your blog is causing it to flip the RSS each time

asmeurer commented 12 years ago

I have no idea what's going on there. Obviously, my blog is just a vanilla WordPress blog, so I don't control any of that. Maybe it's a bug in planet.

By the way, why was gh-pages forced updated? Because of the 1 minute updates?

certik commented 12 years ago

I forced updated it, so that instead of 50+ MB, the git repo only has 2MB. For easier manipulation.

certik commented 12 years ago

So I think that we need to write a simple Python parser, that would parse the rss feed before and after, and tell the number and type of posts, somehow, and decide, whether something has changed or not.

asmeurer commented 12 years ago

Maybe we should contact planet and see if they've already solved this problem.

certik commented 12 years ago

I am currently a little busy, but it shouldn't be hard to fix this issue for good with a simple Python script, checking the changes with the help of git diff.