rapid7 / metasploit-framework

Metasploit Framework
https://www.metasploit.com/
Other
34.17k stars 13.98k forks source link

gh-pages branch is unreasonably huge #4756

Closed todb-r7 closed 9 years ago

todb-r7 commented 9 years ago

Need to do something like this:

https://github.com/richfitz/wood/blob/master/.update-gh-pages.sh

in order to keep git fetch upstream reasonable. Right now, every change touches every generated file, so the fetch times are getting slower and slower. By deleting history, we can save a lot of time on the usual fetch traffic.

cc @wwalker-r7 just so he's aware, though I'll fix (unless you want to beat me to it, Wyatt).

todb-r7 commented 9 years ago

This will do the trick to wipe out history on the branch.

git checkout gh-pages
git branch -m gh-pages-old
checkout --orphan gh-pages
mkdir /tmp/gh
mv index.html /tmp/gh/
mv api/ /tmp/gh/
git rm -rf .
mv /tmp/gh/index.html .
mv /tmp/gh/api/ .
git add index.html 
git add api/
git commit -m "Reboot gh-pages"
git push origin --force
todb-r7 commented 9 years ago

Opened internal ticket GES-2461 to track this.

wwalker-r7 commented 9 years ago

I've assigned the Jira ticket to myself. I'll be looking at this soon.

firefart commented 9 years ago

Why not omit the "generated at" times from the yardoc generation? This way we would have a clean commit history. Example on how to do it with custom templates: http://stackoverflow.com/questions/10959850/how-to-customize-diff-git-to-ignore-yard-date-generation/11444144#11444144

todb-r7 commented 9 years ago

While I think that'd be good for a revision tracking point of view, I've been under the impression that gh-pages style publishing isn't really intended for versioning. You're generally supposed to version the stuff that makes the output, but the output itself should be treated like a blob. When I was working on http://resources.metasploit.com, I was horrified at all the force pushing going on in the docs on how to Middleman deploy, but then realized this is pretty normal. @mbuck-r7 is probably more articulate than me.

todb-r7 commented 9 years ago

At any rate, I don't really care to track the history of the API docs. I'd rather just track the history of the API itself. And I think this is done now, right @wwalker-r7 ?

wwalker-r7 commented 9 years ago

The Jenkins job is updated, and the change to the template for our internal tool is up for code review. As such, it'll clear the history every week after the site is updated.

erran-r7 commented 9 years ago

@todb-r7 not sure the context around this but why not just link to rubydoc.info's metasploit-framework docs?

todb commented 9 years ago

Mostly because i like hosting the API docs nearby the source repo. And also because I didn't realize rubydoc regenerated so often from github repos, I thought that was just a gem thing. :)

Sent from a tiny computer.