Can't link the github repo in the Gemfile when deploying on Heroku

[cyril-sf]

I'm working on a Rails 4 app and needed the latest code from datashift. As it's not released yet, my Gemfile directly links to a commit from the repo.

Everything worked fine until I tried to deploy on Heroku:

The slug size exceeds 300MB, and the main reason is that the datashift git repo needs more than 200MB.

I've tracked down the problem to this folder.

231M vendor/bundle/ruby/2.0.0/bundler/gems/datashift-bfada2fd1319/.git/objects/pack

I don't have currently the time to find which commit is responsible.

[autotelik]

Hi.

think it relates to large files checked in by accident and removed, but seems git delete does not remove them from full history

...been looking into how to remove them completely from history as well but seems slightly tricky so WIP

... perhaps someone withreally good git skillscan advise

In meantime, gem version 0.15.0 I think is released with latest code

or since you wont need to commit any changes, perhaps you can do a shallow checkout without the history, try with  the --depth 1 option

  cheers tom

albums   : http://aqwan.bandcamp.com all music:http://soundcloud.com/aqwan

---

From: Cyril Fluck notifications@github.com To: autotelik/datashift datashift@noreply.github.com Sent: Wednesday, 20 August 2014, 20:25 Subject: [datashift] Can't link the github repo in the Gemfile when deploying on Heroku (#23)

I'm working on a Rails 4 app and needed the latest code from datashift. As it's not released yet, my Gemfile directly links to a commit from the repo. Everything worked fine until I tried to deploy on Heroku: The slug size exceeds 300MB, and the main reason is that the datashift git repo needs more than 200MB. I've tracked down the problem to this folder. 231M vendor/bundle/ruby/2.0.0/bundler/gems/datashift-bfada2fd1319/.git/objects/pack I don't have currently the time to find which commit is responsible. — Reply to this email directly or view it on GitHub.

[cyril-sf]

@autotelik Yes, if I'm correct, a data.tar.gz was pushed at some point.

If I have some time, I'd be happy to learn more git to figure out how to solve that problem.

In the meantime, I have already solved the problem by creating an empty repo into which I pushed the lastest code ( https://github.com/cyril-sf/datashift ).

[pnomolos]

@autotelik @cyril-sf Perhaps take a look at http://rtyley.github.io/bfg-repo-cleaner/

Alternatively, I could look at this, as this issue's also affecting me :)

[cyril-sf]

I don't know if it's possible to remove a file in a previous commit without rewriting all the commits after that one ( I'm making the assumption that this commit is still in the master branch ).

And if this is the case, what are the implications (beside that works on the repo needing to probably do a git reset --hard).

[pnomolos]

Implications are basically that you have to do a git push --force, so anyone downstream who wants to remain close to the original codebase may have some issues when they rebase off of master (the suggested procedure to make sure no old commits are re-introduced). Github has a pretty good article on the process https://help.github.com/articles/remove-sensitive-data

[autotelik]

ok thanks for the tip .. I'm gonna try this http://rtyley.github.io/bfg-repo-cleaner/ ... Implications of current sitution seem dire so think better to at least try and clean it up

[autotelik]

ok, I have deleted the big blobs using this tool and pushed to github. Simple clone seems much faster now and the repo looks to be considerably smaller .... so I will close this Issue for now

du -c -h -s datashift 262M datashift 262M total

du -c -h -s datashift_cleaned 35M datashift_cleaned 35M total

[cyril-sf]

:thumbsup:

[pnomolos]

:thumbsup: Way, way, way faster to clone this time :)

[autotelik]

thanks for the feedback fellas ! even better that it's positive, that was bit nerve racking ;-)