search

stamparm / ipsum

Daily feed of bad IPs (with blacklist hit scores)
The Unlicense
1.58k stars 145 forks source link

Cannot pull origin when list updates, need to re-clone instead #9

Closed yllekz closed 4 years ago

yllekz commented 4 years ago

Hi there, I'm using GitHub Desktop and trying to pull the changes when this list updates, however I get a "Unable to merge unrelated histories in this repository" error every time, resulting in the need for me to re-clone the repo any time there is an update.

Is this repo being fully republished with every update making it incompatible with pulls?

stamparm commented 4 years ago

1) Yup. The reason is pretty simple: avoiding GBs of junk inside the history (effectively forcing regular users to pull and store unnecessary data). For example, each day there are more than 250k IP addresses being pushed to the 1.txt list. As a result, history diffs become huge pretty fast. 2) This was not a decision made in haste. At the beginning, history has been preserved, which resulted with almost unusable git operations over the repository 3) Maybe something like https://github.com/desktop/desktop/issues/1484#issuecomment-484938631 could be of a help to you (i.e. switch --allow-unrelated-histories)

jonathanhoskin commented 4 years ago

As a lurking watcher of this repo, might I suggest using Git LFS to manage the pain of large files in the repo?

stamparm commented 4 years ago

@jonathanhoskin switched all *.txt to Git LFS :)

12345

jonathanhoskin commented 4 years ago

Nice one 👍

stamparm commented 4 years ago

@jonathanhoskin will need to switch back :(

It seems that ipsum is being pulled/downloaded by many people, thus the problem with bandwidth.

222

jonathanhoskin commented 4 years ago

Gah! Sorry to hear that. LFS does seem like the way forward while you are choosing to host files on Github.

While I'm not in a position to provide ongoing financial support, I am in a position to provide this project with free LFS hosting, if you want it.

Let me know if you are interested.

stamparm commented 4 years ago

@jonathanhoskin let it be like this for now. People are already used to old links for direct access (Note: Git LFS required changes to raw access from raw to media files).

p.s. Anyhow, if this will still be bothering you in a month or so, please come back and we'll could figure something out