Open jLynx opened 2 years ago
I'm aware. I'm still trying to resolve this with HF, so maybe give it a bit of time :)
I'm interested in doing some research with similar models in the near future, have you considered making the model available via torrent? I'd happily seed the model indefinitely ;)
Idk why I didn't think to check the archive, thanks for the link. Like I said, I'll seed it long term for those who are interested in the model.
Note: I cloned the repository (not this one, the model repo) on my GitHub account, and I uploaded the model itself on the Internet Archive (32bit version).
Someone on Reddit reached out to me and asked for the 16bit version. I didn't have it, but I found it on 4chan., so I told the Reddit user. They uploaded it too to the Internet Archive and started seeding it.
Admittedly, I am not seeding: I don't use the model personally and I don't have the resources to. However, I invite anyone who wants and can to seed (for this, I thank @Captain-Wet-Beard and all others that are seeding the model).
It is extremely important, for all information, that there isn't a potential censor, or a single point of failure. While trying to convince HuggingFace to allow the model is good, we should not depend on them for our usage of the model.
@yk, I noticed that in https://huggingface.co/ykilcher/gpt-4chan/discussions/4 you asked if you can advertise an alternative download source on HF. Rather than doing it on HF (it's not unreasonable for them to forbid this), I suggest that you make a whole video advertising the new download source. This is something HF cannot prevent in any way, and might actually get (and probably will get) even more attention than a link there. Also, if you plan to use BiTorrent to distribute this, I invite you to use the same files, which people are already seeding.
I think an open source licensed notebook which clones the repo trough Git, downloads the model trough torrent and sets the model running would be helpful.
Note: for those who do not trust me, or random users on Reddit and 4chan (as you shouldn't), Yannic himself published the MD5 hashes of the model here: https://huggingface.co/ykilcher/gpt-4chan and on this comment: https://huggingface.co/ykilcher/gpt-4chan/discussions/4#62a8f9b39e44ab41605b70a3
This is quite smart, because the repo only contains SHA256 hashes, but the Internet Archive uses MD5 in the files.xml file.
So, even without downloading anything, you can go check the hashes on the Internet Archive. I can confirm they are correct for both models.
Torrents. The only way to permanently put something on the internet.
Using download managers for downloads via HTTP protocol is a little more complicated than using P2P. By complexity I mean that some servers do not allow you to request data starting from an arbitrary position. This can happen due to a variety of reasons, but the most basic ones are specific configuration (e.g. because of own policy for downloading files) and outdated server software.
I think @Captain-Wet-Beard asked for a torrent primarily because of convenience.
Hi there, where is the conflg.json for the backup stored on archive?
@cynthiaio see: https://github.com/Aspie96/gpt-4chan-model
thank you for the torrents! <3
4Chan model 2024 😁
I downloaded the torrent but am not sure how to load the model into LMstudio. Can anyone help?
The model that you linked to here https://huggingface.co/ykilcher/gpt-4chan has been removed. Any chance you could reupload somewhere else?