BarfingLemurs
commented
1 year ago @Cerberus1746 the simplest way to get started, assuming you have enough vram, is to train with just a single text file.
Start by using a model such as llama, by launching text-generation-webui with --load-in-8bit with a 7B or 13B model.
After finding something like a youtube transcript or a paragraph of an article you like, put the text file in the training folder. (use however large amounts of data you like, I did it with a 47mb txt file.
Write a name for your lora, and find and select the text file from the gui, to use for your training data.
Click the train button.
After its done, you have your files in the Lora folder. You can go activate/use your lora from the settings.
leandro-benedet-garcia
oobabooga
qayyumabro
aarongerber
mcmonkey4eva
StevenXHK
Description As someone interested in studying the area of text creation, it is frustrating to see that a lot of models do not have the data I need in their models for the text generation to be useful to me. And considering the AI overlords are already overtaking people's jobs (I wish it was a joke) it is a risk that I am not willing to take to stay behind and not use those fantastic tools.
So I immediately after running this project and testing a couple models, I zeroed in the training tab, reading the parameters descriptions does indeed help, after being able to create a embedding in stable diffusion model without needing a google search, I thought I would be able to do the same using text-generation-webui
Nope, and still am unable to even after googling.
I am unable to train anything by just reading the parameters. They lack information or more parameters. I still have question in regards to how the new data needs to be formatted, does it need to be in a specific extension, where do I put the new data, do I need new packages in my environment, what tends to be the hardware requirements to train something, how long does it take (and yes, I know it varies a lot, but estimates are nice thing to have)
Who or what is lora? Acronyms are a very hard thing for someone to google because they can mean multiple things in a single language, and it gets worse when you are not in a English speaking country. Lora here can literally be a name. Of course after I had the illumination of appending the text "AI" to the search bar, I figured out what it meant.
I think it would be cool if the lora page mentioned or linked directly to something that says what lora is, I know it links to something, but it's not worded as something that you need to read or understand before continuing.
The documentation in this link needs improvements, because even though the "The parameters are self-documenting and good defaults are included." we have a single paragraph explaining the current method of how to train thing and then multiple paragraphs with coding samples that steal the attention of what you actually want and a single line explaining that you should just use the interface.
I have ADHD, I spent a entire day wondering what was the point of the UI if you had to use code until I noticed this line "Kept here for reference. The Training tab has much more features than this method." It doesn't have emphasis or anything, and considering it was on top of a block of code, I just thought it mentioned that you had to activate your conda enviroment, and considering I was using google collab which didn't use conda, I just ignored it.
But, we have another problem too. Even if you knew what you are doing, you wouldn't be able to train anything at all because of the bug in #1270 I am aware it's not your fault, but you should mention somewhere in the documentation about that bug. your documentation should be the main source of information about your project. If something prevents a feature to work and can be fixed, you need to mention that there.
Also, that issue was the main source of information to me about how to improve/train a model using the project, but since it doesn't mention where to put things to execute the training, I unfortunately still can't.