cbnsingram
commented
3 hours ago The training produced a less than accurate model. I have rerun the model. The warnings above did not appear this time and model is slightly more accurate.
Closed cbnsingram closed 3 hours ago
cbnsingram
commented
3 hours ago The training produced a less than accurate model. I have rerun the model. The warnings above did not appear this time and model is slightly more accurate.
erew123
commented
3 hours ago Hi @cbnsingram
Glad you got it working. Never seen that one before myself. Looks like its a intermittent/known bug with hugginfaces's transformers (looked up a few articles).
Ill at least add it to the list of known errors etc
Thanks
🔴 If you have installed AllTalk in a custom Python environment, I will only be able to provide limited assistance/support. AllTalk draws on a variety of scripts and libraries that are not written or managed by myself, and they may fail, error or give strange results in custom built python environments.
🔴 Please generate a diagnostics report and upload the "diagnostics.log" as this helps me understand your configuration.
diagnostics.log
Describe the bug The OrderedVocab you are attempting to save contains holes for indices [5024, 5025], your vocabulary could be corrupted !
To Reproduce I ran step one with no errors apparent. I looked at a test and found the output acceptable. I have now started running training and am getting this warning as each .pth is processed. Tuning is still running so I don't know if this is causing corruption of the model.
Screenshots If applicable, add screenshots to help explain your problem.
Text/logs If applicable, copy/paste in your logs here from the console.
Desktop (please complete the following information): AllTalk was updated: [11/25/2024] Custom Python environment: [no] Text-generation-webUI was updated: [11/25/2024]
Additional context This is my first fine tune project. If this is a known issue that relates to my dataset, please point me to how to troubleshoot.