Data is almost ready (some issues with stories breaking when temp is 2, I have to re-generate too many that are below min-tokens, resulting in a CUDA error)
Loading generated data has been streamlined (see utils/process_generations) so that we can easily load data based on its temp/prompt number (or decide to load it all)
Some small considerations
Will not be spending much time on this (as we are otherwise very close to having all data fully generated) but will briefly look into:
Look into updating vllm -> performance updates + new models + seed per request. May actually fix problems that sometimes occurs with CUDA
Whether it would make sense to generate with llama7b also
Some future, not-urgent tasks
Consider whether code in misc needs to be placed somewhere else (do we need that code?)
Updates
utils/process_generations) so that we can easily load data based on its temp/prompt number (or decide to load it all)Some small considerations
Will not be spending much time on this (as we are otherwise very close to having all data fully generated) but will briefly look into:
Some future, not-urgent tasks
miscneeds to be placed somewhere else (do we need that code?)srcwith overview of folders