This pull request restructures the repository to allow for library building and includes additional scripts for constructing the library and documentation.
Changes:
The src/ directory has been renamed to turkish_lm_tuner. The fine-tune script has been moved under experiments/, and the evaluation script has been divided into Evaluator classes (turkish_lm_tuner/evaluator.p) and scripts for conducting experiments, which are now located under experiments/eval.py. Likewise, configuration files have been moved to 'experiments'.
Bash scripts have been updated to reflect the changes in the new structure.
projectto.mland env.yml have been added for library building.
mkdocs.yml and a docs folder, which includes markdown files and tutorial notebooks, have also been included.
BuildREADME.md, which includes instructions for building the library and documentation, has been added for internal use.
This pull request restructures the repository to allow for library building and includes additional scripts for constructing the library and documentation.
Changes:
The
src/directory has been renamed toturkish_lm_tuner. The fine-tune script has been moved underexperiments/, and the evaluation script has been divided into Evaluator classes (turkish_lm_tuner/evaluator.p) and scripts for conducting experiments, which are now located underexperiments/eval.py. Likewise, configuration files have been moved to 'experiments'.Bash scripts have been updated to reflect the changes in the new structure.
projectto.mlandenv.ymlhave been added for library building.mkdocs.ymland adocsfolder, which includes markdown files and tutorial notebooks, have also been included.BuildREADME.md, which includes instructions for building the library and documentation, has been added for internal use.Fixed several errors:
model_save_pathineval.pyhttps://github.com/boun-llm/turkish-lm-tuner/blob/4b2e91df04e928473cd0ec355ac13ca6c656774e/src/eval.py#L72 by replacing it withmodel_path.DatasetProcessorwith logging.