LucknowAI / Lucknow-LLM

Collecting data for Building Lucknow's first LLM
17 stars 27 forks source link

Improved create() method, now it can handle whole text file as input. #30

Closed thor-x-me closed 8 months ago

thor-x-me commented 8 months ago

Changes are as follows:

  1. Change in Prompt -JSON framework must be constructed with double-quotes, previously some of objects were constructed with single quotes. -Starting '[' and closing ']' bracket is removed now and being added manually in create() function of class TrainDataCreator. Previously while calling the function, output was stored as it is which also included brackets in it, each prompt was adding a pair of brackets in it's output, which was a tedious task to remove them manually, now it add '[' before the first and ']' after the last call, which provides proper format

  2. Change in train_data_creator.py

    • A new method is added to handle secret API key in local environment
    • Previous method create() is now takes whole file as input and provide a new file containing list of json objects in a format provided in prompt.txt