Onet test data

[thanathasCh]

Why this PR

Why we need this PR?

• This is a code to clean onet which is in microsoft word file into json. It will be easier to use for traning.

Changes

• Code to read word file text
• Convert all equation to readable LaTeX format (follow LLMU format)
• Save to json format as the following example
• Implement ignore case where we don't want to include these type of question, for example
  • question with image

Waiting for more test case from other onet subject

Related Issues

https://github.com/OpenThaiGPT/openthaigpt-pretraining/issues/64

Checklist

• [ ] PR should be in the Naming convention
• [ ] Assign yourself in to Assigneees
• [ ] Tag related issues
• [ ] Constants name should be ALL_CAPITAL, function name should be snake_case, and class name should be CamelCase
• [ ] complex function/algorithm should have Docstring
• [ ] 1 PR should not have more than 200 lines changes (Exception for test files). If more than that please open multiple PRs
• [ ] At least PR reviewer must come from the task's team (model, eval, data)