edit pyproject / config dedup&decont

[kriangkraitan]

Why this PR

LM-152

Changes

• Write some changes here

Related Issues

Close #

Checklist

• [ ] PR should be in the Naming convention
• [ ] Assign yourself in to Assigneees
• [ ] Tag related issues
• [ ] Constants name should be ALL_CAPITAL, function name should be snake_case, and class name should be CamelCase
• [ ] complex function/algorithm should have Docstring
• [ ] 1 PR should not have more than 200 lines changes (Exception for test files). If more than that please open multiple PRs
• [ ] At least PR reviewer must come from the task's team (model, eval, data)

[linear[bot]]

LM-152 Integrate Main Data Pipeline to include OSCAR Colossal

Input: [image.png](https://uploads.linear.app/03a3f0b5-8e51-4d0f-918c-59e891b8184f/42346582-477b-4bcd-a25c-2796e5f43bd1/185ed459-1db3-4973-9d4e-2ff34cd44841) Output: [image.png](https://uploads.linear.app/03a3f0b5-8e51-4d0f-918c-59e891b8184f/d5cc9ede-582a-4974-a825-e507c59d5595/a575b2a0-8931-4f3b-87ff-94cf2f2df246)

[codecov[bot]]

Codecov Report

All modified lines are covered by tests :white_check_mark:

Comparison is base (5a27d13) 19.39% compared to head (a9ec1c7) 94.15%. Report is 25 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #318 +/- ## =========================================== + Coverage 19.39% 94.15% +74.76% =========================================== Files 25 10 -15 Lines 1392 291 -1101 =========================================== + Hits 270 274 +4 + Misses 1122 17 -1105 ``` | [Flag](https://app.codecov.io/gh/OpenThaiGPT/openthaigpt-pretraining/pull/318/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=OpenThaiGPT) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/OpenThaiGPT/openthaigpt-pretraining/pull/318/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=OpenThaiGPT) | `94.15% <ø> (+74.76%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=OpenThaiGPT#carryforward-flags-in-the-pull-request-comment) to find out more. [see 35 files with indirect coverage changes](https://app.codecov.io/gh/OpenThaiGPT/openthaigpt-pretraining/pull/318/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=OpenThaiGPT)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.