Closed Chawak closed 10 months ago
Please DO:
1. Make `NUM_PROC` adjustable in the Hydra configuration file (`
All modified and coverable lines are covered by tests :white_check_mark:
Comparison is base (
a9ec1c7
) 94.15% compared to head (149eec8
) 94.15%. Report is 1 commits behind head on main.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Why this PR
Why we need this PR? This PR is for this issue https://linear.app/openthaigpt/issue/LM-206/refactor-common-crawl-dataset-pipeline-to-use-the-latest-metadatajson
Changes
Related Issues
Close #
Checklist