search

allenai / dolma

Data and tools for generating and inspecting OLMo pre-training data.
https://allenai.github.io/dolma/
Apache License 2.0
909 stars 94 forks source link

make_wikipedia in getting_started.md #125

Closed leeparkuky closed 5 months ago

leeparkuky commented 7 months ago

The code for running make_wikipedia needs to be edited. The one currently in the document

python scripts/make_wikipedia.py \ --output wikipedia \ --languages simple \ --date 20231001 \ --lang simple \ --num_processes 16

should be written as below:

python scripts/make_wikipedia.py \ --output wikipedia \ --date 20231001 \ --lang simple \ --processes 16

soldni commented 6 months ago

Thank you for the report! Would you have bandwidth for a quick PR?

RohitRathore1 commented 6 months ago

@soldni I have already submitted that PR which is resolving this issue.

soldni commented 5 months ago

closing since fixed in #117. Thanks again @RohitRathore1!