search

AI4Finance-Foundation / FinGPT

FinGPT: Open-Source Financial Large Language Models! Revolutionize 🔥 We release the trained model on HuggingFace.
https://ai4finance.org
MIT License
13.84k stars 1.92k forks source link

Update tokenize_dataset_rows.py #26

Closed Shahupdates closed 1 year ago

Shahupdates commented 1 year ago

In this revised code, I've added exception handling for file operations and JSON parsing, replaced bool with argparse.BooleanOptionalAction for --skip_overlength argument, used the constant MODEL_NAME for better maintainability, and improved memory efficiency by reading the file line by line.

zhumingpassional commented 1 year ago

thanks for your codes.

Shahupdates commented 1 year ago

No problem, happy to contribute.