search

DRAGNLabs / 301r_retnet

2 stars 1 forks source link

Update readme #32

Closed RLin8103 closed 4 months ago

RLin8103 commented 4 months ago

Pull Request: Update README

Overview This pull request updates the README document to provide a more comprehensive and detailed guide on the RetNet vs Transformers project. The revisions include an expanded introduction, detailed installation and setup instructions, and a thorough explanation of the project's features and functionalities. The aim is to make the project more accessible to new contributors and users, ensuring that they have all the information they need to get started.

Key Updates Project Introduction: Clarified the comparison and analysis focus between RetNet and Transformers, utilizing Microsoft's TorchScale. Reference to Original Paper: Added a reference to the foundational paper that inspires this project, enhancing academic credibility. Installation and Setup: Detailed steps for cloning the repository, setting up the Python environment with Mamba, and preparing the environment for running the project's scripts. Features Explanation: Elaborated on the Grid Search feature, Hugging Face and PyTorch Lightning integration, and TorchScale's role in the project. This section now provides a clearer view of the project's capabilities and its technical advancements. Execution Workflow: Outlined the steps to prepare and execute the project, including data download, tokenizer training, data tokenization, and model running instructions. Benchmarking and Results: Mentioned the use of EleutherAI's language model evaluation harness for empirical evaluation, linking to the project's research findings. Acknowledgments and Citations: Updated to recognize contributions from individuals and organizations, and provided citations for referenced works.

Reason for Changes The README update aims to streamline the project onboarding process, provide clear guidance on how to utilize the project's tools, and offer insights into the project's research and development focus. By enhancing the documentation, we hope to facilitate a better understanding and engagement with the project for developers, researchers, and enthusiasts alike.

Request for Review Feedbacks are welcomed on the updated README for any further clarifications or additions that might be necessary.

RLin8103 commented 4 months ago

@nprisbrey, @JayOrten, @JacksonSearle, @KimballJardine, and @calvin-seamons, you can review this pull request.

JayOrten commented 4 months ago

Seems good to me. I'd be interested in @KimballNJardine's thoughts on string quotes in the configuration file. I'd definitely be interested in the other reviewer's thoughts and fresh eyes (@JayOrten, @JacksonSearle, and @KimballNJardine).

See: https://stackoverflow.com/questions/19109912/yaml-do-i-need-quotes-for-strings-in-yaml

Obviously, it's a big discussion with no clear "right" answer. In general, strings in yaml files don't need quotes on strings. For our application and usage, and based on the weird problems John was having, I would say they are unnecessary for our use case. I'm not very concerned about being explicit considering the comments we have.

However, I haven't taken the time to dig into the issue John had, so maybe there's a better fix and we can keep the quotes if everyone really cares about them.

KimballNJardine commented 4 months ago

Seems good to me. I'd be interested in @KimballNJardine's thoughts on string quotes in the configuration file. I'd definitely be interested in the other reviewer's thoughts and fresh eyes (@JayOrten, @JacksonSearle, and @KimballNJardine).

See: https://stackoverflow.com/questions/19109912/yaml-do-i-need-quotes-for-strings-in-yaml

Obviously, it's a big discussion with no clear "right" answer. In general, strings in yaml files don't need quotes on strings. For our application and usage, and based on the weird problems John was having, I would say they are unnecessary for our use case. I'm not very concerned about being explicit considering the comments we have.

However, I haven't taken the time to dig into the issue John had, so maybe there's a better fix and we can keep the quotes if everyone really cares about them.

I am fine without the quotes.