search

arashgholami / MaTT

LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs
2 stars 1 forks source link

Request for the MaTT Dataset #2

Open dyishappy opened 2 months ago

dyishappy commented 2 months ago

Request for the MaTT Dataset

Hello,

I recently came across your work on MaTT, and I'm very interested in exploring the dataset you mentioned. I noticed that the dataset hasn't been uploaded yet, and I was wondering if there is an expected release date or if there is any way I could gain access to it earlier for research purposes.

Your work on the hierarchical mathematical benchmark is fascinating, and I believe the dataset would be incredibly useful for my own research.

Thank you for your time and for sharing your work with the community!

arashgholami commented 2 months ago

Hi, thank you for your interest. Sure, I updated the Readme file, and you can find the dataset in the folder Data. Here is the link: https://github.com/arashgholami/MaTT/blob/main/Data/Data.jsonl Please let me know if you still have difficulty and cannot access to the Data.jsonl. Here is the paper: https://arxiv.org/abs/2406.05194 And here is the format of each question: "Path", "Question", "Choices", "Correct Answer". For example the first question is as follows:

{"Path": ["Mathematics", "Pure Mathematics", "Algebra", "Abstract Algebra", "Group Theory", "Group", "Definitions and group axioms"], "Question": "G= set of all integers, a.b=a-b, H=set of all positive integers, a.b = ab, where ab is the usual product of integers. Which one of G and H form a group?", "Choices": ["Only G", "Only H", "Both of G and H.", "None of them."], "Correct Answer": "D"}

dyishappy commented 2 months ago

Hello,

Thank you so much for your prompt response and for the detailed explanation. I truly appreciate your time and effort in sharing this information. I believe that the dataset will be immensely helpful for my research, and I’m excited to explore it further. 😄

arashgholami commented 2 months ago

You're very welcome and let me know if you have any further questions.