search

aribornstein / MathTeacherChallenge

Microsoft Datahack 2018 Math Teacher Challenge
MIT License
3 stars 1 forks source link

A few questions #2

Open YotamSandbank opened 6 years ago

YotamSandbank commented 6 years ago
  1. What is the target column which we will be scored by? Is it 'ans_simple'? Do we also need to provide the equations?
  2. What is the evaluation metric?
  3. Which datasets can we use to train our model?
  4. Will we be provided with a test set consisting of only the raw text questions? When will we get that set? How many rows are in it?
  5. Will there be an option to submit our answers before the end of the competition to find out how well we are doing?
ghost commented 6 years ago

We join the questions

aribornstein commented 6 years ago
  1. you are scored by ans_simple order doesn't matter we expect for you to write the evaluation code\
  2. The evaluation metric is hard accuracy based on ans_simple we will also ask you to present to me the model and approach you took to ensure that you didn't over fit the test set or cheat
  3. You can use any dataset as long as it doesn't contain any values in the test set
  4. The test set has already been provided see the attached notebook
  5. There is no evaluation tool but you can come find me and I will be happy to guide, provide advice and tell you how you are doing