reasoning-machines / pal

PaL: Program-Aided Language Models (ICML 2023)
https://reasonwithpal.com
Apache License 2.0
462 stars 58 forks source link

Bugs in gsmhardv2.json #21

Open vazirim opened 3 months ago

vazirim commented 3 months ago

Hello,

Thank you for the gsmhardv2.json dataset. Here are 2 bugs in the 3rd and 17th entries in it:

3rd entry: input: James decides to run 1793815 sprints 1793815 times a week. He runs 60 meters each sprint. How many total meters does he run a week?

code:

def solution():
    """James decides to run 3 sprints 3 times a week.  He runs 60 meters each sprint.  How many total meters does he run a week?"""
    sprints_per_day = 1793815
    days_per_week = 3
    meters_per_sprint = 60
    total_sprints = sprints_per_day * days_per_week
    total_meters = total_sprints * meters_per_sprint
    result = total_meters
    return result

The days_per_week variable in the code does not correspond to 1793815 a week. The docstring has incorrect information wrt to the input problem.

17th entry:

input: Marie ordered one chicken meal that costs $12, 5 packs of milk that costs $3 each, 4 apples that cost $1.3299286 each, and some boxes of pizza. Marie paid a total of $3299286. How many boxes of pizza did Marie order if each box costs $8.3299286?

code

def solution():
    """Marie ordered one chicken meal that costs $12, 5 packs of milk that costs $3 each, 4 apples that cost $1.50 each, and some boxes of pizza. Marie paid a total of $50. How many boxes of pizza did Marie order if each box costs $8.50?"""
    chicken_meal = 12
    milk_packs = 5
    milk_cost = 3
    apples = 4
    apple_cost = 1.5
    pizza_cost = 8.5
    total_paid = 3299286
    total_cost = chicken_meal + milk_packs * milk_cost + apples * apple_cost
    pizza_cost_total = total_paid - total_cost
    pizza_boxes = pizza_cost_total / pizza_cost
    result = pizza_boxes
    return result

The apple_cost variable should be $1.3299286 per the problem input. Again the docstring does not have the correct value (or the problem input needs to be changed).