jmbejara / comp-econ-sp19

Main Course Repository for Computational Methods in Economics (Econ 21410, Spring 2019)
48 stars 26 forks source link

HW 6 (SQL Part) - Feedback system marks results 'Incorrect' falsely #47

Closed Killavata closed 5 years ago

Killavata commented 5 years ago

The feedback system used in the linked Kaggle notebooks will sometimes mark the answers incorrect, even when it's not the case. Moreover, using the proposed solutions will often yield 'Incorrect' result as well.

Also, I believe that the dataset referenced in the 'As & With' notebook has changed since the notebook was posted, so the Q6 and the proposed solution to it aren't relevant anymore.

jmbejara commented 5 years ago

Thanks for pointing this out. Which specific questions are producing errors? One thing that helped for me is resetting the query config every time. Also, casting the max bytes argument to an int seems to help with reliability. That is, I changed the line

# Set up the query
query_job = client.query(zero_pollution_query, job_config=safe_config)

to this

# Set up the query
safe_config = bigquery.QueryJobConfig(maximum_bytes_billed=int(1e9))
query_job = client.query(zero_pollution_query, job_config=safe_config)

every time it came up.

I'm going to take a look at the 'As & With` right now.

jmbejara commented 5 years ago

I took a look at it. Yep. Just skip Q6 on "As and With". And, yeah, it's too bad that some of the proposed solutions are showing as incorrect (though they are correct). Recent changes must have broken parts of the tutorial. For those seeing this post, these are the parts of the tutorial that I'm finding errors in:

Thanks for pointing these out!