search

kippnorcal / google_classroom

Google Classroom Data Pipeline
GNU General Public License v3.0
22 stars 9 forks source link

StudentSubmissions request writing dupes to db #34

Closed dchess closed 4 years ago

dchess commented 4 years ago 
StudentSubmissions(classroom_service).get_and_write_to_db(
            sql, course_ids, debug=config.DEBUG
        )

When this call encounters certain courses, it is loading way more records than exist in the course. For instance there is a course with 22 assignments and 29 students, that returns 100Ks of records. It should only be pulling back ~600+ records.

dchess commented 4 years ago

@zkagin I tried removing the tenacity retry decorator but that does not seem to be the cause of the issue.

dchess commented 4 years ago

https://github.com/kipp-bayarea/google_classroom/blob/2ae90c574d376cb0bf487050df80591ea11c5cc4/api.py#L393-L400

@zkagin: I think the issue is that this request object doesn't ever get passed the page_token!