search

move-coop / parsons

A python library of connectors for the progressive community.
https://www.parsonsproject.org/
Other
264 stars 132 forks source link

[Bug] van.upload_scores() takes a Parsons Table, creating memory issues #755

Open elyse-weiss opened 2 years ago

elyse-weiss commented 2 years ago

van.upload_scores() takes a Parsons Table as one of its inputs. When loading scores, you area almost always submitting a full state. We've experienced that this often creates memory issues when actually running scripts using this function. Seems like it may make more sense to have a csv or other file as the input.

Detailed Description

N/A

To Reproduce

Try loading scores to Texas or California ;)

Your Environment

We are running this function within Civis Platform.

Additional Context

N/A

Priority

Medium

Jason94 commented 2 years ago

Having an easy way to do this makes sense. It would be an API breaking change. If we weren't comfortable with that, we could (1) create a second method (upload_scores_csv or something). Or (2) change the current method to check if the tbl parameter is a parsons.Table or a string, and treat the string as a CSV filename.

elyse-weiss commented 2 years ago

I would totally be ok with creating a second method!