search

propublica / politwoops_sunlight

Politwoops web front end
Other
44 stars 30 forks source link

politwoops-worker.py painfully slow #47

Open tttp opened 6 years ago

tttp commented 6 years ago

Hi,

I'm trying to run my own instance, the worker pushing into the db is really slow (like 1 tweet per minute slow).

Could you push the schema? I might miss an index or have a weird table

What's the configuration you have for beanstalk? it might be that I don't have enough available connection too

(bit in the dark, shooting from the hip, any suggestion welcome)

dwillis commented 6 years ago

@tttp Sorry about the delay in responding to this. We've seen similar issues, and I don't think it's the database. Our beanstalk config is pretty vanilla, nothing out of the ordinary.