search

mclevey / podlm

Probabilistic Opinion Dynamics with Language Models
MIT License
1 stars 0 forks source link

Refactor anything in John's pipeline prototype that needs to be refactored #19

Open mclevey opened 10 months ago

mclevey commented 10 months ago

I produced this pipeline, as well as the podlm and curry packages in a roughly 6 week manic sprint. It's a "minimal viable product (MVP)" for the team to improve while I'm on parental leave, twinning all day every day. 😳 😀 ❤️

My goal was to prototype the whole pipeline from start to finish (except for the final Bayesian stance models, which I am still prototyping and will not think much about until my parental leave if finished), and as such there are many things that could be improved. At this point I think I've fixed most of the big things, but there are still some small inconsistencies in how I do things, and there are likely many places where I could be doing things in more efficient ways. As you review and wrap your head around this code base, think about whether there are better / more efficient ways we could be doing things, because this is a complex pipeline that will take a while to run on the full Reddit dataset.

mclevey commented 10 months ago

Labelling this as "help wanted," but it's not really a proper "Issue."