Throne3d / 2epub-various-ruby

A scraper of various glowfic collections
1 stars 0 forks source link

Speed up serialization (flagging objects dirty?) #9

Open Throne3d opened 7 years ago

Throne3d commented 7 years ago

It is currently extremely slow. Or, well, not that slow for how much stuff there is, but kinda, because a single chapter of about 3000 replies can take a second or a few to do and this is very slow by computer time. Ugh. Maybe look at flagging objects 'dirty' on changes – means lots of custom accessors and maybe some monkey patching of the hashes and arrays? (Or "hashes" and "arrays" that are actually wrappers around them that then mark dirty on interaction through the limited set of things available.) Ugh.

Done a bit? Also included oj as a JSON serializer which seems to have sped things up some. Unsure.

It ate so much memory omg, disabled for Alicorn's sandboxes. Augh.