Investigate if large PDF generation can be done in a more RAM-efficient way

jrochkind commented 4 years ago

Re #302, creating our large multi-page PDFs is taking a huge amount of RAM, making resource management/sysadmin more complex.

Can we create multi-page PDFs in a more RAM efficient way?

We might want lower-resolution PDFs -- this would result in smaller file size for users to download, and might be preferable? currently PDFs can be pretty massive. Although current size is based on trying to give 300dpi for printing. Lower-res PDFs would also result in lower RAM consumption.
We could try to use different tools for PDF creation. We are currently using prawn. Other tools may be more efficient in their use of RAM or allow "Streaming" PDF writing in some way. That could be the ruby tool HexaPDF, or it could be shelling out to a non-ruby command line tool of some kind.

jrochkind commented 3 years ago

We actually already did this.

jrochkind commented 3 years ago

This is about generating multi-page PDFs ("on-demand derivatives"), not about generating JPG thumbnails from PDFs. I don't think it's relevant to the other issue... which I don't have at hand and isn't linked here.

eddierubeiz commented 3 years ago

Sorry - you're absolutely right!

sciencehistory / scihist_digicoll

Investigate if large PDF generation can be done in a more RAM-efficient way #592