inukshuk / jekyll-scholar

jekyll extensions for the blogging scholar
MIT License
1.13k stars 101 forks source link

Reduce jekyll-scholar build time? #347

Open FlamingLasrSwrd opened 1 year ago

FlamingLasrSwrd commented 1 year ago

The build time for a single post with only 10 references jumps from 1.3 seconds to 215 seconds with jekyll-scholar enabled:

Screenshot from 2022-10-29 18-35-41

Screenshot from 2022-10-29 18-36-29

Building the whole website of 23 posts (with --profile and --trace) takes more than 90 minutes: Screenshot from 2022-10-29 22-58-14

Is this typical behavior? My default bib file has 4000+ references. Is that just too large for jekyll-scholar? Each post only has a few dozen references drawn from --files of a few hundred references. Is there something I can change in Zotero to speed things up?

inukshuk commented 1 year ago

This is probably a legitimate issue, because the bibliographies are currently not cached. So if you render a lot of pages, or even few pages with different bibliographies using the --file option, the bib files get parsed again and again. If the files are large and parsing takes a few seconds this just deteriorates quickly.

We need to add a bibliography cache to address this.

gwgundersen commented 1 year ago

+1 for this feature.

When I add a citation to my .bib file, my entire site is rebuilt. This takes 20-30 minutes. Here, the length of time isn't Jekyll Scholar per se; it's Jekyll Katex re-rendering all the pages. However, maybe a bibliography cache would help Jekyll know which pages need to be rebuilt?