search

mbutterick / pollen-users

please use https://forums.matthewbutterick.com/c/typesetting/ instead
https://forums.matthewbutterick.com/c/typesetting/
53 stars 0 forks source link

Is parallel mode known to crash a lot? #114

Closed benknoble closed 2 years ago

benknoble commented 2 years ago

On small projects, I get a handful of early crashes, and then everything runs smoothly.

On a large project, I get 10s of early crashes, then sporadically one or two every 5.

(By "crash" I mean an individual job fails and is retried serially at the end, without issue.)

Some crashes I saw were caused by car on an eof, file-system issues with copying files or getting sizes, etc. (Some of these were diagnosed by adding to the with-handlers in parallel-render.)

mbutterick commented 2 years ago

Yes, it is more finicky, for reasons I’ve never been able to pin down. Roughly, it has something to do with race errors that arise when parallel places compete for access to the same set of files in the filesystem. Try using fewer parallel jobs.