messense / mupdf-rs

Rust binding to mupdf
GNU Affero General Public License v3.0
109 stars 24 forks source link

Segmentation fault whenever multi-threading with rayon #85

Open kkew3 opened 6 months ago

kkew3 commented 6 months ago

I use mupdf like this:

use camino::{Utf8Path, Utf8PathBuf};
use mupdf::error::Error;
use mupdf::document::Document;
use rayon::prelude::*;

fn pdftotext(pdf: Utf8PathBuf) -> Result<String, Error> {
    let file = Document::open(pdf.as_str())?;
    let mut content = String::new();
    for page in file.pages()? {
        content.push_str(&page?.to_text()?);
    }
    Ok(content)
}

fn main() {
    // populate many pdf paths:
    let many_pdf_files: Vec<Utf8PathBuf> = vec![];
    // seg fault:
    let results: Vec<_> = many_pdf_files.into_par_iter().map(pdftotext).collect();
}

pdftotext is called on many pdf files. Whenever I use rayon to parallelize the program, seg fault occurs. The more threads I use, the faster seg fault happens. The program won't crash if I simply iterate over all pdf files in a single thread (e.g., replacing into_par_iter() with into_iter()).

I'm using mupdf version 0.4.2 from https://crates.io/crates/mupdf.

kkew3 commented 6 months ago

I try the main branch (a7e5f64ca383c69c532f78cec44274bc148961f0) today. The seg fault issue disappear. But a new issue occurs. With mupdf-0.4.2 on crates.io, single threaded, it takes 310 seconds to convert 2000+ pdf files to text, 5 of them tagged as containing invalid utf-8. With the main branch, three parallel threads, it takes 450 seconds to convert the same collection of pdf files to text, 30 of them tagged as having the utf8 error. In conclusion, the conversion becomes slower and issues more utf-8 errors than before. I report this issue separately at #86.