coobird / thumbnailator

Thumbnailator - a thumbnail generation library for Java
MIT License
5.08k stars 780 forks source link

Multi-threaded processing of images with large pixel width and height leads to memory OutOfMemoryError #208

Closed Lychengit closed 1 year ago

Lychengit commented 1 year ago

Expected behavior

Please describe what you are expecting the library to perform.

Multi-threaded processing of images with large pixel width and height leads to memory OutOfMemoryError

Actual behavior

Please describe the actual behavior you are experiencing, including stack trace and other information which would help diagnose the issue.

demo: JVM(jdk8):-Xmx300m

public static ThreadPoolExecutor buildCheckThreadPool() { ThreadFactory threadFactory = new ThreadFactoryBuilder() .setNameFormat("-thread-%d") .setUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() { @Override public void uncaughtException(Thread t, Throwable e) { e.printStackTrace(); } }) .build(); ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(25, 25, 60L, TimeUnit.MILLISECONDS, new LinkedBlockingDeque(100), threadFactory, new ThreadPoolExecutor.AbortPolicy() );

    return threadPoolExecutor;
}

public static void main(String[] args) throws IOException, InterruptedException {
    ThreadPoolExecutor threadPoolExecutor = buildCheckThreadPool();
    System.out.println("start!!!!!!!!!!!!!!");
    System.setProperty("thumbnailator.conserveMemoryWorkaround", "true");
    File file = new File("D:\\fcs\\test\\test\\像素宽高10000左右.jpeg");
    for (int i = 0; i < 5; i++) {
        final int finalI = i;
        threadPoolExecutor.execute(new Runnable() {
            @Override
            public void run() {
                long t = System.currentTimeMillis();
                try (ByteArrayOutputStream outputStream = new ByteArrayOutputStream()) {
                    Thumbnails.of(new FileInputStream(file)).size(2700, 2700).outputFormat("png").toOutputStream(outputStream);
                    byte[] bytes = outputStream.toByteArray();
                    saveFile("D:\\fcs\\test\\test\\test"+ finalI+".png", bytes);
                } catch (Throwable e) {
                    e.printStackTrace();
                    System.out.println("image"+ finalI);
                }
            }
        });
    }
}

public static void saveFile(String filepath,byte [] data)throws Exception{
    File file  = new File(filepath);
    FileOutputStream fos = new FileOutputStream(file);
    fos.write(data,0,data.length);
    fos.flush();
    fos.close();
}

image

Steps to reproduce the behavior

Please enter step-by-step instructions for reproducing the actual behavior. Including code can be helpful in diagnosing issue, but please keep the code to a minimal that will reproduce the behavior.

Original image used for testing: 像素宽高10000左右

Environment

Please provide vendor and version information for the Operating System, JDK, and Thumbnailator. Please feel free to add any other information which may be pertinent.