search

monitorjbl / excel-streaming-reader

An easy-to-use implementation of a streaming Excel reader using Apache POI
Apache License 2.0
959 stars 345 forks source link

High CPU utilisation spikes recorded #262

Open jayantjain001 opened 2 years ago

jayantjain001 commented 2 years ago

Hi team, I am trying to read max size xlsx files of around 112 MBs . I am able to read them properly but my server which stays at around 30-40% CPU usage normally shows frequent CPU spikes of 90-98% while reading the file . Means this file reading jar is itself hitting CPU of around 20-30 % utilisation. My usage of jar is as below : `

  InputStream is = null;
  Workbook workbook = null;

  int minLineDataLength=columnCount;

  try {
    is = new FileInputStream(new File(filePath));

    workbook = StreamingReader.builder()
        .rowCacheSize(100)
        .bufferSize(4096)
        .password(password)
        .open(is);`
jayantjain001 commented 2 years ago

@monitorjbl Please provide help for the same , or a change in config can help . I can keep at max 5,000 rows in memory as it is a multi threaded code... Please suggest what changes in config are required to keep cpu util low

jayantjain001 commented 2 years ago

@pjfanning can you help here ?

jayantjain001 commented 2 years ago

Just updated please help @pjfanning @monitorjbl