search

salvois / LargeXlsx

A .net library to write large Excel files in XLSX format with low memory consumption using streamed write.
Other
204 stars 34 forks source link

Add ZIP64 support for huge files #3

Closed easysummer closed 3 years ago

easysummer commented 3 years ago

Hello!

Thanks for such a cool little library.

I got an exception when trying to write a huge file.

System.NotSupportedException: "Attempted to write a stream that is larger than 4GiB without setting the zip64 option"
  at SharpCompress.Writers.Zip.ZipWriter.ZipWritingStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.Write(String value)
at System.IO.StreamWriter.Write(String format, Object[] arg)
at LargeXlsx.Worksheet.Write(String value, XlsxStyle style)
at LargeXlsx.XlsxWriter.DoOnWorksheet(Action action)
at testApp.Program.Main(String[] args) in /.../csharp/testApp/Program.cs:49

I have not found a parameter on using the zip64 flag in the library API. Can you tell me how to solve this problem?

I used .NET Core 5 version. Also tried to use .NET 4.6.2 on a Windows machine - same exception. Version library - latest (1.2.0).

❯ dotnet --info
.NET SDK (reflecting any global.json):

Version:   5.0.100
 Commit:    5044b93829

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.15
 OS Platform: Darwin
 RID:         osx.10.15-x64
 Base Path:   /usr/local/share/dotnet/sdk/5.0.100/

Host (useful for support):
  Version: 5.0.0
  Commit:  cf258a14b7

.NET SDKs installed:
  3.1.404 [/usr/local/share/dotnet/sdk]
  5.0.100 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.10 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.10 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
salvois commented 3 years ago

Hello @easysummer thanks for reporting this issue! I'm having a look and be back to you as soon as possible.

salvois commented 3 years ago

Hello @easysummer looks like enabling the ZIP64 format is not enough to support your use case. The SharpCompress library, used by LargeXlsx, appears to produce a ZIP file that is not compatible with Excel when enabling ZIP64. Excel complains about a corrupted file but it is able to repair it (LibreOffice, on the other hand, appears to have no support for ZIP64 altogether). I will try to understand what Excel considers wrong about the generated file (looks like https://rzymek.github.io/post/excel-zip64/ may be a good start) and try to patch SharpCompress. Thanks, Salvo

salvois commented 3 years ago

Hello @easysummer I've just opened a pull request to SharpCompress that solves the issue (https://github.com/adamhathcock/sharpcompress/pull/601), letting us to enable ZIP64. Hope it is considered for merge, so that a next version of LargeXlsx can take advantage of it. Thanks, Salvo

salvois commented 3 years ago

Hello @easysummer I have just released version 1.3.0 of LargeXlsx, which lets you enable the ZIP64 compression. Please see the updated documentation about constructing an XlsxWriter.

I have tested ZIP64 both on small files with Excel and LibreOffice with no problems, and on a file larger than 4 GiB uncompressed, which works fine in Excel but it is reported as corrupt in LibreOffice. I don't know the reason for the latter and further testing could be helpful.

Look forward to your feedback! Thanks, Salvo