Dicom File Larger than 2 GB

[Murtaza-Farooqi]

DICOMFileReader.Read throws an exception in case of file is larger than or equal to 2 GB. Array dimensions exceeded supported range.

I am using .Net 4.7.2 and the latest stable version of Evil-Dicom from nuget packages i.e. 2.0.6.5

Any help would be much appreciated.

[crcrewso]

Can you try compiling your code as x64 only (not any) that 2 GB file limit could be due to the old windows 32-bit max file limits.

[Murtaza-Farooqi]

Yes i have already tried it to compile as x64 but got the same error.

[itwouldbewise]

C# has a (VERY unfortunate in my opinion) limit on a couple of things.

(1) the number of bytes in a single C# object (like an array) - limited to 2GB, but can be overridden with this https://hrnjica.net/2012/07/22/with-net-4-5-10-years-memory-limit-of-2-gb-is-over/ (2) the number of elements in an array (limited to ~2 billion). To my knowledge there is no way around this limitation.

So if you are dealing with a List then I believe (you might try it and confirm) that even with gcAllowVeryLargeObjects enabled that you will still have a problem since there is still the 2 billion element limit and therefore gcAllowVeryLargeObjects doesn't actually help you at all. I believe the gcAllowVeryLargeObjects would only help with something like an int[] where each element is 4 bytes, so you could have 2 billion int values which is ~8GB...

Anyway that's my understanding of it, but I could be wrong.

On Mon, Mar 4, 2024 at 11:08 PM Murtaza-Farooqi @.***> wrote:

Yes i have already tried it to compiling on x64 but got the same error.

— Reply to this email directly, view it on GitHub https://github.com/rexcardan/Evil-DICOM/issues/106#issuecomment-1978031163, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK2CCXVPQUCLR4IRKZP46TYWVOOZAVCNFSM6AAAAABEEVVVTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYGAZTCMJWGM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[isachpaz]

check this out:

https://learn.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/gcallowverylargeobjects-element?redirectedfrom=MSDN

On Tue, Mar 5, 2024 at 7:08 AM Murtaza-Farooqi @.***> wrote:

Yes i have already tried it to compiling on x64 but got the same error.

— Reply to this email directly, view it on GitHub https://github.com/rexcardan/Evil-DICOM/issues/106#issuecomment-1978031163, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANS7VSAZQ5E2S7NKWECYJTYWVOO3AVCNFSM6AAAAABEEVVVTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYGAZTCMJWGM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[Murtaza-Farooqi]

check this out: https://learn.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/gcallowverylargeobjects-element?redirectedfrom=MSDN … On Tue, Mar 5, 2024 at 7:08 AM Murtaza-Farooqi @.> wrote: Yes i have already tried it to compiling on x64 but got the same error. — Reply to this email directly, view it on GitHub <#106 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANS7VSAZQ5E2S7NKWECYJTYWVOO3AVCNFSM6AAAAABEEVVVTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYGAZTCMJWGM . You are receiving this because you are subscribed to this thread.Message ID: @.>

That didn't help. Still getting the Array Dimension exceeded the supported range.

I am getting the error in DICOMFileReader.Read Method. on the below line. elements = metaElements.Concat(DICOMElementReader.ReadAllElements(dr, syntax, enc)).ToList();

[itwouldbewise]

Yes, as I mentioned, the gcAllowVeryLargeObjects setting doesn't actually do anything for byte[] or List since there is still the number-of-elements limitation of 2^31, and I'm guessing that the DICOM file has an element that EvilDICOM is parsing into a List.

As far as I know, you are out of luck parsing a >2^31 byte DICOM element with .NET C#. I can think of a few alternatives:

1. Adding some hacks like splitting it up the element into multiple separate List or byte[] (I bet this would be really messy to do this in EvilDICOM, but of course the alternative would be to roll your own DICOM parser).
2. If you have control over the DICOM data generation, generate it with a smaller amount of data. (is this a deformable registration file? What kind of DICOM file is it?)
3. You could try using a C++ (or alternative) DICOM parser to parse it into something that .NET C# can use.

Not sure if there's much else you could do... Like I said, this is a super unfortunate design decision that Microsoft engineers made with the .NET architecture.

Message ID: @.***>

[Murtaza-Farooqi]

thanks for your suggestion. It is a Mamograph DICOM image. I think I should mark this issue as close now.