Enable support for NDB_CRYPT_CYCLIC

[hughbe]

This implementation works as is

2003_high-encryption_quickbrownfox3.pst.zip

[hughbe]

Hmm we do seem to get an exception when reading attachment tables in the message with NID 2097220

{"Out of bounds error attempting to map HNPAGEMAP from byte array of length 2724 at offset 56657"}
    Data: {System.Collections.ListDictionaryInternal}
    HResult: -2146233088
    HelpLink: null
    IPForWatsonBuckets: 0x00007ffb135c3ddc
    InnerException: null
    IsTransient: false
    Message: "Out of bounds error attempting to map HNPAGEMAP from byte array of length 2724 at offset 56657"
    RemoteStackTrace: null
    Source: "XstReader"
    StackTrace: "   at XstReader.Map.MapType[T](Byte[] buffer, Int32 offset) in C:\\Users\\hughbe\\source\\repos\\XstReader\\Map.cs:line 27\r\n   at XstReader.LTP.ReadHeapOnNode(FileStream fs, UInt64 dataBid) in C:\\Users\\hughbe\\source\\repos\\XstReader\\LTP.cs:line 744\r\n   at XstReader.LTP.ReadTableInternal[T](FileStream fs, BTree`1 subNodeTree, UInt64 dataBid, PropertyGetters`1 g, Action`2 idGetter, Action`2 storeProp) in C:\\Users\\hughbe\\source\\repos\\XstReader\\LTP.cs:line 432\r\n   at XstReader.LTP.ReadTable[T](FileStream fs, BTree`1 subNodeTree, NID nid, PropertyGetters`1 g, Action`2 idGetter, Action`2 storeProp) in C:\\Users\\hughbe\\source\\repos\\XstReader\\LTP.cs:line 153\r\n   at XstReader.XstFile.ReadMessageTables(FileStream fs, BTree`1 subNodeTree, Message m, Boolean isAttached) in C:\\Users\\hughbe\\source\\repos\\XstReader\\XstFile.cs:line 503\r\n   at XstReader.XstFile.ReadMessageDetails(Message m) in C:\\Users\\hughbe\\source\\repos\\XstReader\\XstFile.cs:line 245\r\n   at XstReader.Mai
nWindow.listMessages_SelectionChanged(Object sender, SelectionChangedEventArgs e) in C:\\Users\\hughbe\\source\\repos\\XstReader\\MainWindow.xaml.cs:line 163"
    TargetSite: {T MapType[T](Byte[], Int32)}
    WatsonBuckets: null
    _HResult: -2146233088
    _className: null
    _data: {System.Collections.ListDictionaryInternal}
    _dynamicMethods: null
    _exceptionMethod: null
    _exceptionMethodString: null
    _helpURL: null
    _innerException: null
    _ipForWatsonBuckets: 0x00007ffb135c3ddc
    _message: "Out of bounds error attempting to map HNPAGEMAP from byte array of length 2724 at offset 56657"
    _remoteStackIndex: 0
    _remoteStackTraceString: null
    _safeSerializationManager: {System.Runtime.Serialization.SafeSerializationManager}
    _source: "XstReader"
    _stackTrace: {sbyte[384]}
    _stackTraceString: null
    _watsonBuckets: null
    _xcode: -532462766
    _xptrs: 0x0000000000000000

[Dijji]

Thanks for the encrypted pst, as I didn't have one of this kind. But what are the steps to reproduce the crash? I have downloaded your pst, and having removed the throw, I seem to be able to look at emails and see their attachments with no problems.

Also, for future reference, please do not submit pull requests directly to master. Instead, create a branch in your fork with your changes and use that for a pull request. This lets me merge your changes into a branch in my repository and test them before deciding whether to propagate them to master.

[hughbe]

Thanks for the encrypted pst, as I didn't have one of this kind. But what are the steps to reproduce the crash? I have downloaded your pst, and having removed the throw, I seem to be able to look at emails and see their attachments with no problems.

Yeah most of them work. Except for the 76th message!!

That is the message of NID 2097220 to assist with debugging

Also, for future reference, please do not submit pull requests directly to master. Instead, create a branch in your fork with your changes and use that for a pull request. This lets me merge your changes into a branch in my repository and test them before deciding whether to propagate them to master.

Sorry! Wasn't sure about how it all works

[Dijji]

If encryption weren't involved, I would say this was a simple case of a corrupt file. The heap header, HNHDR, is invalid almost everywhere, and given that other messages appear to be intact, corruption is the obvious answer.

However, as encryption is involved, the corruption could be the result of a subtle error in the decryption algorithm. It would have to be subtle, as it seems to be getting most things right. I'm not sure how to proceed to track any potential error down, and given the very limited damage that it is causing, even if it does exist, I'm not sure how much effort it is worth.

What do you think?

[hughbe]

It does look like some public PST viewers CAN read this: e.g.

I will do more checking

[hughbe]

BTW @Dijji do you have a library/collection of test PST and OST files - I’m using your lib to develop a swift port for a Mac/iOS app adj would like to be able to test it! Obviously I don’t want any personal data but if you have some that don’t contain information or whatever that would be awesome

Thanks for the library - it is AWESOME!