Sent Email Not Contain Email Address

[rtarling]

Hi

If I send an email from my Outlook client, then parse the sent item from my Sent folder, I can't get hold of the correct senderEmail. Instead I see:

/o=WinUKMail/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=95fcdac4177a4b5caebfd0a70fc21041-19ocs1340040

The senderName is my display name, which is as expected.

There is no header information that I can parse either. Outlook can display the correct address so I assume it is buried in the source object somewhere.

I would be very grateful if you could look into this for me as I am a bit stuck. I can send you an example msg file if it helps.

Kind Regards Roland

[kenjiuno]

Hi, could you provide .msg file including this situation? If you don't mind please attach it to ku@digitaldolphins.jp

[kenjiuno]

Hi.

Although I'm not good at Exchange Server product, this seems to be normal behavior.

The mail is sent from Exchange transport (not SMTP). The mail address prefix IMCEAEX is Internet Mail Connector Encapsulated Address (IMCEA) encapsulation method using EX transport.

• Recipient resolution in Exchange Server | Microsoft Docs
• How the proxyAddresses attribute is populated in Azure AD - Active Directory | Microsoft Docs

You'll be able to check the address in Attribute Editor in Active Directory Users and Computers: https://activedirectoryfaq.com/2014/10/ad-attribute-editor-missing-make-search-visible/

• Proxy-Addresses attribute - Win32 apps | Microsoft Docs
• Legacy-Exchange-DN attribute - Win32 apps | Microsoft Docs

[malynel]

Hi, i ran into the extact same issue. Could you figure out what goes wrong? Is a .msg still needed? Also tested @freiraumio ... strangely enough with that, senderEmail is not always present. Could also provide screenshots from results if needed. Regards

[kenjiuno]

Ok I have updated msgreader https://www.npmjs.com/package/@kenjiuno/msgreader/v/1.5.3 with a fix: https://github.com/HiraokaHyperTools/msgreader/commit/f94a53a4f2542e0194595461cf63242d00761489

creatorSMTPAddress or lastModifierSMTPAddress will point a valid email address, if they are available.

Especially it tends to be attached if the mail is sent from Exchange Server like Exchange Online (using Microsoft 365 accounts) or such.

They are described at Meaning of MAPI identifiers 0x5D0A and 0x5D0B

[malynel]

Hi again! Thanks for your quick fix, i appreciate that! Unfortunately i forgot to mention earlier, that the email address of an recipients object does also have this exchange-style address if refering to an internal address. Is there any chance to get the normal address and add it to one recipient as well? Regards

[kenjiuno]

Ok I have published 1.5.4 https://www.npmjs.com/package/@kenjiuno/msgreader/v/1.5.4

smtpAddress in recipients will be optionally available on Exchange transport.

[malynel]

Hi, thanks again for your quick response. Very nice, but there is still one last thing missing. I have some mails, where neither creatorSMTPAddress nor lastModifierSMTPAddress is present. In that cases the smtp address resides in0x3ffa.

Found https://github.com/bbottema/outlook-message-parser/blob/master/src/main/java/org/simplejavamail/outlookmessageparser/model/OutlookMessage.java and experimented with that and your package. Got about 20 mails and can reproduce that behavoir.

• from external -> smtp address present in senderEmail
• from internal -> smtp address present in creatorSMTPAddress and/or lastModifierAddress
• from internal, but creatorSMTPAddress and lastModifierAddress missing -> present at 0x3ffa

I would appreciate if you could (one last time) add this field. thanks in advance and regards :-)

[kenjiuno]

Ok 1.5.5 has been published https://www.npmjs.com/package/@kenjiuno/msgreader/v/1.5.5

Please check if email is contained in lastModifierName (0x3ffa) in some specific cases...

In my case, my name is stored in lastModifierName:

If you utilize Microsoft Outlook, OutlookSpy 4.1 (https://www.dimastr.com/outspy/home.htm) can be used for deep property inspection:

[rtarling]

Many thanks for your hard work on this - much appreciated.

So to summarise, to get hold of the Sender's email address, should I go down the following list in the order shown, testing for a valid email address, until I hit bingo :-)

• senderEmail
• smtpAddress
• creatorSMTPAddress
• lastModifierSMTPAddress
• lastModifierName

Thanks Roland

[malynel]

Hi, also a big thank you from me. I have now tested with my 20 mails and always get the sender and address back as expected. Thanks again ;)

[kenjiuno]

Thanks everyone, now closing.

[DevPan29]

Hi @kenjiuno , I comment on this thread because I have the same issue: as other users sometimes, in the senderEmail field I receive /O=EXCHANGE/OU=EXCHANGE ADMINISTRATIVE GROUP... and I already used the creatorSMTPAddress as field to get the senderEmail, but I noticed that sometimes the senderName (that is the name of the sender) it's different from the creatorSMTPAddress that take as value my address (and this is wrong).

In my case, the correct value for the FROM field, is the senderName and I would this value in email-format Have you some hints, on how to do it?

I don't know why, the creatorSMTPAddress field take as value my address; I'm not the sender of the emails that I'm testing.

[kenjiuno]

@DevPan29 Hi thanks for reporting, if you can, please attach the .msg file (having its problem) against ku@digitaldolphins.jp?

[kenjiuno]

@DevPan29 Ok you can try 1.7.4-alpha.1.

inetAcctName will help.

• https://www.npmjs.com/package/@kenjiuno/msgreader
• https://hiraokahypertools.github.io/msgreader/typedoc/interfaces/MsgReader.FieldsData.html#inetAcctName
• https://github.com/HiraokaHyperTools/msgreader/blob/4370966248909d4490e4c3cac42978520bdaf5a0/test/voteItems.json#L19

[DevPan29]

Hi @kenjiuno , I've just tried the version 1.7.4-alpha.1 , but I have verified that in the inetAcctName field almost for my cases, contains the TO recipient and not the FROM recipient. Here some logs:

the senderName field contains the right name of the sender (FROM) but I'm not able to retrieve his address; the inetAcctName field contains the right email of the receiver (TO); the creatorSMTPAddress field contains my email address, but I'm not involved in the email thread; the lastModifierName field contains another name , that isn't involved in the email thread.

Kind regards.

[kenjiuno]

@DevPan29 Currently I don't have good recommendation of this.

Alternatively at 1.7.4-alpha.2 you can try node cli parse --include-raw-props test\sent.msg in order to review all kind of properties stored in msg.

• https://hiraokahypertools.github.io/msgreader/typedoc/interfaces/MsgReader.FieldsData.html#rawProps
• https://hiraokahypertools.github.io/msgreader/typedoc/interfaces/MsgReader.RawProp.html

This output will include rawProps.

Please try node cli parse --include-raw-props MSGFILE against your msg file, and then if you have any interests set of propertyTag, propertySet, and propertyLid please let me know.

{
  dataType: 'msg',
  attachments: [],
  recipients: [
    {
      dataType: 'recipient',
      name: "'xmailuser@xmailserver.test'",
      rawProps: [Array],
      email: 'xmailuser@xmailserver.test',
      addressType: 'SMTP',
      recipType: 'to'
    }
  ],
  rawProps: [
    {
      propertyTag: '0064001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'SMTP'
    },
    {
      propertyTag: '003b0102',
      propertySet: undefined,
      propertyLid: undefined,
      value: [Uint8Array]
    },
    {
      propertyTag: '0c1f001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'xmailuser@xmailserver.test'
    },
    {
      propertyTag: '001a001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'IPM.Note'
    },
    {
      propertyTag: '00410102',
      propertySet: undefined,
      propertyLid: undefined,
      value: [Uint8Array]
    },
    {
      propertyTag: '0c190102',
      propertySet: undefined,
      propertyLid: undefined,
      value: [Uint8Array]
    },
    {
      propertyTag: '0e0a0102',
      propertySet: undefined,
      propertyLid: undefined,
      value: [Uint8Array]
    },
    {
      propertyTag: '0037001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'Sent time'
    },
    {
      propertyTag: '003d001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: ''
    },
    {
      propertyTag: '0042001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'xmailuser'
    },
    {
      propertyTag: '0070001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'Sent time'
    },
    {
      propertyTag: '0c1d0102',
      propertySet: undefined,
      propertyLid: undefined,
      value: [Uint8Array]
    },
    {
      propertyTag: '0e03001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: ''
    },
    {
      propertyTag: '1000001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'Test mail\r\n\r\n'
    },
    {
      propertyTag: '0065001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'xmailuser@xmailserver.test'
    },
    {
      propertyTag: '00710102',
      propertySet: undefined,
      propertyLid: undefined,
      value: [Uint8Array]
    },
    {
      propertyTag: '0c1a001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'xmailuser'
    },
    {
      propertyTag: '0c1e001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'SMTP'
    },
    {
      propertyTag: '0e02001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: ''
    },
    {
      propertyTag: '0e04001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: "'xmailuser@xmailserver.test'\x00"
    },
    {
      propertyTag: '0e28001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: '00000003\x01xmailuser@xmailserver.test\x01xmailuser@xmailserver.test'
    },
    {
      propertyTag: '300b0102',
      propertySet: undefined,
      propertyLid: undefined,
      value: [Uint8Array]
    },
    {
      propertyTag: '0e1d001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: 'Sent time'
    },
    {
      propertyTag: '0e29001f',
      propertySet: undefined,
      propertyLid: undefined,
      value: '00000003\x01xmailuser@xmailserver.test\x01xmailuser@xmailserver.test'
    },
    {
      propertyTag: '10090102',
      propertySet: undefined,
      propertyLid: undefined,
      value: [Uint8Array]
    },
    {
      propertyTag: '8004001f',
      propertySet: '00062008-0000-0000-c000-000000000046',
      propertyLid: '00008554',
      value: '15.0'
    }
  ],
  senderEmail: 'xmailuser@xmailserver.test',
  subject: 'Sent time',
  body: 'Test mail\r\n\r\n',
  senderName: 'xmailuser',
  senderAddressType: 'SMTP',
  compressedRtf: Uint8Array(1409) [
    125,   5,   0,   0, 184,  11,   0,   0,  76,  90,  70, 117,
    221, 181, 128, 185,   3,   0,  10,   0, 114,  99, 112, 103,
     57,  51,  50,  65,   3,  67, 104, 116, 109, 108,  49,   3,
     49,  98, 252, 105, 100,   4,   0,   3,  48,   1,   3,   1,
    247,  10, 128,   2, 164, 131,   3, 228,   5, 225,  80,  71,
    111, 116, 104,  13, 224, 255,   2, 128,  16, 115,   0,  80,
      4,  86,   5, 225,  17, 155,  14,  65,   3,   1,   9,   2,
      0,  99, 104,  10, 192, 115, 101, 116,  50,  55,   6,   0,
      6, 195,  17, 245,
    ... 1309 more items
  ],
  creationTime: 'Mon, 15 Feb 2021 08:19:21 GMT',
  lastModificationTime: 'Mon, 15 Feb 2021 08:19:21 GMT',
  clientSubmitTime: 'Mon, 15 Feb 2021 08:19:04 GMT',
  messageDeliveryTime: 'Mon, 15 Feb 2021 08:19:00 GMT'
}

[DevPan29]

Dear @kenjiuno , I'm not able to process the email object through the command node cli parse --include-raw-props test\sent.msg ; I've cloned the repo that actually has the version 1.7.4-alpha.3 running these operations:

git clone https://github.com/HiraokaHyperTools/msgreader.git
cd msgreader
yarn
node cli parse --include-raw-props mail_problem.msg

and I receive this error:

internal/modules/cjs/loader.js:984
  throw err;
  ^

Error: Cannot find module 'commander'
Require stack:
- C:\SRDEV\workspaces\MsgReadder kenjuno clone\msgreader\cli.js
←[90m    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:981:1
5)←[39m
←[90m    at Function.Module._load (internal/modules/cjs/loader.js:863:27)←[39m
←[90m    at Module.require (internal/modules/cjs/loader.js:1043:19)←[39m
←[90m    at require (internal/modules/cjs/helpers.js:77:18)←[39m
    at Object.<anonymous> (C:\SRDEV\workspaces\MsgReadder kenjuno clone\msgreader\
cli.js:1:17)
←[90m    at Module._compile (internal/modules/cjs/loader.js:1157:30)←[39m
←[90m    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
←[39m
←[90m    at Module.load (internal/modules/cjs/loader.js:1001:32)←[39m
←[90m    at Function.Module._load (internal/modules/cjs/loader.js:900:14)←[39m
←[90m    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main
.js:74:12)←[39m {
  code: ←[32m'MODULE_NOT_FOUND'←[39m,
  requireStack: [
    ←[32m'C:\\SRDEV\\workspaces\\MsgReadder kenjuno clone\\msgreader\\cli.js'←[39m

  ]
}

I receive the same error also if I run the simpler node cli -h.

I've installed also npm install commander --save and now the error is changed: Error: Cannot find module './lib/MsgReader'

Do you have any hints? Thank you for your support.

[kenjiuno]

Hi.

These commands are in right way, no problem.

git clone https://github.com/HiraokaHyperTools/msgreader.git
cd msgreader
yarn
node cli parse --include-raw-props mail_problem.msg

From your output of node cli, yarn seems to fail the preparation of msgreader.

Error: Cannot find module 'commander'

Do you observe any problems from output of yarn command?

This is my yarn command output on Windows 10:

yarn
yarn install v1.22.4
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@2.3.2: The platform "win32" is incompatible with this module.
info "fsevents@2.3.2" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > typedoc-plugin-rename-defaults@0.1.0" has incorrect peer dependency "typedoc@0.20.x".
[4/4] Building fresh packages...
$ npm run build && npm run test

> @kenjiuno/msgreader@1.7.4-alpha.3 build
> npm run build:types && npm run build:js

> @kenjiuno/msgreader@1.7.4-alpha.3 build:types
> tsc --emitDeclarationOnly

> @kenjiuno/msgreader@1.7.4-alpha.3 build:js
> babel src --out-dir lib --extensions .ts

Successfully compiled 10 files with Babel (1840ms).

> @kenjiuno/msgreader@1.7.4-alpha.3 test
> npm run mocha

> @kenjiuno/msgreader@1.7.4-alpha.3 mocha
> set NODE_ENV=test && mocha

  MsgReader
    test1.msg
      √ exact match with pre rendered data (except on compressedRtf)
    test2.msg
      √ exact match with pre rendered data (except on compressedRtf)
      √ verify attachment: A.txt
    msgInMsg.msg
      √ exact match with pre rendered data (except on compressedRtf)
      √ testMsgAttachment0 === testMsgAttachments0
      √ re-parse and verify rebuilt inner testMsgAttachments0
    msgInMsgInMsg.msg
      √ exact match with pre rendered data (except on compressedRtf)
      √ re-parse and verify rebuilt inner testMsgAttachments0
      √ re-parse and verify rebuilt inner testMsgAttachments0AndItsAttachments0
    Subject.msg
      √ exact match with pre rendered data (except on compressedRtf)
    sent.msg
      √ exact match with pre rendered data (except on compressedRtf)
    sent2.msg
      √ exact match with pre rendered data (except on compressedRtf)
    longerFat.msg
      √ re-parse and verify rebuilt inner testMsgAttachments0
    longerDifat.msg
      √ re-parse and verify rebuilt inner testMsgAttachments0
    attachAndInline.msg
      √ exact match with pre rendered data (except on compressedRtf)
    voteItems.msg
      √ exact match with pre rendered data (except on compressedRtf)
    voteNo.msg
      √ exact match with pre rendered data (except on compressedRtf)
    voteYes.msg
      √ exact match with pre rendered data (except on compressedRtf)

  Burner
    √ known boundary tests
    - sequential boundary tests

  toHexStr
    √ tests

  DataStream
    √ little.readUint32
    √ big.readUint32
    √ little.offset.readUint32
    √ big.offset.readUint32
    √ little.buffer.readUint32
    √ little.buffer.offset.readUint32

  msftUuidStringify
    √ basic

  toHex
    √ toHex1
    √ toHex2
    √ toHex4

  30 passing (124ms)
  1 pending

Done in 14.98s.

Also please confirm the version of node and yarn. Although msgreader doesn't declare engines in package.json, msgreader may not run on older version of node such as v10.

C:\>node -v
v14.17.3

C:\>yarn -v
1.22.4

[DevPan29]

Hi @kenjiuno , I finally processed my mails (I needed almost node version 12.20) and I can tell you that for my specific cases I found what I need in these fields: 5d02001f

5d01001f

So please, could you kindly make them available as readeable field in the MsgData?

Kind regards.

[kenjiuno]

Ok 1.7.4-alpha.4 is available now.

Please check:

• https://hiraokahypertools.github.io/msgreader/typedoc/interfaces/MsgReader.FieldsData.html#senderSmtpAddress
• https://hiraokahypertools.github.io/msgreader/typedoc/interfaces/MsgReader.FieldsData.html#sentRepresentingSmtpAddress

[DevPan29]

Hi @kenjiuno , all works as expected. Thank you so much for your support.

will you release a version without suffix -alpha.4 ?

Kind regards.

[kenjiuno]

Yes 1.7.4 has been published just now.

https://www.npmjs.com/package/@kenjiuno/msgreader/v/1.7.4

[DevPan29]

Great, thank you again.

[DevPan29]

Hi @kenjiuno , doing some tests all works and the FROM field is correctly retrieved, but unfortunately I'm not able to process another mail, that I used for test. using node cli parse --include-raw-props mail.msg

I receive this error:

C:\workspaces\msgreader\lib\MsgReader.js:24
function _createForOfIteratorHelper(o, allowArrayLike) { var it = typeof Symbol !== "undefined" && o[Symbol.iterator] || o["@@iterator"]; if (!it) { if (Array.isArray(o) || (it = _unsupportedIterableToArray(o)) || allowArrayLike && o &&
typeof o.length === "number") { if (it) o = it; var i = 0; var F = function F() {}; return { s: F, n: function n() { if (i >= o.length) return { done: true }; return { done: false, value: o[i++] }; }, e: function e(_e) { throw _e; }, f:
F }; } throw new TypeError("Invalid attempt to iterate non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method."); } var normalCompletion = true, didErr = false, err; return { s: function
 s() { it = it.call(o); }, n: function n() { var step = it.next(); normalCompletion = step.done; return step; }, e: function e(_e2) { didErr = true; err = _e2; }, f: function f() { try { if (!normalCompletion && it["return"] != null) it[
"return"](); } finally { if (didErr) throw err; } } }; }

                                     ^

RangeError: Invalid typed array length: 524297
    at new Uint8Array (<anonymous>)
    at Function.memcpy (C:\workspaces\msgreader\lib\DataStream.js:1764:19)
    at DataStream.readUint16Array (C:\workspaces\msgreader\lib\DataStream.js:538:18)
    at DataStream.readUCS2String (C:\workspaces\msgreader\lib\DataStream.js:1140:52)
    at MsgReader.fieldsNameIdDir (C:\workspaces\msgreader\lib\MsgReader.js:536:36)
    at MsgReader.fieldsDataDirInner (C:\workspaces\msgreader\lib\MsgReader.js:249:14)
    at MsgReader.fieldsDataDir (C:\workspaces\msgreader\lib\MsgReader.js:454:16)
    at MsgReader.fieldsDataReader (C:\workspaces\msgreader\lib\MsgReader.js:566:12)
    at MsgReader.parseMsgData (C:\workspaces\msgreader\lib\MsgReader.js:577:19)
    at MsgReader.getFileData (C:\workspaces\msgreader\lib\MsgReader.js:594:32)

I can told you that the same mail with the 1.6.6version of your library works.

Have you any hints on what could be the cause of error?

[kenjiuno]

Ok 1.7.5-alpha.1 may fix that kind of error.

[DevPan29]

Kind @kenjiuno , unfortunately after installing 1.7.5-alpha.1 I received the same error.

[kenjiuno]

Hi @DevPan29 could you send me the msg file that causes the problem? For now I cannot decide how to fix about this. Possibly msgreader's bug or msg file corruption.

[DevPan29]

Hi @kenjiuno , as you said, I think it's a problem related to the corruption of the msg file. For now I'll ignore it. What final version should I use? 1.7.4 ?

[kenjiuno]

Hi, this is explanation for developers about RangeError: Invalid typed array length: 524297 error:

The reason 1.6.6 is workable:

• Named properties parser is not shipped then.
• [MS-OXMSG]: Named Property Mapping Storage | Microsoft Docs

The reason 1.7.4 will not work:

• Named properties parser has shipped since 1.7.0. 62ee14768e0872d317eda3a746c6a953cb5ab0ec
• The parser will throw an error if it inputs msg having broken named properties.

The bumping to 1.8.x is for internal purpose: changing the compiler from Babel to tsc.

For now 1.7.4 is latest stable. @kenjiuno/msgreader - npm

[DevPan29]

hi @kenjiuno , thanks again for your support.

Best regards.

[DevPan29]

deal @kenjiuno , I would have yet need your help: managing a mail sent from a SHARED MAILBOX with the current exposed fields from MsgData I'm not able to retrive the sender. So after processing the email object with the command node cli parse --include-raw-props I would have kindly need of this field exposed:

propertyTag: '8016001e', propertySet: '00062008-0000-0000-c000-000000000046', propertyLid: '00008580'

Could you support me on this request?

Kind regards.

[kenjiuno]

Hi @DevPan29 That property PidLidInternetAccountName is already exposed as inetAcctName. https://github.com/HiraokaHyperTools/msgreader/blob/d01e674db27716a4481bec1957645b32f43df678/src/MsgReader.ts#L387-L404

Isn't it exposed?

[DevPan29]

Hi @kenjiuno , I'm prettry sure that that value refers to the TO recipient. I tested it with other mails and I match the TO. unfortunately my test mail in this case has the same FROM and TO, for this reason I'm not able to detect what is the right value to expose.

I'll try to find another solution, because in the sender field I have the string 'EXCHANGE,BLABLA' Kind regards

[DevPan29]

Hi @kenjiuno , I've just sent you to this email address 'ku@digitaldolphins.jp' an email object to process.

Kind regards. Alessandro

[kenjiuno]

Thanks I have received zipped msg from you.

I think that inetAcctName doesn't mean neither from/to address. It is only informational to Outlook: this mail should belong to that mail account written in inetAcctName.

If you send a mail from me@consoto.com to you@consoto.com, inetAcctName will be me@consoto.com (msg in sent folder).

If you received a mail from you@consoto.com (and if you use me@consoto.com), inetAcctName will be me@consoto.com (msg in inbox folder).