Won't open some of my .mbox files

[Tom5421]

Hi,

I have some mbox files generated by https://github.com/joeyates/imap-backup but for some reason it won't open them.

I have used mboxviewer for multiple Gmail exports and it worked well, so I believe I am using it correctly. I am just not sure why it won't open the files produced by imap-backup.

Any suggestions would be helpful.

[zigm]

Hi,

MBox Viewer will try to validate that the file is of the mbox type based on header fields. I sometimes tweak the criteria so it is always possible that the validation is too strict. Another possibility I believe is that file name contains non Ansi characters. I am in the process of porting MBox Viewer to UNICODE but this turned out to be more challenging an time consuming than I expected.

Anyway, MBox Viewer requires From: header field to be present in the mail header. I also consider To, MessageID, Subject and Mime Version header fields during validation.

If you can provide small sample mail archive file or first say 1000 lines of very large archive., I will investigate and resolve the reported issue.

Also, let me know what version of MBox Viewer you are running.

Zbigniew

[Tom5421]

Hi,

I am using version v1.0.3.37.

I can't seem to find a MessageID part in the file.

Is there a way to disable these validations?

Thanks, Tom

[zigm]

Hi,

There is no option to disable the validation. Possible it might be useful for people with expertise like you.

Current validation criteria are as below and it could be relaxed I suppose or disabled if user is sure that the archive file is valid.

bMboxFrom indicates that first header line such as below was detected.

From 1572669128982579937@xxx Tue Jul 11 23:02:36 +0000 2017

MessageID is optional in many cases but you will need MIME-Version: field instead.

Hope you can generate one of these two fields in combination with To, FROM and Subject..

In the past I was considering to allow user to configure a set of the header fields for validation but it was never implemented.

if (bFrom)
{
    if (( bMimeVersion && (bMboxFrom_ || bMessageID)) ||
        (bMboxFrom && bMessageID) ||
        ((bFrom && bSubject) && (bMimeVersion || bMessageID)) ||
        ((bTo && bSubject) && (bMimeVersion || bMessageID))
        )
}

Thanks, Zbigniew

[Tom5421]

Yeah, I think having an option to disable the validation would be excellent and this would hopefully be quite a quick addition to the tool - Potentially saving any future issues of similar nature too.

What do you think about adding this?

[zigm]

Lack of any validation is pretty risky since MBox Viewer attempts to validate all files, including binary, in a given folder and/or sub-folders if requested.

MBox Viewer implements defensive code in many cases but lack of any validation may sometimes crash Viewer I suppose. Nobody wants to use crashing unreliable application -:).

More header fields that can be used to validate, less likely we will see problems.

Sounds in your case you don't have messageId and MIME-version header fields.

I suppose that at least one of the header fields, From, To, Subject or Date, should be required to validate mail archive file in addition to required first line as below example (not required in *.eml files):

From 1572669128982579937@xxx Tue Jul 11 23:02:36 +0000 2017

More required fields used for validation, the better. Current MBox Viewer relies on messageId and MIME-version fields because they are very MIME specific/expected.

Do you have all or some of the From, To, Subject or Date header fields in your files ?

[Tom5421]

Yeah, so in my archive I have the following headers: From, To, Subject & Date.

From: 1,213 occurrences To: 7,582 occurrences Subject: 177 occurrences Date: 188 occurrences

I'm not sure why the number of from/to varies so much.

[zigm]

Hi,

I will add an option to relax mail file validation to support cases similar to yours and I will try to create new package today.

But it seems to me you may have problem other than problem with MBox Viewer.

Not sure why number of Subject and Date occurrences is so low. Number of To occurrences might be much higher if you have large number of people on To list.

I prefer to relax the validation rules and not to disable completely.

MBox Viewer will assume file is valid if any of the From, To, Subject or Date header field is present. It will also verify that the mail header is valid. Header ends when line is blank. All header field labels must end with ':" character.

Hopefully that will work in your case. Let you know when package is ready.

[Tom5421]

That's great, I will be ready to test it as soon as possible.

Thank you!

[zigm]

Did a quick release of v1.0.3.38. Go ahead, try and let me know if the new released helps to resolve your case.

You set/reset relaxed mail file validation as follow:

File --> Development Options --> Relax Mail file Validation

The settings persists across startups, no warning if set.

[Tom5421]

Yep, that is working - Thank you for your efforts in doing this!

[zigm]

Hi,

Great. Regarding your project imap-backup. What if any library are you using to connect to IMAP and download mails?

Are you using oauth2 for user authentication?

MBox Viewer has an option to connect to SMTP server and forward emails. I rely on MailKit open source library to implement this capability. I implemented oauth2 but it was too much hassle and I support application password only for authentication.

It looks like MBox Viewer has to be blessed by Google as trusted application. Also, user would see Pop up Screen asking to accept something, don't remember. It scared me, I suspect users would be scared too.

[Tom5421]

I'm not using OAuth, just using the IMAP/SMTP protocols, you can however enable IMAP support for a Gmail account see here: https://support.google.com/mail/answer/7126229

Hope this helps

[zigm]

Thanks