search

SpamScope / spamscope

Fast Advanced Spam Analysis Tool
https://pypi.python.org/pypi/SpamScope
Apache License 2.0
292 stars 59 forks source link

SpamAssassin returns empty dictionary #17

Closed bmarsh9 closed 6 years ago

bmarsh9 commented 6 years ago

With certain emails, the output of Spamscope shows SpamAssassin as a empty Dictionary. If I run the email through the Spamassassin CLI with spamassassin -t , it parses it fine.

Is there any reason why Spamscope is returning a empty dictionary when it should not?

Thank you

bmarsh9 commented 6 years ago

Here is a example email that failed to show results for SpamAssassin:

Received: from prod.outlook.com (2603:10b6:910:15::25) by prod.outlook.com (2a01:111:e400:59ba::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1228.23; Wed, 1 Oct 2018 18:05:32 +0000 Received: from prod.protection.outlook.com (2a03:111:f400:7e32::216) by outlook.office365.com (2606:10b6:930:15::85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 17.20.1250.20 via Frontend Transport; Wed, 1 Oct 2018 18:05:32 +0000 Authentication-Results: spf=pass (sender IP is 100.145.126.221) smtp.mailfrom=example.com; example-to.com; dkim=none (message not signed) header.d=none;example-to.com; dmarc=bestguesspass action=none header.from=example.com; Received-SPF: Pass (protection.outlook.com: domain of example.com designates 107.14.166.232 as permitted sender) receiver=protection.outlook.com; client-ip=122.13.10.244; helo=cdptpa-cmomta01.email.rr.com; Received: from cdptpa-cmomta01.email.rr.com (122.13.166.212) by DF.protection.outlook.com (10.112.100.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1250.2 via Frontend Transport; Wed, 17 Oct 2018 18:05:32 +0000 Received: from cdptpa-web09 ([174.143.174.228]) by smtp with ESMTPA id CqCJgk0clvMX5; Wed, 17 Oct 2018 18:05:31 +0000 Message-ID: 20181345CD31.PKFYD.61244.root@example Date: Wed, 17 Oct 2018 14:05:31 -0400 From: John Test test@example.com To: toaddr@example.com Subject: RE: Office MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Priority: 3 (Normal) Sensitivity: Normal X-Originating-IP: from 11.58.28.211 by webmail.example.com; Wed, 17 Oct 2018 18:05:31 +0000 X-CMAE-Envelope: MS4wfEMbv1HIPsPZFd73LdHK4/W8+x7fwISDFSDFWERWERW4Ln7qrR08ueTt+Ck0TXp9GxRzVBc91UC4Q/kNYWDNQpNYav4SYtCdOLe0xi+TV1nxGGGYuVHJzBxH VaSXxONNFDFSzeC3SSoV43X9A2e2286OQf79263498234SDFSDFqv94Pb8an6clor1A== Return-Path: To@example.com X-MS-Exchange-Organization-Network-Message-Id: 85425318-e0a3-44ea-88450-0028340345 X-EOPAttributedMessage: 0 X-EOPTenantAttributedMessage: 4a809fb2-0c7f-4201-95da-06989273479236f:0 X-MS-Exchange-Organization-MessageDirectionality: Incoming X-Forefront-Antispam-Report: CIP:;IPV:NLI;CTRY:US;EFV:NLI; X-Microsoft-Exchange-Diagnostics: 1;;1:SK1Odwerwer3LkfpNAhzRQ345345tnRtinsk1+FYPmdUkrbAHMwu/HC9+CsQv5dOymEW79W6ZZOTtSBSmNx4DzyR6q3BHn6ZoYYvNGJ+cXpyQTb13eNZcyTaQPS X-MS-Exchange-Organization-AuthSource: protection.outlook.com X-MS-Exchange-Organization-AuthAs: Anonymous X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 85425318-e0a3-40ea-8810-08d6345b2132 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(5600074)(711020)(4605076)(4608076)(1401270)(1414027)(71702078)(7193020);SRVR:B0850; X-Microsoft-Exchange-Diagnostics: 1;;3:fuClKPbrERQpYgevg46k8eY2aUmhBhtxaczB6g1QG+xEO/nzGU3gsaegjEWiJxhILvS8hl6v5cfn9U2gQUs0bdzRCJBdjnpt4Jm4Oh11Nvr2kLYAQCB46Bzoo05qj5ji1OFgqUZSTelY2/JPeDnIIuqS19ZK/v5JD93fs1ReTTBv3gLmBhVOv8pM05EdOU+fZ/QVCg+30peKtk6kVXnrV/88+zKdJ26POvBq8+QYdoiuTcNzAFYhzt/UUA1mAbPUBvr/rgygVP7srGNrWZwtnJXR0KzZ3uOon6/i6s3PkaSJ0RA8U7EYJptWW52sfHIVMDfaytuzddoUkRYGLQkQbkaeBxeZR3N+IQrId1h1AM8=;25:Go3pc+J/J6qmYXFq3+euv/L47BST/F25lM/MZX+hJow334B3Y9HROI3gAEHzIfFrpstpatEUx1MSRAFk8GBkS9gbVR0FRiywS4X7dKN7RJAZTy5E1SmuS3NYXusjLGqzG/b6oISrXfIcXKm0qFsSCdWH6sxSinjh4psn1foLwO+eWGs8AnQ7mpkQSwv85Ca3/+9OG82eM60JhQp+zebPYV20le2QLdorvW5a3vDp6aLCF7CC/DfLgYwaU4hYC90tp9xUWGL6Vsb+AajUKyWV/dEq5dqYEfZN2prkojTclUhLSMZeQX4uo3LW9gLWeTiKOpjNr/z9Qu3AzPNe4duOFA== X-MS-TrafficTypeDiagnostic: BN4P389273409: X-LD-Processed: 4a809fb2-0c7f-4201-95da-06953b7d506f,ExtAddr

bmarsh9 commented 6 years ago

Any feedback on this? I am looking through the code base on this issue to determine why Spamscope wont collect the Spamassassin output but have not found the right file yet. Could you maybe point me in the right direction? Thanks!

fedelemantuano commented 6 years ago

Hi @bm1391, I will give you an answer. I have to test it.

fedelemantuano commented 6 years ago

Sorry @bm1391,

but I need raw mail to replicate the error. Your snippet it's not parsed:

{
  "body": "cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1228.23; Wed, 1 Oct\n2018 18:05:32 +0000\nReceived: from prod.protection.outlook.com\n(2a03:111:f400:7e32::216) by outlook.office365.com\n(2606:10b6:930:15::85) with Microsoft SMTP Server (version=TLS1_2,\ncipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 17.20.1250.20 via Frontend\nTransport; Wed, 1 Oct 2018 18:05:32 +0000\nAuthentication-Results: spf=pass (sender IP is 100.145.126.221)\nsmtp.mailfrom=example.com; example-to.com; dkim=none (message not signed)\nheader.d=none;example-to.com; dmarc=bestguesspass action=none\nheader.from=example.com;\nReceived-SPF: Pass (protection.outlook.com: domain of example.com\ndesignates 107.14.166.232 as permitted sender)\nreceiver=protection.outlook.com; client-ip=122.13.10.244;\nhelo=cdptpa-cmomta01.email.rr.com;\nReceived: from cdptpa-cmomta01.email.rr.com (122.13.166.212) by\nDF.protection.outlook.com (10.112.100.147) with Microsoft\nSMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id\n15.20.1250.2 via Frontend Transport; Wed, 17 Oct 2018 18:05:32 +0000\nReceived: from cdptpa-web09 ([174.143.174.228])\nby smtp with ESMTPA\nid CqCJgk0clvMX5; Wed, 17 Oct 2018 18:05:31 +0000\nMessage-ID: 20181345CD31.PKFYD.61244.root@example\nDate: Wed, 17 Oct 2018 14:05:31 -0400\nFrom: John Test test@example.com\nTo: toaddr@example.com\nSubject: RE: Office\nMIME-Version: 1.0\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nX-Priority: 3 (Normal)\nSensitivity: Normal\nX-Originating-IP: from 11.58.28.211 by webmail.example.com; Wed, 17 Oct 2018 18:05:31 +0000\nX-CMAE-Envelope: MS4wfEMbv1HIPsPZFd73LdHK4/W8+x7fwISDFSDFWERWERW4Ln7qrR08ueTt+Ck0TXp9GxRzVBc91UC4Q/kNYWDNQpNYav4SYtCdOLe0xi+TV1nxGGGYuVHJzBxH\nVaSXxONNFDFSzeC3SSoV43X9A2e2286OQf79263498234SDFSDFqv94Pb8an6clor1A==\nReturn-Path: To@example.com\nX-MS-Exchange-Organization-Network-Message-Id: 85425318-e0a3-44ea-88450-0028340345\nX-EOPAttributedMessage: 0\nX-EOPTenantAttributedMessage: 4a809fb2-0c7f-4201-95da-06989273479236f:0\nX-MS-Exchange-Organization-MessageDirectionality: Incoming\nX-Forefront-Antispam-Report: CIP:;IPV:NLI;CTRY:US;EFV:NLI;\nX-Microsoft-Exchange-Diagnostics: 1;;1:SK1Odwerwer3LkfpNAhzRQ345345tnRtinsk1+FYPmdUkrbAHMwu/HC9+CsQv5dOymEW79W6ZZOTtSBSmNx4DzyR6q3BHn6ZoYYvNGJ+cXpyQTb13eNZcyTaQPS\nX-MS-Exchange-Organization-AuthSource: protection.outlook.com\nX-MS-Exchange-Organization-AuthAs: Anonymous\nX-MS-PublicTrafficType: Email\nX-MS-Office365-Filtering-Correlation-Id: 85425318-e0a3-40ea-8810-08d6345b2132\nX-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(5600074)(711020)(4605076)(4608076)(1401270)(1414027)(71702078)(7193020);SRVR:B0850;\nX-Microsoft-Exchange-Diagnostics: 1;;3:fuClKPbrERQpYgevg46k8eY2aUmhBhtxaczB6g1QG+xEO/nzGU3gsaegjEWiJxhILvS8hl6v5cfn9U2gQUs0bdzRCJBdjnpt4Jm4Oh11Nvr2kLYAQCB46Bzoo05qj5ji1OFgqUZSTelY2/JPeDnIIuqS19ZK/v5JD93fs1ReTTBv3gLmBhVOv8pM05EdOU+fZ/QVCg+30peKtk6kVXnrV/88+zKdJ26POvBq8+QYdoiuTcNzAFYhzt/UUA1mAbPUBvr/rgygVP7srGNrWZwtnJXR0KzZ3uOon6/i6s3PkaSJ0RA8U7EYJptWW52sfHIVMDfaytuzddoUkRYGLQkQbkaeBxeZR3N+IQrId1h1AM8=;25:Go3pc+J/J6qmYXFq3+euv/L47BST/F25lM/MZX+hJow334B3Y9HROI3gAEHzIfFrpstpatEUx1MSRAFk8GBkS9gbVR0FRiywS4X7dKN7RJAZTy5E1SmuS3NYXusjLGqzG/b6oISrXfIcXKm0qFsSCdWH6sxSinjh4psn1foLwO+eWGs8AnQ7mpkQSwv85Ca3/+9OG82eM60JhQp+zebPYV20le2QLdorvW5a3vDp6aLCF7CC/DfLgYwaU4hYC90tp9xUWGL6Vsb+AajUKyWV/dEq5dqYEfZN2prkojTclUhLSMZeQX4uo3LW9gLWeTiKOpjNr/z9Qu3AzPNe4duOFA==\nX-MS-TrafficTypeDiagnostic: BN4P389273409:\nX-LD-Processed: 4a809fb2-0c7f-4201-95da-06953b7d506f,ExtAddr",
  "received": [
    {
      "delay": 0,
      "from": "prod.outlook.com",
      "hop": 1
    }
  ],
  "(2a01": "111:e400:59ba::16) with Microsoft SMTP Server (version=TLS1_2,",
  "has_defects": false,
  "(2603": "10b6:910:15::25) by prod.outlook.com"
}
bmarsh9 commented 6 years ago

I believe that is the raw mail that I attached.

fedelemantuano commented 6 years ago

Hi @bm1391,

to manage the issue correctly please use the new issue template that I uploaded. I'm closing this issue because I can't solve it without all data.