awalon / MailToTelegramForwarder

MailToTelegramForwarder is a Python based daemon that will fetch mails from a remote IMAP server and forward them via Telegram API.
GNU General Public License v3.0
40 stars 18 forks source link

Failed to send Telegram message "Can't parse entities: character '.' " #13

Closed phant closed 11 months ago

phant commented 2 years ago

Instead original message i receive "❌ Failed to send Telegram message (UID: 159) to '': Can't parse entities: character '.' is reserved and must be escaped with the preceding ''"

Mail that failed to forward ``` Return-Path: Delivered-To: b@b.c Received: from backup-srv (backup-srv [10.74.1.2]) by mail.b.c (Postfix) with ESMTP id 79F6380FAD for ; Sat, 15 Oct 2022 05:05:51 +0500 (+05) Authentication-Results: mail.b.c; dkim=none MIME-Version: 1.0 From: a@b.c To: b@b.c Date: 15 Oct 2022 05:05:51 +0500 Subject: =?utf-8?B?SW50ZWzCriBSYXBpZCBTdG9yYWdlIFRlY2hub2xvZ3k6IEluZm9ybWF0aW9uIG5vdGlmaWNhdGlvbiBvbiBCQUNLVVAtU1JW?= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 Vm9sdW1lIFI1OiBSZWJ1aWxkaW5nIGNvbXBsZXRlLg0KDQpTeXN0ZW0gUmVwb3J0DQoNClN5 c3RlbSBJbmZvcm1hdGlvbg0K0JjQvNGPINCe0KE6IE1pY3Jvc29mdCBXaW5kb3dzIDcg0J/R gNC+0YTQtdGB0YHQuNC+0L3QsNC70YzQvdCw0Y8gDQrQktC10YDRgdC40Y8g0J7QoTogNi4x Ljc2MDEgU2VydmljZSBQYWNrIDEgNzYwMQ0K0KHQuNGB0YLQtdC80L3QvtC1INC40LzRjzog QkFDS1VQLVNSVg0K0J/RgNC+0LjQt9Cy0L7QtNC40YLQtdC70Ywg0YHQuNGB0YLQtdC80Ys6 IFN5c3RlbSBtYW51ZmFjdHVyZXINCtCc0L7QtNC10LvRjCDRgdC40YHRgtC10LzRizogU3lz dGVtIFByb2R1Y3QgTmFtZQ0K0J/RgNC+0YbQtdGB0YHQvtGAOiBHZW51aW5lSW50ZWwgSW50 ZWw2NCBGYW1pbHkgNiBNb2RlbCA0MiBTdGVwcGluZyA3IDMsMSAg0JPQk9GGDQpCSU9TOiBB bWVyaWNhbiBNZWdhdHJlbmRzIEluYy4sIDE5MDUNCg0KSW50ZWzCriBSYXBpZCBTdG9yYWdl IFRlY2hub2xvZ3kNCktpdCBpbnN0YWxsZWQ6ICAxMS4xLjAuMTAwNg0KVXNlciBpbnRlcmZh Y2UgdmVyc2lvbjogIDExLjEuMC4xMDA2DQpMYW5ndWFnZTogIEVuZ2xpc2ggKFVuaXRlZCBT dGF0ZXMpDQpJbnRlbCBSQUlEIGNvbnRyb2xsZXI6ICBJbnRlbChSKSBEZXNrdG9wL1dvcmtz dGF0aW9uL1NlcnZlciBFeHByZXNzIENoaXBzZXQgU0FUQSBSQUlEIENvbnRyb2xsZXINCk51 bWJlciBvZiBTQVRBIHBvcnRzOiAgNg0KUkFJRCBvcHRpb24gUk9NIHZlcnNpb246ICAxMS4w LjAuMTMzOQ0KRHJpdmVyIHZlcnNpb246ICAxMS4xLjAuMTAwNg0KSVNESSB2ZXJzaW9uOiAg MTEuMS4wLjEwMDYNCg0KSGFyZHdhcmUgSW5mb3JtYXRpb24NCk5hbWU6ICBBcnJheV8wMDAw ICAgICAgIA0KU2l6ZTogIDI3LjMgVEINCkF2YWlsYWJsZSBzcGFjZTogIDAgR0INCkRpc2sg ZGF0YSBjYWNoZTogIEVuYWJsZWQNCk51bWJlciBvZiB2b2x1bWVzOiAgMQ0KVm9sdW1lIG1l bWJlcjogIFI1DQpOdW1iZXIgb2YgZGlza3M6ICA1DQpBcnJheSBkaXNrOiAgV0QtV1g2MUQ4 OFJFU0o4DQpBcnJheSBkaXNrOiAgV0QtV1g1MUQ4OERVTEZODQpBcnJheSBkaXNrOiAgV0Qt V1hKMUgyNkxNSzlNDQpBcnJheSBkaXNrOiAgV0QtV1g2MUQ4OEhMVjlKDQpBcnJheSBkaXNr OiAgV0QtV1g2MUQ4OEhMTEs2DQoNCk5hbWU6ICBSNQ0KU3RhdHVzOiAgTm9ybWFsDQpUeXBl OiAgUkFJRCA1DQpTaXplOiAgMjEuOCBUQg0KRGF0YSBzdHJpcGUgc2l6ZTogIDEyOCBLQg0K V3JpdGUtYmFjayBjYWNoZTogIEVuYWJsZWQNClN5c3RlbSB2b2x1bWU6ICBObw0KSW5pdGlh bGl6ZWQ6ICBZZXMgIA0KVmVyaWZpY2F0aW9uIGVycm9ycyBmb3VuZDogIDANCkJsb2NrcyB3 aXRoIG1lZGlhIGVycm9yczogIDANClBoeXNpY2FsIHNlY3RvciBzaXplOiAgNDA5NiBCeXRl cw0KTG9naWNhbCBzZWN0b3Igc2l6ZTogIDUxMiBCeXRlcw0KUGFyZW50IGFycmF5OiAgQXJy YXlfMDAwMCAgICAgICANCk51bWJlciBvZiBkaXNrczogIDUNCkFycmF5IGRpc2s6ICBXRC1X WDYxRDg4UkVTSjgNCkFycmF5IGRpc2s6ICBXRC1XWDUxRDg4RFVMRk4NCkFycmF5IGRpc2s6 ICBXRC1XWEoxSDI2TE1LOU0NCkFycmF5IGRpc2s6ICBXRC1XWDYxRDg4SExWOUoNCkFycmF5 IGRpc2s6ICBXRC1XWDYxRDg4SExMSzYNCg0KRGlzayBvbiBwb3J0IDANClBvcnQgbG9jYXRp b246ICBJbnRlcm5hbA0KU3RhdHVzOiAgTm9ybWFsDQpUeXBlOiAgSGFyZCBkaXNrDQpVc2Fn ZTogIEFycmF5IGRpc2sNClNpemU6ICA1LjUgVEINClNlcmlhbCBudW1iZXI6ICBXRC1XWDYx RDg4UkVTSjgNCk1vZGVsOiAgV0RDIFdENjBFRlJYLTY4TDBCTjENCkZpcm13YXJlOiAgODIu MDBBODINClN5c3RlbSBkaXNrOiAgTm8NClBhc3N3b3JkIHByb3RlY3RlZDogIE5vDQpEaXNr IGRhdGEgY2FjaGU6ICBFbmFibGVkDQpOYXRpdmUgY29tbWFuZCBxdWV1aW5nOiAgWWVzICAN ClNBVEEgdHJhbnNmZXIgcmF0ZTogIDYgR2Ivcw0KUGh5c2ljYWwgc2VjdG9yIHNpemU6ICA0 MDk2IEJ5dGVzDQpMb2dpY2FsIHNlY3RvciBzaXplOiAgNTEyIEJ5dGVzDQoNCkRpc2sgb24g cG9ydCAxDQpQb3J0IGxvY2F0aW9uOiAgSW50ZXJuYWwNClN0YXR1czogIE5vcm1hbA0KVHlw ZTogIEhhcmQgZGlzaw0KVXNhZ2U6ICBBcnJheSBkaXNrDQpTaXplOiAgNS41IFRCDQpTZXJp YWwgbnVtYmVyOiAgV0QtV1g1MUQ4OERVTEZODQpNb2RlbDogIFdEQyBXRDYwRUZSWC02OEww Qk4xDQpGaXJtd2FyZTogIDgyLjAwQTgyDQpTeXN0ZW0gZGlzazogIE5vDQpQYXNzd29yZCBw cm90ZWN0ZWQ6ICBObw0KRGlzayBkYXRhIGNhY2hlOiAgRW5hYmxlZA0KTmF0aXZlIGNvbW1h bmQgcXVldWluZzogIFllcyAgDQpTQVRBIHRyYW5zZmVyIHJhdGU6ICA2IEdiL3MNClBoeXNp Y2FsIHNlY3RvciBzaXplOiAgNDA5NiBCeXRlcw0KTG9naWNhbCBzZWN0b3Igc2l6ZTogIDUx MiBCeXRlcw0KDQpEaXNrIG9uIHBvcnQgMg0KUG9ydCBsb2NhdGlvbjogIEludGVybmFsDQpT dGF0dXM6ICBOb3JtYWwNClR5cGU6ICBIYXJkIGRpc2sNClVzYWdlOiAgQXZhaWxhYmxlDQpT aXplOiAgMjMzIEdCDQpTZXJpYWwgbnVtYmVyOiAgWjJBTVZDTjENCk1vZGVsOiAgU1QyNTBE TTAwMC0xQkQxNDENCkZpcm13YXJlOiAgS0M0NQ0KU3lzdGVtIGRpc2s6ICBZZXMgIA0KUGFz c3dvcmQgcHJvdGVjdGVkOiAgTm8NCkRpc2sgZGF0YSBjYWNoZTogIEVuYWJsZWQNCk5hdGl2 ZSBjb21tYW5kIHF1ZXVpbmc6ICBZZXMgIA0KU0FUQSB0cmFuc2ZlciByYXRlOiAgMyBHYi9z DQpQaHlzaWNhbCBzZWN0b3Igc2l6ZTogIDQwOTYgQnl0ZXMNCkxvZ2ljYWwgc2VjdG9yIHNp emU6ICA1MTIgQnl0ZXMNCg0KRGlzayBvbiBwb3J0IDMNClBvcnQgbG9jYXRpb246ICBJbnRl cm5hbA0KU3RhdHVzOiAgTm9ybWFsDQpUeXBlOiAgSGFyZCBkaXNrDQpVc2FnZTogIEFycmF5 IGRpc2sNClNpemU6ICA1LjUgVEINClNlcmlhbCBudW1iZXI6ICBXRC1XWEoxSDI2TE1LOU0N Ck1vZGVsOiAgV0RDIFdENjBFRlJYLTY4TDBCTjENCkZpcm13YXJlOiAgODIuMDBBODINClN5 c3RlbSBkaXNrOiAgTm8NClBhc3N3b3JkIHByb3RlY3RlZDogIE5vDQpEaXNrIGRhdGEgY2Fj aGU6ICBFbmFibGVkDQpOYXRpdmUgY29tbWFuZCBxdWV1aW5nOiAgWWVzICANClNBVEEgdHJh bnNmZXIgcmF0ZTogIDMgR2Ivcw0KUGh5c2ljYWwgc2VjdG9yIHNpemU6ICA0MDk2IEJ5dGVz DQpMb2dpY2FsIHNlY3RvciBzaXplOiAgNTEyIEJ5dGVzDQoNCkRpc2sgb24gcG9ydCA0DQpQ b3J0IGxvY2F0aW9uOiAgSW50ZXJuYWwNClN0YXR1czogIE5vcm1hbA0KVHlwZTogIEhhcmQg ZGlzaw0KVXNhZ2U6ICBBcnJheSBkaXNrDQpTaXplOiAgNS41IFRCDQpTZXJpYWwgbnVtYmVy OiAgV0QtV1g2MUQ4OEhMVjlKDQpNb2RlbDogIFdEQyBXRDYwRUZSWC02OEwwQk4xDQpGaXJt d2FyZTogIDgyLjAwQTgyDQpTeXN0ZW0gZGlzazogIE5vDQpQYXNzd29yZCBwcm90ZWN0ZWQ6 ICBObw0KRGlzayBkYXRhIGNhY2hlOiAgRW5hYmxlZA0KTmF0aXZlIGNvbW1hbmQgcXVldWlu ZzogIFllcyAgDQpTQVRBIHRyYW5zZmVyIHJhdGU6ICAzIEdiL3MNClBoeXNpY2FsIHNlY3Rv ciBzaXplOiAgNDA5NiBCeXRlcw0KTG9naWNhbCBzZWN0b3Igc2l6ZTogIDUxMiBCeXRlcw0K DQpEaXNrIG9uIHBvcnQgNQ0KUG9ydCBsb2NhdGlvbjogIEludGVybmFsDQpTdGF0dXM6ICBO b3JtYWwNClR5cGU6ICBIYXJkIGRpc2sNClVzYWdlOiAgQXJyYXkgZGlzaw0KU2l6ZTogIDUu NSBUQg0KU2VyaWFsIG51bWJlcjogIFdELVdYNjFEODhITExLNg0KTW9kZWw6ICBXREMgV0Q2 MEVGUlgtNjhMMEJOMQ0KRmlybXdhcmU6ICA4Mi4wMEE4Mg0KU3lzdGVtIGRpc2s6ICBObw0K UGFzc3dvcmQgcHJvdGVjdGVkOiAgTm8NCkRpc2sgZGF0YSBjYWNoZTogIEVuYWJsZWQNCk5h dGl2ZSBjb21tYW5kIHF1ZXVpbmc6ICBZZXMgIA0KU0FUQSB0cmFuc2ZlciByYXRlOiAgMyBH Yi9zDQpQaHlzaWNhbCBzZWN0b3Igc2l6ZTogIDQwOTYgQnl0ZXMNCkxvZ2ljYWwgc2VjdG9y IHNpemU6ICA1MTIgQnl0ZXMNCg0KDQo= ```
jbdu12 commented 1 year ago

Same problem

awalon commented 1 year ago

Hello, thank you very much for reporting with test data as well. A the moment I cannot reproduce this problem with my recently updated system. Is this bug still occuring on more recnt Python (3.11.2) and/or telegram-bot (13.3-1) module? Greets

phant commented 1 year ago

was Python (3.9.2) and telegram-bot (13.3-1) on debian 11 now Python (3.11.2) and telegram-bot (13.15) on debian 12

it is possible to send mail with specific UID: 159 to bot again?

awalon commented 1 year ago

Specific message can re forwarded by: 1) Stop service 2) Change search config to: "search: (UID 159)" 3) Mark mail as unread, using your mail client 4) Start service again

Reset can be done by setting search back to your custom value or comment it out for default value ("search: (UID ${lastUID}:* UNSEEN)"). Restart service and your back to normal operation.

phant commented 1 year ago

same

Jul 16 21:25:50 mail2tg-fw systemd[1]: Started mail-to-telegram-forwarder@mailToTelegramForwarder.service - Mail to Telegram Forwarder - Service for: 'mailToTelegramForwarder'.
Jul 16 21:25:50 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: Mailboxes:
Jul 16 21:25:50 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: b'(\\HasNoChildren) "." "INBOX"'
Jul 16 21:25:50 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: Processing mailbox...
Jul 16 21:25:50 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: Most recent UID: '355'
Jul 16 21:25:50 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: Reading mails having UID more recent than '355', using search: '(UID 159)'
Jul 16 21:25:50 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: Parsed mail with UID '159': 'Intel<C2><AE> Rapid Storage Technology: Information notification on BACKUP\-SRV'
Jul 16 21:25:50 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: Got 1 new mail(s) to forward, using most recent UID: '159'
Jul 16 21:25:51 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: <E2><9D><8C> Failed to send Telegram message (UID: 159) to 'xxxxxxx': Can't parse entities: character '.' is reserved and must be escaped with the preceding '\' [/usr/local/bin/mailToTelegramForwarder:504 in 'send_message']
Jul 16 21:26:01 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: Parsed mail with UID '159': 'Intel<C2><AE> Rapid Storage Technology: Information notification on BACKUP\-SRV'
Jul 16 21:26:01 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: Got 1 new mail(s) to forward, using most recent UID: '159'
Jul 16 21:26:02 mail2tg-fw mailToTelegramForwarder[7112]:  Mail to Telegram Forwarder: <E2><9D><8C> Failed to send Telegram message (UID: 159) to 'xxxxxxx': Can't parse entities: character '.' is reserved and must be escaped with the preceding '\' [/usr/local/bin/mailToTelegramForwarder:504 in 'send_message']
Jul 16 21:26:09 mail2tg-fw systemd[1]: Stopping mail-to-telegram-forwarder@mailToTelegramForwarder.service - Mail to Telegram Forwarder - Service for: 'mailToTelegramForwarder'...
phant commented 11 months ago

❌ Failed to send Telegram message (UID: 545) to 'xxxx': Can't parse entities: character '.' is reserved and must be escaped with the preceding ''

Mail to Telegram Forwarder: <E2><9D><8C> Failed to send Telegram message (UID: 545) to 'xxxx': Can't parse entities: character '.' is reserved and must be escaped with the preceding '\' [/usr/local/bin/mailToTelegramForwarder:504 in 'send_message']

mail failed to forward

``` Return-Path: Delivered-To: tg-forward@example.co Received: from pve8.corp.loc (unknown [172.16.16.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.example.co (Postfix) with ESMTPSA id 9EAE7803A4 for ; Wed, 29 Nov 2023 11:35:08 +0500 (+05) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.co; s=mail; t=1701239708; bh=KwGJEnt/Wkij98ZYj1uj5zMVvoWzzd9LKiFNbTJ15BU=; h=Date:To:From:Subject; b=Obqr9PIWdnAaCdFop3IQp5i6tlQMnH/WNueI7x2NqLNVROtTGJ8yVl/lq16QavuMs 2KBeE8V4aw5HLAriiv0JAerznqNEwDw0gv8Bly/0hMhC3Jweb7RcA3elsXhMoZyCd6 QBn4Iv0p8nmIfvL4/V0QumaQ0o8FHpXeA94ER374= Date: Wed, 29 Nov 2023 11:35:08 +0500 To: alarm@example.co From: proxmox-zed@example.co Subject: Raid status Good on pve8 Message-Id: <20231129113508.482253@pve8.corp.loc> X-Mailer: swaks v20201014.0 jetmore.org/john/code/swaks/ RAID OK - Arrays: OK:2 Bad:0 - Disks: OK:12 Bad:0 -- Controller information -- -- ID | H/W Model | RAM | Temp | BBU | Firmware c0 | Intel(R) RAID Controller RS3DC040 | 1024MB | 37C | Good | FW: 24.21.0-0132 c1 | Intel(R) RAID Controller RS3DC080 | 1024MB | 34C | Good | FW: 24.21.0-0132 -- Array information -- -- ID | Type | Size | Strpsz | Flags | DskCache | Status | OS Path | CacheCade |InProgress c0u0 | RAID-6 | 3490G | 256 KB | RA,WB | Default | Optimal | /dev/sda | None |None c1u0 | RAID-6 | 10471G | 256 KB | RA,WB | Disabled | Optimal | /dev/sdb | None |None -- Disk information -- -- ID | Type | Drive Model | Size | Status | Speed | Temp | Slot ID | LSI ID c0u0p0 | SSD | 50026B72831158F0 KINGSTON SEDC600M1920G SCEKH5.1 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 18C | [252:0] | 4 c0u0p1 | SSD | 50026B7282E25347 KINGSTON SEDC500M1920G SCEKJ2.8 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 19C | [252:1] | 5 c0u0p2 | SSD | 50026B7283115891 KINGSTON SEDC600M1920G SCEKH5.1 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 19C | [252:2] | 6 c0u0p3 | SSD | 50026B728311590B KINGSTON SEDC600M1920G SCEKH5.1 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 18C | [252:3] | 7 c1u0p0 | SSD | 50026B7282C8B152 KINGSTON SEDC500M1920G SCEKJ2.8 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 18C | [252:0] | 8 c1u0p1 | SSD | 50026B7282CF3923 KINGSTON SEDC500M1920G SCEKJ2.8 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 18C | [252:1] | 9 c1u0p2 | SSD | 50026B72831158C9 KINGSTON SEDC600M1920G SCEKH5.1 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 19C | [252:2] | 10 c1u0p3 | SSD | 50026B72831158C3 KINGSTON SEDC600M1920G SCEKH5.1 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 18C | [252:3] | 11 c1u0p4 | SSD | 50026B7282E25321 KINGSTON SEDC500M1920G SCEKJ2.8 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 18C | [252:4] | 12 c1u0p5 | SSD | 50026B7282CF35F8 KINGSTON SEDC500M1920G SCEKJ2.8 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 19C | [252:5] | 13 c1u0p6 | SSD | 50026B7283115991 KINGSTON SEDC600M1920G SCEKH5.1 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 18C | [252:6] | 14 c1u0p7 | SSD | 50026B7283115925 KINGSTON SEDC600M1920G SCEKH5.1 | 1.745 TB | Online, Spun Up | 6.0Gb/s | 18C | [252:7] | 15 ```

2kva commented 11 months ago

I solved this problem with this fix

#diff -u mailToTelegramForwarder.py.orig mailToTelegramForwarder.py
--- mailToTelegramForwarder.py.orig   2023-10-12 03:25:44.000000000 +0300
+++ mailToTelegramForwarder.py    2023-11-10 11:27:19.285306190 +0300
@@ -818,7 +818,8 @@
                         else:
                             # remove last "\"
                             content = re.sub(r'\\*$', '', content)
-                        content += "... (first " + str(max_len) + " characters)"
+                        #content += "... (first " + str(max_len) + " characters)"
+                        content += "\\.\\.\\. \\(first " + str(max_len) + " characters\\)"

             # attachment summary
             attachments_summary = ""
phant commented 11 months ago

@2kva Thank you. It works for me too. @awalon please use this suggestion in new release.