search

moralmunky / Home-Assistant-Mail-And-Packages

Home Assistant integration providing day of package counts and USPS informed delivery images.
MIT License
644 stars 82 forks source link

ISSUE: Amazon.de sensor.mail_amazon_packages sensor never turns on #751

Open CHirch opened 2 years ago

CHirch commented 2 years ago

Describe the bug The sensor.mail_amazonpackages is always zero. A view in the logfile shows that the programm recognizes the delivery today mail. I can find the raw mail in my logfile. Next thing in logfile is this entry: (SyncWorker_6) [custom_components.mail_andpackages.helpers] Looking for: will arrive: This repeats with similar terms. Unfortunately none of the terms is included in the mail text. The mail text doesn't include any information about the delivery date. Only the subject contains the information "Zustellung: heute" ("Delivery: Today")

Next in logfile is: _(SyncWorker_6) [custom_components.mail_andpackages.helpers] Amazon email search address: ['shipment-tracking@amazon.it', 'conferma-spedizione@amazon.it']

Environment (please complete the following information):

Logs 2022-11-15 03:02:07.396 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Amazon email search address: ['shipment-tracking@amazon.de', 'conferma-spedizione@amazon.de'] 2022-11-15 03:02:07.396 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] DEBUG subject: None 2022-11-15 03:02:07.396 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] DEBUG imap_search: (OR FROM "shipment-tracking@amazon.de" FROM "conferma-spedizione@amazon.de" SINCE 09-Nov-2022) 2022-11-15 03:02:07.431 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] DEBUG email_search value: ('OK', [b'93']) 2022-11-15 03:02:07.432 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Amazon emails found: 1 2022-11-15 03:02:07.498 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Email Multipart: True 2022-11-15 03:02:07.498 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Content Type: multipart/alternative 2022-11-15 03:02:07.498 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Amazon Subject: Zustellung heute: Artikel 2022-11-15 03:02:07.498 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] RAW EMAIL: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable

Amazon.de

Hallo,

Ihr Paket mit Artikel wird zugestellt.

Um die Sicherheit unserer Kunden und Zustellpartner zu gewährleisten, stellen wir jetzt Pakete vor Ihrer Haustür ab und nehmen einen sicheren Abstand ein, bevor Sie die Sendung entgegennehmen.

Wenn Sie uns einen sicheren Ablageort mitteilen möchten, aktualisieren Sie bitte die Lieferanweisungen auf unserer Seite Ihr Paket verfolgen: https://www.amazon.de/gp/css/shiptrack/view.html? ...

Ihr Paket verfolgen: https://www.amazon.de/gp/css/shiptrack/view.html? ...

Bestellung #303-...

_Artikel_

Dies ist eine automatisch versendete E-Mail. Bitte antworten Sie nicht auf diese Nachricht, da die E-Mail-Adresse nicht zum Empfang von E-Mails eingerichtet ist. 2022-11-15 03:02:07.498 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Looking for: will arrive: 2022-11-15 03:02:07.498 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Looking for: estimated delivery date is: 2022-11-15 03:02:07.498 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Looking for: guaranteed delivery date is: 2022-11-15 03:02:07.498 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Looking for: Arriving: 2022-11-15 03:02:07.498 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Looking for: Arriverà: 2022-11-15 03:02:07.499 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Looking for: arriving: 2022-11-15 03:02:07.499 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Looking for: Dostawa: 2022-11-15 03:02:07.499 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Looking for: Zustellung: 2022-11-15 03:02:07.499 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Amazon email search address: ['shipment-tracking@amazon.it', 'conferma-spedizione@amazon.it'] 2022-11-15 03:02:07.499 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] DEBUG subject: None 2022-11-15 03:02:07.499 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] DEBUG imap_search: (OR FROM "shipment-tracking@amazon.it" FROM "conferma-spedizione@amazon.it" SINCE 09-Nov-2022) 2022-11-15 03:02:07.535 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] DEBUG email_search value: ('OK', [b'']) 2022-11-15 03:02:07.535 DEBUG (SyncWorker_6) [custom_components.mail_and_packages.helpers] Amazon emails found: 0

Thank you for this very handy integration, great work.

firstof9 commented 2 years ago

Try the beta version.

CHirch commented 2 years ago

I've updated to 0.3.5-b19 and will let you know what will happen at my next delivery day. Thank you so far for the quick response.

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

CHirch commented 2 years ago

Try the beta version.

Unfortunately still doesn't work. :(

Only these lines have been added to the logs:

Looking for: Entrega:
Looking for: A chegar:
firstof9 commented 2 years ago

Usually Amazon sends an email with the estimated date they'll be delivering your package. Do they not do that for the .de version?

CHirch commented 2 years ago

You get two mails from Amazon: 1) shipment confirmation. From: versandbestaetigung@amazon.de, Subject: ..."wurde versandt!" Message content:

Guten Tag,

wir m=C3=B6chten dir hiermit mitteilen, dass wir deine Bestellung versandt =
haben.
Deine Sendung befindet sich nun auf dem Versandweg; eine =C3=84nderung durc=
h dich oder unseren Kundenservice ist nicht mehr m=C3=B6glich.

Zustellung:
            Montag, 28 November

Verfolge deine(n) Artikel:https://smile.amazon.de/gp/css/shiptrack/view.htm=
l/ref=...

2) Note on the day of the delivery From: shipment-tracking@amazon.de, Subject: "Zustellung heute:"... Message content:

Amazon.de
---------------------------------------------------------------------------=
-----------------------------------------------------------------

Hallo,

Ihr Paket mit ....... und 1 weiteren Artikel wird zu=
gestellt.=20

Um die Sicherheit unserer Kunden und Zustellpartner zu gew=C3=A4hrleisten, =
stellen wir jetzt Pakete vor Ihrer Haust=C3=BCr ab und nehmen einen sichere=
n Abstand ein, bevor Sie die Sendung entgegennehmen.

Wenn Sie uns einen sicheren Ablageort mitteilen m=C3=B6chten, aktualisieren=
 Sie bitte die Lieferanweisungen auf unserer Seite Ihr Paket verfolgen:=20
            https://www.amazon.de/gp/css/shiptrack/view.html?ie=3DUTF8&orde=
rID=.........

Ihr Paket verfolgen:
https://www.amazon.de/gp/css/shiptrack/view.html?ie=.......

Bestellung #302-.......

I reckon the second mail is detected by the programme but the first mail contains the information the programme is looking for.

firstof9 commented 2 years ago

I'll see what I can get sorted.

firstof9 commented 2 years ago

Please test 0.3.5-b20 this should catch your emails now.

CHirch commented 2 years ago

Unfortunately no success yet.... Log:

[custom_components.mail_and_packages.helpers] First pass: Freitag, 2 Dezember
2022-12-02 01:48:30.547 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 2 Dezember
2022-12-02 01:48:30.547 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Invalid date format found for language it_IT. (time data 'Freitag, 2 Dezember' does not match format '%A, %B %d')
2022-12-02 01:48:30.548 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 2 Dezember
2022-12-02 01:48:30.548 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Invalid date format found for language it_IT.UTF-8. (time data 'Freitag, 2 Dezember' does not match format '%A, %B %d')
2022-12-02 01:48:30.548 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 2 Dezember
2022-12-02 01:48:30.548 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Invalid date format found for language pl_PL. (time data 'Freitag, 2 Dezember' does not match format '%A, %B %d')
2022-12-02 01:48:30.548 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 2 Dezember
2022-12-02 01:48:30.549 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Invalid date format found for language pl_PL.UTF-8. (time data 'Freitag, 2 Dezember' does not match format '%A, %B %d')
2022-12-02 01:48:30.549 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 2 Dezember
2022-12-02 01:48:30.549 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Invalid date format found for language de_DE. (time data 'Freitag, 2 Dezember' does not match format '%A, %B %d')
2022-12-02 01:48:30.549 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 2 Dezember
2022-12-02 01:48:30.550 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Invalid date format found for language de_DE.UTF-8. (time data 'Freitag, 2 Dezember' does not match format '%A, %B %d')
2022-12-02 01:48:30.550 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 2 Dezember
2022-12-02 01:48:30.550 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Invalid date format found for language es_ES. (time data 'Freitag, 2 Dezember' does not match format '%A, %B %d')
2022-12-02 01:48:30.550 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 2 Dezember
2022-12-02 01:48:30.551 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Invalid date format found for language es_ES.UTF-8. (time data 'Freitag, 2 Dezember' does not match format '%A, %B %d')
2022-12-02 01:48:30.551 DEBUG (SyncWorker_3) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 2 Dezember
firstof9 commented 2 years ago

Well it found the date (good), it for some reason wouldn't translate it for timestamp processing (bad). This is going to be similar to the other languages, I'll keep researching.

CHirch commented 2 years ago

The date format in the Amazon mail is not a usual German date format. Usually we have a dot after the day like in every other language ;) Maybe just change '%A, %B %d' to %A, %d %B.

What might be more problematic is that there are sometimes date ranges in the mail like

<td> <p> <span class=3D"uniqueSpan"> Zustellung: </span> <br> =
<b style=3D"color:#009900;"> Samstag, 3 Dezember -<br> Montag, 5 Dezember <=
/b> </p>=20

I think it would be more accurate to detect the notification on the day of delivery ("Zustellung heute")

firstof9 commented 2 years ago

I think it would be more accurate to detect the notification on the day of delivery

That'd be nice, but doesn't happen like that in all places.

Pr0mises commented 1 year ago

Having the same problem with DE amazon. I tried a couple of things and inlcuded this line into helper.py:

elif "de_DE" in lang:
    new_arrive_date = arrive_date
    time_format = "%A, %d %B"

Running it local on windows and linux (locale de_DE.UTF8 is installed) does work. Using HASS docker same result appears:

Invalid date format found for language de_DE (time data 'Samstag, 8 April' does not match format '%A, %d %B')
Invalid date format found for language DE.UTF-8 (time data 'Samstag, 8 April' does not match format '%A, %d %B')

Which could mean de_DE(.UTF8) is not supported inside the docker file.

PS: Infact using this code results that there is no de_DE(.UTF8) installed?:

for lang in locale.locale_alias.values():
    if "de" in lang:
        print(lang)
    if "utf8" in lang:
        print(lang)

there is only de_DE.ISO8859-1 , why don't I've this error message then? Locale error: ... Using de_DE.ISO8859-1 local inside the docker doesn't work either...

>>> import locale
>>> import datetime
>>> locale.setlocale(locale.LC_TIME, 'de_DE.ISO8859-1')
'de_DE.ISO8859-1'
>>> datetime.datetime.today().strftime('%x')
'04/14/23'
>>> datetime.datetime.today().strftime('%c')
'Fri Apr 14 03:11:39 2023'

I'm confused

firstof9 commented 1 year ago

You'd need to install the language pack from inside the docker image.

Pr0mises commented 1 year ago

You'd need to install the language pack from inside the docker image.

Well I couldn't figure out how to do that though

firstof9 commented 1 year ago

Looks like a bit "to do" but here's a link: https://grrr.tech/posts/2020/add-locales-to-alpine-linux-docker-image/

Pr0mises commented 1 year ago

Thanks will check it out. Do you know why I don't the error for no locale language xyz found? Shouldn't the expection hit if theres no such language package installed?

firstof9 commented 1 year ago

That's what the

Invalid date format found for language de_DE (time data 'Samstag, 8 April' does not match format '%A, %d %B')
Invalid date format found for language DE.UTF-8 (time data 'Samstag, 8 April' does not match format '%A, %d %B')

messages are about.

Pr0mises commented 1 year ago

I see, I thought it should hit this expection, atleast I hit it on windows

try: 
       locale.setlocale(locale.LC_TIME, lang) 
except Exception as err: 
       _LOGGER.info("Locale error: %s (%s)", err, lang) 
      continue
firstof9 commented 1 year ago

You'd only see that if you had info level or higher/lower (I forget which direction is which on debug levels) debugging enabled.

Pr0mises commented 1 year ago

As my hass build takes some more time. I did some more testing with the result, locale is just random and the expection doesn't check if the system supports this locale it only checks if the locale exists in the locale enum...

According to the ISO8859-1 supported characters, every character used in the date should be supported on every system and every code I post below should work but the result is absolutely random on multiple systems.

The playground wasn't able to use any of it, windows worked very well and my debian worked only with predefined de_DE.UTF8 which doesn't work on the hass docker...

I've no clue what's going on and if it's just a problem of the different versions.

Some information: Windows:

Python 3.10.6

Linux Raspbian:

locale -a C | C.UTF-8 | de_DE.utf8 | en_GB.utf8 Python 3.9.2

Playground:

Python 3.8.5

PS: using playground

Python 3.10.2

none of the codes worked.

PPS: Using the hass default docker none of the codes worked as well (de_LI.UTF8 is supported..)

Python 3.10.10

Using this code:

import locale
import datetime

AMAZON_LANGS = [
    "de_DE",
    "de_DE.UTF-8",
    "de_DE.ISO8859-1",
    "de_AT.ISO8859-1",
    "de_LI.UTF-8",
    "de_LI",
    "de_LI.UTF8",
    "German"
]
arrive_date = "Samstag, 20 April"
time_format = None
new_arrive_date = None

def printf(format, *values):
    print(format % values )

for lang in locale.locale_alias.values():
    if "de_" in lang:
        print(lang)

for lang in AMAZON_LANGS:
    try:
        locale.setlocale(locale.LC_TIME, lang)
    except Exception as err:
        printf("Locale error: %s (%s)", err, lang)
        continue

    if "de_" in lang:
        new_arrive_date = arrive_date
        time_format = "%A, %d %B"
    elif "German" in lang:
        new_arrive_date = arrive_date
        time_format = "%A, %d %B"
    else:
        new_arrive_date = arrive_date
        time_format = "%A, %B %d"

    try:
        dateobj = datetime.datetime.strptime(
            new_arrive_date, time_format
        )
    except ValueError as err:
        printf(
            "ValueError: %s (%s)", lang, err
        )
        continue
    else:
        print("Date: ", dateobj)

Playground:

de_DE.ISO8859-1 de_AT.ISO8859-1 de_BE.ISO8859-1 de_CH.ISO8859-1 de_DE.ISO8859-1 de_IT.ISO8859-1 de_LI.UTF-8 de_LU.ISO8859-1 de_DE.ISO8859-1 de_DE.ISO8859-1 de_DE.ISO8859-1 de_CH.ISO8859-1 de_DE.ISO8859-1 ValueError: de_DE (time data 'Samstag, 20 April' does not match format '%A, %d %B') ValueError: de_DE.UTF-8 (time data 'Samstag, 20 April' does not match format '%A, %d %B') ValueError: de_DE.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') ValueError: de_AT.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') ValueError: de_LI.UTF-8 (time data 'Samstag, 20 April' does not match format '%A, %d %B') ValueError: de_LI (unknown locale: de_LI) ValueError: de_LI.UTF8 (time data 'Samstag, 20 April' does not match format '%A, %d %B') ValueError: German (time data 'Samstag, 20 April' does not match format '%A, %d %B')

Windows:

de_DE.ISO8859-1 de_AT.ISO8859-1 de_BE.ISO8859-1 de_CH.ISO8859-1 de_DE.ISO8859-1 de_IT.ISO8859-1 de_LI.UTF-8 de_LU.ISO8859-1 de_DE.ISO8859-1 de_DE.ISO8859-1 de_DE.ISO8859-1 de_CH.ISO8859-1 de_DE.ISO8859-1 Date: 1900-04-20 00:00:00 Date: 1900-04-20 00:00:00 Locale error: unsupported locale setting (de_DE.ISO8859-1) Locale error: unsupported locale setting (de_AT.ISO8859-1) Date: 1900-04-20 00:00:00 ValueError: de_LI (unknown locale: de_LI) Date: 1900-04-20 00:00:00 Date: 1900-04-20 00:00:00

Linux:

de_DE.ISO8859-1 de_AT.ISO8859-1 de_BE.ISO8859-1 de_CH.ISO8859-1 de_DE.ISO8859-1 de_IT.ISO8859-1 de_LI.UTF-8 de_LU.ISO8859-1 de_DE.ISO8859-1 de_DE.ISO8859-1 de_DE.ISO8859-1 de_CH.ISO8859-1 de_DE.ISO8859-1 Locale error: unsupported locale setting (de_DE) Date: 1900-04-20 00:00:00 Locale error: unsupported locale setting (de_DE.ISO8859-1) Locale error: unsupported locale setting (de_AT.ISO8859-1) Locale error: unsupported locale setting (de_LI.UTF-8) Locale error: unsupported locale setting (de_LI) Locale error: unsupported locale setting (de_LI.UTF8) Locale error: unsupported locale setting (German)

Using this code:

import locale
import datetime

arrive_date = "Samstag, 20 April"
time_format = None
new_arrive_date = None

def printf(format, *values):
    print(format % values )

oldlocal = locale.getlocale()
available_locales = []
for l in locale.locale_alias:
    if "de_" in l:
        try:
            locale.setlocale(locale.LC_TIME, l)
            available_locales.append(l)
        except:
            pass
for i in available_locales:
    print(i)

for lang in available_locales:
    try:
        locale.setlocale(locale.LC_TIME, lang)
    except Exception as err:
        printf("Locale error: %s (%s)", err, lang)
        continue

    printf("locale: %s", locale.getlocale(locale.LC_TIME))

    if "de_" in lang:
        new_arrive_date = arrive_date
        time_format = "%A, %d %B"
    elif "German" in lang:
        new_arrive_date = arrive_date
        time_format = "%A, %d %B"
    else:
        new_arrive_date = arrive_date
        time_format = "%A, %B %d"

    try:
        dateobj = datetime.datetime.strptime(
            new_arrive_date, time_format
        )
    except ValueError as err:
        printf(
            "ValueError: %s (%s)", lang, err
        )
        continue
    else:
        print("Date: ", dateobj)
else:
    printf("Unsupported locale")

Playground:

de_at de_be de_ch de_de de_it de_li.utf8 de_lu locale: ('de_AT', 'ISO8859-1') ValueError: de_at (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_BE', 'ISO8859-1') ValueError: de_be (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_CH', 'ISO8859-1') ValueError: de_ch (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_DE', 'ISO8859-1') ValueError: de_de (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_IT', 'ISO8859-1') ValueError: de_it (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_LI', 'UTF-8') ValueError: de_li.utf8 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_LU', 'ISO8859-1') ValueError: de_lu (time data 'Samstag, 20 April' does not match format '%A, %d %B') Unsupported locale

Windows:

de_at de_be de_ch de_de de_it de_li.utf8 de_lu locale: ('de_AT', 'ISO8859-1') Date: 1900-04-20 00:00:00 locale: ('de_BE', 'ISO8859-1') Date: 1900-04-20 00:00:00 locale: ('de_CH', 'ISO8859-1') Date: 1900-04-20 00:00:00 locale: ('de_DE', 'ISO8859-1') Date: 1900-04-20 00:00:00 locale: ('de_IT', 'ISO8859-1') Date: 1900-04-20 00:00:00 locale: ('de_LI', 'UTF-8') Date: 1900-04-20 00:00:00 locale: ('de_LU', 'ISO8859-1') Date: 1900-04-20 00:00:00 Unsupported locale

Raspbian:

Unsupported locale

Using this code:

import locale
import datetime

arrive_date = "Samstag, 20 April"
time_format = None
new_arrive_date = None

def printf(format, *values):
    print(format % values )

oldlocal = locale.getlocale()
available_locales = []
for l in locale.locale_alias.items():
    if "de_" in l[1]:
        try:
            locale.setlocale(locale.LC_TIME, l[1])
            available_locales.append(l[1])
        except:
            pass
for i in available_locales:
    print(i)

for lang in available_locales:
    try:
        locale.setlocale(locale.LC_TIME, lang)
    except Exception as err:
        printf("Locale error: %s (%s)", err, lang)
        continue

    printf("locale: %s", locale.getlocale(locale.LC_TIME))

    if "de_" in lang:
        new_arrive_date = arrive_date
        time_format = "%A, %d %B"

    try:
        dateobj = datetime.datetime.strptime(
            new_arrive_date, time_format
        )
    except ValueError as err:
        printf(
            "ValueError: %s (%s)", lang, err
        )
        continue
    else:
        print("Date: ", dateobj)
else:
    printf("Unsupported locale")

Playground:

de_DE.ISO8859-1 de_AT.ISO8859-1 de_BE.ISO8859-1 de_CH.ISO8859-1 de_DE.ISO8859-1 de_IT.ISO8859-1 de_LI.UTF-8 de_LU.ISO8859-1 de_DE.ISO8859-1 de_DE.ISO8859-1 de_DE.ISO8859-1 de_CH.ISO8859-1 de_DE.ISO8859-1 locale: ('de_DE', 'ISO8859-1') ValueError: de_DE.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_AT', 'ISO8859-1') ValueError: de_AT.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_BE', 'ISO8859-1') ValueError: de_BE.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_CH', 'ISO8859-1') ValueError: de_CH.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_DE', 'ISO8859-1') ValueError: de_DE.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_IT', 'ISO8859-1') ValueError: de_IT.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_LI', 'UTF-8') ValueError: de_LI.UTF-8 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_LU', 'ISO8859-1') ValueError: de_LU.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_DE', 'ISO8859-1') ValueError: de_DE.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_DE', 'ISO8859-1') ValueError: de_DE.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_DE', 'ISO8859-1') ValueError: de_DE.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_CH', 'ISO8859-1') ValueError: de_CH.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') locale: ('de_DE', 'ISO8859-1') ValueError: de_DE.ISO8859-1 (time data 'Samstag, 20 April' does not match format '%A, %d %B') Unsupported locale

Windows:

de_LI.UTF-8 locale: ('de_LI', 'UTF-8') Date: 1900-04-20 00:00:00 Unsupported locale

Linux Raspbian:

Unsupported locale

According to the ISO8859-1 supported characters, every character used in the date should be supported, why doesn't it work then? Which means on every system every code should work, I'm confused as hell.

firstof9 commented 1 year ago

As far as I can tell it does work if you're using the PyPi install of Home Assistant. It's something funky with the containers, I don't know why.

Pr0mises commented 1 year ago

Well not only containers, my linux is based on debian (raspbian) and it's not working as expected as well. Hass without docker is no option for me right now

firstof9 commented 1 year ago

It should still parse your delivered amazon emails.

Pr0mises commented 1 year ago

It's no even tracking that, the order numbers are tracked, that's it

firstof9 commented 1 year ago

what does your delivered emails from and subject have?

Pr0mises commented 1 year ago

I'm working on multiple fixes for germany rn (btw the codebase is awesome) Done:

Work in Progress:

Needs verification:

Added:

Pr0mises commented 1 year ago

If you could help to fix the locale thing it would be awesome as I'm completly lost there. Haven't seen such behaviour ever in python

Pr0mises commented 1 year ago

I did a workaround for the locale thing, i exclude the dayname and it's working this way.

But I've still one bug if the subject looks like that: Amazon Subject: Versendet: Für diese Amazon-Lieferung ist ein Einmalpasswort erforderlich, I receive Invalid date format found for language it_IT. (unconverted data remains: for date 14 April. If it looks like this e.g. Amazon Subject: Deine Amazon.de-Bestellung mit "product_name..." wurde versandt! it's working.

# Loop through all the langs for date format
for lang in AMAZON_LANGS:
    try:
        locale.setlocale(locale.LC_TIME, lang)
    except Exception as err:
        _LOGGER.debug("Locale error: %s (%s)", err, lang)
        continue

    if search == "Zustellung:":
        new_arrive_date = arrive_date.split(",", 1)[
            1
        ].strip()
        time_format = "%d %B"
    elif (
        "today" in arrive_date or "tomorrow" in arrive_date
    ):
        new_arrive_date = arrive_date.split(",")[1].strip()
        time_format = "%B %d"
    elif arrive_date.endswith(","):
        new_arrive_date = arrive_date.rstrip(",")
        time_format = "%A, %B %d"
    elif "," not in arrive_date:
        new_arrive_date = arrive_date
        time_format = "%A %d %B"
    else:
        new_arrive_date = arrive_date
        time_format = "%A, %B %d"

    try:
        dateobj = datetime.datetime.strptime(
            new_arrive_date, time_format
        )
        _LOGGER.debug("Valid date format found.")
    except ValueError as err:
        _LOGGER.debug(
            "Invalid date format found for language %s. (%s)",
            lang,
            err,
        )
        continue

    if (
        dateobj.day == datetime.date.today().day
        and dateobj.month == datetime.date.today().month
    ):
        deliveries_today.append("Amazon Order")
firstof9 commented 1 year ago

I've made some adjustments on this branch:

https://github.com/firstof9/Home-Assistant-Mail-And-Packages/tree/amazon-update

Can you see if they help? Thanks.

Pr0mises commented 1 year ago

Problem still persists with your branch with subject Versendet: Für diese Amazon-Lieferung ist ein Einmalpasswort erforderlich:

2023-04-19 01:00:03.058 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Amazon Subject: Versendet: Für diese Amazon-Lieferung ist ein Einmalpasswort erforderlich
2023-04-19 01:00:03.060 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: will arrive:
2023-04-19 01:00:03.061 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: estimated delivery date is:
2023-04-19 01:00:03.061 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: guaranteed delivery date is:
2023-04-19 01:00:03.061 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: Arriving:
2023-04-19 01:00:03.061 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: Arriverà:
2023-04-19 01:00:03.061 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: arriving:
2023-04-19 01:00:03.061 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: Dostawa:
2023-04-19 01:00:03.061 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: Zustellung:
2023-04-19 01:00:03.062 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] First pass: Freitag, 14 April
2023-04-19 01:00:03.062 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.070 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language it_IT. (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')
2023-04-19 01:00:03.070 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.072 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language it_IT.UTF-8. (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')
2023-04-19 01:00:03.072 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.074 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pl_PL. (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')
2023-04-19 01:00:03.074 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.076 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pl_PL.UTF-8. (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')
2023-04-19 01:00:03.076 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.078 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language de_DE. (unconverted data remains:
2023-04-19 01:00:03.078 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.087 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language de_DE.UTF-8. (unconverted data remains:
2023-04-19 01:00:03.087 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.089 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language es_ES. (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')
2023-04-19 01:00:03.089 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.102 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language es_ES.UTF-8. (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')
2023-04-19 01:00:03.102 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.104 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pt_PT. (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')
2023-04-19 01:00:03.104 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.106 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pt_PT.UTF-8. (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')
2023-04-19 01:00:03.106 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.130 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pt_BR. (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')
2023-04-19 01:00:03.131 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.132 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pt_BR.UTF-8. (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')
2023-04-19 01:00:03.132 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:03.134 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language . (time data 'Freitag, 14 April\n\tAm' does not match format '%A, %B %d')

Correct parse, with subject Deine Amazon.de-Bestellung mit "<PRODUCT>..." wurde versandt!:

2023-04-19 01:00:02.459 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Amazon Subject:Deine Amazon.de-Bestellung mit "<PRODUCT>..." wurde
2023-04-19 01:00:02.460 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: will arrive:
2023-04-19 01:00:02.461 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: estimated delivery date is:
2023-04-19 01:00:02.461 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: guaranteed delivery date is:
2023-04-19 01:00:02.461 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: Arriving:
2023-04-19 01:00:02.461 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: Arriverà:
2023-04-19 01:00:02.461 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: arriving:
2023-04-19 01:00:02.461 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: Dostawa:
2023-04-19 01:00:02.462 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Looking for: Zustellung:
2023-04-19 01:00:02.462 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] First pass: Freitag, 14 April
2023-04-19 01:00:02.462 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.464 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language it_IT. (time data 'Freitag, 14 April' does not match format '%A, %B %d')
2023-04-19 01:00:02.465 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.473 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language it_IT.UTF-8. (time data 'Freitag, 14 April' does not match format '%A, %B %d')
2023-04-19 01:00:02.473 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.474 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pl_PL. (time data 'Freitag, 14 April' does not match format '%A, %B %d')
2023-04-19 01:00:02.475 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.476 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pl_PL.UTF-8. (time data 'Freitag, 14 April' does not match format '%A, %B %d')
2023-04-19 01:00:02.477 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.493 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Valid date format found.
2023-04-19 01:00:02.493 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.495 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Valid date format found.
2023-04-19 01:00:02.495 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.497 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language es_ES. (time data 'Freitag, 14 April' does not match format '%A, %B %d')
2023-04-19 01:00:02.497 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.499 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language es_ES.UTF-8. (time data 'Freitag, 14 April' does not match format '%A, %B %d')
2023-04-19 01:00:02.499 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.510 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pt_PT. (time data 'Freitag, 14 April' does not match format '%A, %B %d')
2023-04-19 01:00:02.511 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.513 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pt_PT.UTF-8. (time data 'Freitag, 14 April' does not match format '%A, %B %d')
2023-04-19 01:00:02.513 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.514 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pt_BR. (time data 'Freitag, 14 April' does not match format '%A, %B %d')
2023-04-19 01:00:02.515 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.516 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language pt_BR.UTF-8. (time data 'Freitag, 14 April' does not match format '%A, %B %d')
2023-04-19 01:00:02.517 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Arrive Date: Freitag, 14 April
2023-04-19 01:00:02.537 DEBUG (SyncWorker_1) [custom_components.mail_and_packages.helpers] Invalid date format found for language . (time data 'Freitag, 14 April' does not match format '%A, %B %d')

also looping through all languages even if a hit is detected looks odd.

But overall it looks good. Wanted to do the functions after if I've done all of my changes

firstof9 commented 1 year ago

Ya, I'll have it stop after a valid date is found.

firstof9 commented 1 year ago

I've pushed a new commit to that branch, it should stop when it hits a valid date format now.

Pr0mises commented 1 year ago

I've pushed a new commit to that branch, it should stop when it hits a valid date format now.

Will check later

eric1905 commented 1 year ago

Hello, I am also from germany and have an issue with amazon. Maybe it is already fixed in your branch or maybe it is not discovered yet.

I got an EMail from versandbestaetigung with the following date range: Zustellung: Mittwoch, 14 Juni - Donnerstag, 15 Juni

My sensor stays to 0. And when I enable debug logs I see the following:

2023-06-14 08:14:34.137 DEBUG (SyncWorker_9) [custom_components.mail_and_packages.helpers] Looking for: will arrive: 2023-06-14 08:14:34.138 DEBUG (SyncWorker_9) [custom_components.mail_and_packages.helpers] Looking for: estimated delivery date is: 2023-06-14 08:14:34.138 DEBUG (SyncWorker_9) [custom_components.mail_and_packages.helpers] Looking for: guaranteed delivery date is: 2023-06-14 08:14:34.138 DEBUG (SyncWorker_9) [custom_components.mail_and_packages.helpers] Looking for: Arriving: 2023-06-14 08:14:34.138 DEBUG (SyncWorker_9) [custom_components.mail_and_packages.helpers] Looking for: Arriverà: 2023-06-14 08:14:34.138 DEBUG (SyncWorker_9) [custom_components.mail_and_packages.helpers] Looking for: arriving: 2023-06-14 08:14:34.138 DEBUG (SyncWorker_9) [custom_components.mail_and_packages.helpers] Looking for: Dostawa: 2023-06-14 08:14:34.138 DEBUG (SyncWorker_9) [custom_components.mail_and_packages.helpers] Looking for: Zustellung: 2023-06-14 08:14:34.138 DEBUG (SyncWorker_9) [custom_components.mail_and_packages.helpers] First pass: Donnerstag, 15 Juni Die Sendung geht an:

So it is ignoring the first date of the daterange and just using the second one. Do you have this behaviour as well or is it already fixed? How can I check out a specific branch in hacs to test your branch?

Pr0mises commented 1 year ago

@eric1905 what version are you using? 3.15 is the new one if I'm not misleading. Also I had no daterange to test, maybe it's undiscovered.

eric1905 commented 1 year ago

@Pr0mises I am on 0.3.16 but I thought @firstof9 pushed it to that branch:

https://github.com/firstof9/Home-Assistant-Mail-And-Packages/tree/amazon-update

But I just saw that this branch doesn't exist anymore so maybe it is already in the master branch

aebgit commented 1 year ago

Is there any update/progress on this?

eric1905 commented 1 year ago

For me this is still an issue.

The sensors sensor.mail_amazon_packages and sensor.mail_amazon_packages_delivered have the attribute

order:

The state of sensor.mail_amazon_packages is always 0 --> This is my issue. I got mails from Amazon.de versandbestaetigung@amazon.de

And from Amazon.de shipment-tracking@amazon.de

firstof9 commented 1 year ago

The amazon stuff needs refactoring, I just haven't had the time to do it.

firstof9 commented 9 months ago

This should be working in 0.3.20 please let me know if it doesn't solve your issue.

silke-99 commented 5 days ago

Hi, just saw this issue and can report this is still the case for me. I'm using version 0.4.1.

The related Amazon entities do contain the correct order# in the "Order" attribute, so it seems to be picking up the e-mail correctly. However, the state of the entity is still zero.

image

firstof9 commented 5 days ago

Is your order to be delivered today? If not, then 0 packages is correct.

silke-99 commented 5 days ago

yes it was announced and delivered today. The state never went away from zero.

firstof9 commented 5 days ago

You may have the wrong sensor enabled, the one in the image is for Amazon Hub.

silke-99 commented 5 days ago

Ah my bad, the issue was on user (my) level. The delivery came Tuesday, but the E-Mail announced it for Wednesday. (I should recalibrate my number detection :-) )

Eventually the state of "Amazon Packages" became "1" today.

Regarding the different entities I was confused because all four Amazon related entities had the order# in their attributes, so it wasn't clear to me which one would be valid. I'm a relatively new user of this integration.

Thanks for your help!

firstof9 commented 5 days ago

Regarding the different entities I was confused because all four Amazon related entities had the order# in their attributes, so it wasn't clear to me which one would be valid. I'm a relatively new user of this integration.

Yes all the amazon order numbers get added to the sensors.