Error messages with dl_docs

[fsdjfhadflkjhsd]

Hi! First of all, thanks for all the work you put into this project and for sharing it here! High appreciated!

When I saw you updated it with a fix for the dl_docs problems, I went right ahead and downloaded everything from my TR-Acc. I downloaded a total of 100s of documents and while scrolling through the lines to see if everything worked smoothy, I noticed some "error-line", like such (maybe like 15 or so in total out of 100s of downloads):

13:41:17 2XX/9XX: Kauf StockA -- Di. 18.06. um 16:43 Uhr (Europe/Berlin) Received error message: '546 E {"errors":[{"errorCode":"BAD_REQUEST","errorField":null,"errorMessage":"Bad Request","meta":{"source":"PLATFORM"}}]}' 13:41:17 ('546', {'type': 'timelineDetail', 'id': '8g56243-6casd-1fff-22b8-034ed40e4d1c'}, {'errors': [{'errorCode': 'BAD_REQUEST', 'errorField': None, 'errorMessage': 'Bad Request', 'meta': {'source': 'PLATFORM'}}]})

I reran the command several times and the errors showed up in exactly the same lines every time.

Any idea what the problem might be? Just wanted to share it also to contribute as best as I can here by at least reporting what I found.

Cheers!

[marzzzello]

As I said it is not really fixed. It just ignores the errors for now

[somebody-somewhere-over-the-rainbow]

errors do not lead to a stop of the script but I am still not able to do a dl_docs. After two errors it hangs and nothing happens and no files are being downloaded...

14:11:58 Logged in
14:11:58 Awaiting #1  timeline
14:11:58 Received #1  timeline, awaiting #2  timeline
14:11:59 Received #2  timeline, awaiting #3  timeline
14:11:59 Received #3  timeline, awaiting #4  timeline
14:11:59 Received #4  timeline, awaiting #5  timeline
14:11:59 Received #5  timeline, awaiting #6  timeline
14:11:59 Received #6  timeline, awaiting #7  timeline
14:11:59 Received #7  timeline, awaiting #8  timeline
14:11:59 Received #8  timeline, awaiting #9  timeline
14:11:59 Received #9  timeline, awaiting #10 timeline
14:11:59 Received #10 timeline, awaiting #11 timeline
14:11:59 Received #11 timeline, awaiting #12 timeline
14:11:59 Received #12 timeline, awaiting #13 timeline
14:12:00 Received #13 timeline, awaiting #14 timeline
14:12:00 Received #14 timeline, awaiting #15 timeline
14:12:00 Received #15 timeline, awaiting #16 timeline
14:12:00 Received #16 timeline, awaiting #17 timeline
14:12:00 Received #17 timeline, awaiting #18 timeline
14:12:00 Received #18 (last) timeline
Received error message: '19 E {"errors":[{"errorCode":"BAD_REQUEST","errorField":null,"errorMessage":"Bad Request","meta":{"source":"PLATFORM"}}]}'
14:12:00 ('19', {'type': 'timelineDetail', 'id': '6ccfee3d-3585-4965-bf2d-1d83bbea57df'}, {'errors': [{'errorCode': 'BAD_REQUEST', 'errorField': None, 'errorMessage': 'Bad Request', 'meta': {'source': 'PLATFORM'}}]})
14:12:00 Received #19 (last) timeline
14:12:00   1/540: Abschluss Q4/2019 -- Mi. 01.01. um 00:59 Uhr (Europe/Berlin)
14:12:00   2/540:   -- Do. 30.05. um 11:53 Uhr (Europe/Berlin)
14:12:00   3/540: Basisinformationen -- Do. 30.05. um 11:53 Uhr (Europe/Berlin)
14:12:00   4/540:   -- Do. 30.05. um 11:53 Uhr (Europe/Berlin)
14:12:00   5/540:   -- Do. 30.05. um 11:55 Uhr (Europe/Berlin)
Received error message: '28 E {"errors":[{"errorCode":"BAD_REQUEST","errorField":null,"errorMessage":"Bad Request","meta":{"source":"PLATFORM"}}]}'
14:12:00 ('28', {'type': 'timelineDetail', 'id': '97ab309c-3101-45da-98f7-36d92172e041'}, {'errors': [{'errorCode': 'BAD_REQUEST', 'errorField': None, 'errorMessage': 'Bad Request', 'meta': {'source': 'PLATFORM'}}]})
14:12:00   6/540:   -- Do. 30.05. um 11:55 Uhr (Europe/Berlin)
14:12:00   7/540: Core MSCI EM IMI (Acc) -- Sparplan
14:12:01   8/540: Core MSCI World -- Sparplan
14:12:01   9/540: Core MSCI World -- Sparplan

[marzzzello]

Hm weird, I wanted to investigate the error more. The last version was just a quick fix which seems to work for most

[somebody-somewhere-over-the-rainbow]

yeah - no worries. I just wanted to make you aware ... if I can assist in any way, please tell me (not a coder though)

[somebody-somewhere-over-the-rainbow]

Correction: For some reason after the 3rd try it worked

[fsdjfhadflkjhsd]

As I said it is not really fixed. It just ignores the errors for now

Alright, superb, I get it! :) Just wanted to make sure to share anything that might help to improve the tool.

Closed 🥇

[CodingMaster3000]

I still get an error message but it is downloading correctly now. Thank you for the help.

[bloodydeluxe]

I get the error, but its not downloading. anyone else got the problem?

[fsdjfhadflkjhsd]

I get the error, but its not downloading. anyone else got the problem?

are you on the newest version?

[bloodydeluxe]

yes

[runzl0r]

Same happened for me. I can’t download the files. The script tells me 3 errors and after that it simply stops. Looks like this: 9/405…. Sixt

I waited almost an hour but it just stopped. I tried restarting it several times. Sometimes it runs up to 8 the next time to 9 but there appear no documents.

[bloodydeluxe]

did anyone get it working again?

edit: today it worked. got some errors in it, but got my pdf's

[Cornacchia83]

I currently also have the problem that the downloads no longer work. @marzzzello Can you fix it again?

[marzzzello]

Fixed in https://github.com/marzzzello/pytr/commit/0de1685432ceac9befff1fcb9030103f5720e59e

[runzl0r]

18:19:17  11/405: Unterlagen Aktualisierung -- Di. 05.11. um 07:54 Uhr (Europe/Berlin)
Traceback (most recent call last):
  File "/home/pi/.local/bin/pytr", line 10, in <module>
    sys.exit(main())
  File "/home/pi/.local/lib/python3.7/site-packages/pytr/main.py", line 152, in main
    asyncio.get_event_loop().run_until_complete(dl.dl_loop())
  File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/home/pi/.local/lib/python3.7/site-packages/pytr/dl.py", line 46, in dl_loop
    await self.tl.timelineDetail(response, self, max_age_timestamp=self.since_timestamp)
  File "/home/pi/.local/lib/python3.7/site-packages/pytr/utils.py", line 197, in timelineDetail
    timestamp = datetime.strptime(doc['detail'], '%d.%m.%Y').timestamp() * 1000
  File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime
    (data_string, format))
ValueError: time data 'Unsere Preise, fair und transparent' does not match format '%d.%m.%Y'

This is now the latest error I receive. It appears in python 3.8 and 3.7. Looks like it catches a wrong phrase somewhere?

[marzzzello]

This is now the latest error I receive. It appears in python 3.8 and 3.7. Looks like it catches a wrong phrase somewhere?

Check if 1c7b251 fixes your issue. I also made a new release. P.S. You can open a new issue next time as this is an independent issue

[runzl0r]

Thanks, works like a charm now. I just wonder why TR doesn't want us to have these files, they explain they don't have them but clearly they are there and well sorted. Thanks again

[trulyarya]

Hey there. I have the same issue:

22:42:03  1119/1119: Kauf Zoom -- Fr. 18.02. um 19:33 Uhr (Europe/Berlin)
22:42:03 Received all details
Traceback (most recent call last):
  File "/opt/homebrew/bin/pytr", line 8, in <module>
    sys.exit(main())
  File "/opt/homebrew/lib/python3.9/site-packages/pytr/main.py", line 157, in main
    asyncio.get_event_loop().run_until_complete(dl.dl_loop())
  File "/opt/homebrew/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/opt/homebrew/lib/python3.9/site-packages/pytr/dl.py", line 64, in dl_loop
    await self.tl.timelineDetail(response, self, max_age_timestamp=self.since_timestamp)
  File "/opt/homebrew/lib/python3.9/site-packages/pytr/utils.py", line 365, in timelineDetail
    export_transactions(dl.output_path / 'other_events.json', dl.output_path / 'account_transactions.csv')
  File "/opt/homebrew/lib/python3.9/site-packages/pytr/utils.py", line 107, in export_transactions
    lang = getlocale()[0].split('_')[0]
AttributeError: 'NoneType' object has no attribute 'split'

I am on the latest version (0.1.1).

[marzzzello]

@trulyarya That's a new issue. What's your system locale? Which OS?

[trulyarya]

@trulyarya That's a new issue. What's your system locale? Which OS?

Servus @marzzzello !

Thanks for the quick reply.

My locale:

LANG=""
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

My system:

[marzzzello]

Mh, I see. You can try to start pytr with LANG=en_US pytr dl_docs ... as a workaround until I push a fix

[trulyarya]

I did that and even tried other languages, still the same AttributeError: 'NoneType' object has no attribute 'split' error as above, and funnily enough on the same stock purchase (Zoom).

Doing

import locale
locale.getlocale()

gives this (Python 3.9.10):

(None, 'UTF-8')

[C0D3D3V]

What does the following return:

import locale
locale.getdefaultlocale()

You could also try: LC_CTYPE="en_GB.UTF-8" pytr dl_docs

because LC_CTYPE overwrites LANG

[marzzzello]

I think some LC_ variable needs to be set but never mind, I implemented a fallback fix in https://github.com/marzzzello/pytr/commit/5264843751ad5c8465351c9f3422b00643891496 and released a new version (0.1.2)

[trulyarya]

What does the following return:

import locale
locale.getdefaultlocale()

Still gave me (None, 'UTF-8').

I even did export LANG=en_US.UTF-8 in zsh terminal, which gave me LANG="en_US.UTF-8" when typing locale. But even after this, LANG=en_US pytr dl_docs PATH or simply pytr dl_docs PATH gave the same AttributeError like before.

You could also try: LC_CTYPE="en_GB.UTF-8" pytr dl_docs

because LC_CTYPE overwrites LANG

Thank you! I tried LC_CTYPE="en_US.UTF-8" pytr dl_docs and it worked flawlessly (with pytr v0.1.1 )!

---

I think some LC_ variable needs to be set but never mind, I implemented a fallback fix in 5264843 and released a new version (0.1.2)

Awesome. Now it works without setting/changing the language. Thanks for the fast version update!