Errors when parsing data

[jaison-jai-john]

Fetching recorded orders... Exception in thread Thread-2 (process): Traceback (most recent call last): File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1009, in _bootstrap_inner self.run() File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\threading.py", line 946, in run self._target(*self._args, *self._kwargs) File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\site-packages\keyboard_generic.py", line 58, in process if self.pre_process_event(event): File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\site-packages\keyboard__init.py", line 218, in pre_process_event callback(event) File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\site-packages\keyboard\init.py", line 649, in handler = lambda e: (event_type == KEY_DOWN and e.event_type == KEY_UP and e.scan_code in _logically_pressed_keys) or (event_type == e.event_type and callback()) File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\site-packages\keyboard\init__.py", line 637, in callback = lambda callback=callback: callback(args) File "C:\Users\LENOVO\Desktop\albion_scraper\mouse_move.py", line 158, in searchResources orders = thread.get_data() File "C:\Users\LENOVO\Desktop\albion_scraper\test.py", line 163, in get_data self.parse_data() File "C:\Users\LENOVO\Desktop\albion_scraper\test.py", line 149, in parse_data self.parsed.append(datapoint(list(json.loads(log).values()))) File "C:\Users\LENOVO\Desktop\albion_scraper\test.py", line 32, in init data[17] = datetime.strptime(data[17][0:16], "%Y-%m-%dT%H:%M") IndexError: list index out of range

im getting this error when recording data. im guessing it has something to do with incorrect parsing? i even get errors where the date is not in correct format

[jaison-jai-john]

Traceback (most recent call last): File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1009, in _bootstrap_inner self.run() File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\threading.py", line 946, in run self._target(*self._args, *self._kwargs) File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\site-packages\keyboard_generic.py", line 58, in process if self.pre_process_event(event): File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\site-packages\keyboard__init.py", line 218, in pre_process_event callback(event) File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\site-packages\keyboard\init.py", line 649, in handler = lambda e: (event_type == KEY_DOWN and e.event_type == KEY_UP and e.scan_code in _logically_pressed_keys) or (event_type == e.event_type and callback()) File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib\site-packages\keyboard\init__.py", line 637, in callback = lambda callback=callback: callback(args) File "C:\Users\LENOVO\Desktop\albion_scraper\mouse_move.py", line 158, in searchResources orders = thread.get_data() File "C:\Users\LENOVO\Desktop\albion_scraper\test.py", line 168, in get_data self.parse_data() File "C:\Users\LENOVO\Desktop\albion_scraper\test.py", line 154, in parse_data self.parsed.append(datapoint(list(json.loads(log).values()))) File "C:\Users\LENOVO\Desktop\albion_scraper\test.py", line 37, in init data[17] = datetime.strptime(data[17].replace(" ","")[0:16], "%Y-%m-%dT%H:%M") File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib_strptime.py", line 568, in _strptime_datetime tt, fraction, gmtoff_fraction = _strptime(data_string, format) File "C:\Users\LENOVO\AppData\Local\Programs\Python\Python310\lib_strptime.py", line 349, in _strptime raise ValueError("time data %r does not match format %r" % ValueError: time data '2022-03-13-(T19:' does not match format '%Y-%m-%dT%H:%M'

here a random '(' appears thus making it not in format.

[jaison-jai-john]

i placed a print statement to check the length of the parsed data and occassionally it goes below 19 thus creating errors like out of index

[hrichharms]

@GreeedWasTaken sorry to hear about this. how often does this kind of thing pop up and how important is the order expiry date for your use case? My best guess is that this is a case of UDP unreliability, in which adding "(T" to the PROBLEMS list on line 7 in AlbiPy.py might help. Although if you don't particularly care about expiry date, this could also obviously be replaced by placing line 17 into a try/except block for IndexError exceptions. Either of these solutions should work. If they fail, please let me know and I'll help you as best I can.

[Vmprogram]

Hello, I'm sorry I don't know English well, I'll use a translator. During the collection time sometimes come not correct example: 20 22-07-14T06:33:45.842459 i solved this problem replacing: data[17] = datetime.strptime(data[17][0:16], "%Y-%m-%dT%H:%M") on the: data[17] = data[17].replace(' ','') data[17] = datetime.strptime(data[17][0:16], "%Y-%m-%dT%H:%M")

[hrichharms]

Hello, I'm sorry I don't know English well, I'll use a translator. During the collection time sometimes come not correct example: 20 22-07-14T06:33:45.842459 i solved this problem replacing: data[17] = datetime.strptime(data[17][0:16], "%Y-%m-%dT%H:%M") on the: data[17] = data[17].replace(' ','') data[17] = datetime.strptime(data[17][0:16], "%Y-%m-%dT%H:%M")

interesting, does the code still work in other cases with that change?

[banex2win]

i know am a bit late to the party but i came across the same problem, and i made a stack overflow question and its a pretty simple answer : https://stackoverflow.com/questions/77563756/sniffing-game-market-data-is-throwing-an-error-python/77569315#77569315

the solution is legit just change line 32 in the code to the following try: data[17] = datetime.strptime(data[17][0:16], "%Y-%m-%dT%H:%M") except ValueError: data[17] = datetime(1900, 1, 1, 0, 0) except TypeError: data[17] = datetime(1900, 1, 1, 0, 0)

hope this helps someone

[banex2win]

as it turns out to be its an error that doesn't happen much, it happens once every 30-70 inputs on average but without the above code it throws errors randomly all the time

[hrichharms]

@banex2win thank you very much, would you like to create a pull request with this change? if not I will add it in just as soon as I get around to running some tests on it

[banex2win]

if possible i can get your discord to send you the code as am not very familiar with gethub

[banex2win]

okay so i created my solution to the problem in this fork https://github.com/banex2win/AlbiPy

its just a way to get arround corrupted data, cause sometimes some data isnot deciphered so itsnot put into the data list and that puts everything out of range , am not the best at explaining but its pretty much slapping ductape over it and calling it aday

if you want i can get in vc with you and tell you how i came up with that solution