encountering various errors with other cities

[Catchears]

Nice script - thought I'd test it with some other cities, but ran into 3 errors you might want to be aware of :)

Nortorf

>>> python arnis.py --city "Nortorf" --state "Schleswig Holstein" --country "Deutschland" ...
Fetching data...
Chosen server: https://overpass.kumi.systems/api/interpreter
Parsing data...
Processing data...
Traceback (most recent call last):
  File "C:\path\arnis.py", line 592, in <module>
    imgarray = processData(rawdata)
  File "C:\path\arnis.py", line 350, in processData
    if (imgLanduse[i[1]][i[0]] == 0): imgLanduse[i[1]][i[0]] = landuseType
IndexError: index 8846204 is out of bounds for axis 0 with size 4818

Gnutz

python arnis.py --city "Gnutz" --state "Schleswig Holstein" --country "Deutschland" ...
Fetching data...
Chosen server: https://maps.mail.ru/osm/tools/overpass/api/interpreter
Parsing data...
Processing data...
Traceback (most recent call last):
  File "C:\path\arnis.py", line 592, in <module>
    imgarray = processData(rawdata)
  File "C:\path\arnis.py", line 504, in processData
    if (str(img[y][x][0])[:1] != 5 and str(img[y][x][0])[:1] != 6 and str(img[y][x][0])[:1] != 7): img[i[1]][i[0]] = int("2" + str((wallHeight + 1)))
UnboundLocalError: local variable 'y' referenced before assignment

Kiel tries to request 590 GiB of memory - didn't happen with Hamburg, maybe it didn't get that far though (ran into the same IndexError as Nortorf).

python arnis.py --city "Kiel" --state "Schleswig Holstein" --country "Deutschland"  ...
Fetching data...
Chosen server: https://overpass-api.de/api/interpreter
Parsing data...
Traceback (most recent call last):
  File "C:\path\arnis.py", line 592, in <module>
    imgarray = processData(rawdata)
  File "C:\path\arnis.py", line 199, in processData
    img = np.zeros((round((greatestElementY - lowestElementY) / resDownScaler) + 5, round((greatestElementX - lowestElementX) / resDownScaler) + 5, 1), np.uint8)
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 590. GiB for an array with shape (1113821, 568481, 1) and data type uint8

[louis-e]

Hi there, thanks for opening an issue! Just fixed the Gnutz and Kiel issue. The Nortorf crash is most likely caused by two cities with the same name in the same state. I already had that problem during development and it took me quite a while to figure out haha! That's why I included the state tag, which had the purpose of minimizing the chance to get two different cities with the same name and a large distance between them returned. I'm not 100% sure tho if this really causes the crash in this case, but I think so. For now I can implement another check which should avoid that exact error to occur again. Will do that later!

[sparr]

$ python3 arnis.py --city Arnis --state "Schleswig Holstein" --country "Deutschland" --path ~/.minecraft/saves/asdf
Fetching data...
Chosen server: https://maps.mail.ru/osm/tools/overpass/api/interpreter
Parsing data...
Traceback (most recent call last):
  File "/home/sparr/src/arnis/arnis.py", line 571, in <module>
    imgarray = processData(rawdata)
  File "/home/sparr/src/arnis/arnis.py", line 178, in processData
    img = np.zeros((round((greatestElementY - lowestElementY) / resDownScaler) + 5, round((greatestElementX - lowestElementX) / resDownScaler) + 5, 1), np.uint8)
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 4.94 TiB for an array with shape (993759, 5466422, 1) and data type uint8

I get the memory allocation problem even with the documented and namesake city. Changing resDownScaler = 100 to resDownScaler = 1000 eliminates the problem by presumably reducing the memory allocation attempt by a factor of 10x10=100 to a barely manageable 49.4GiB, although I wasn't willing to wait for it to run to completion. Changing it to 3000 for another 9x reduction to fit in my available RAM made it run quickly, although I can't confirm it had any effect on my game world.

[louis-e]

That's definitely an unexpected behavior. I just tried several ways to reproduce this issue on multiple machines but none of them run into the same problem.

Can you run the script again with the --debug parameter and send me the arnis-debug-raw_data.json file over e.g. pastebin?

[sparr]

After update to current main I cannot reproduce that error.

[AndiLeni]

I will leave my found issue here as well:

This error occurs for Augsburg, Bayern, Deutschland

Traceback (most recent call last):
  File "C:\Users\andil\Desktop\arnis\arnis.py", line 10, in <module>
    run()
  File "C:\Users\andil\Desktop\arnis\src\main.py", line 119, in run
    imgarray = processData(rawdata, args)
  File "C:\Users\andil\Desktop\arnis\src\processData.py", line 162, in processData
    and int(element["tags"]["building:levels"]) <= 8
ValueError: invalid literal for int() with base 10: '1.5'

[louis-e]

Just released a hotfix for this problem. Thanks for the report!

[savvasenok]

Wien, Österreich

Traceback (most recent call last): File "/home/.../Desktop/arnis/arnis.py", line 10, in run() File "/home/.../Desktop/arnis/src/main.py", line 123, in run imgarray = processData(rawdata, args) File "/home/.../Desktop/arnis/src/processData.py", line 199, in processData and int(float(element["tags"]["building:levels"])) <= 8 ValueError: could not convert string to float: 'EG'

[louis-e]

@savvasenok Pushed a hotfix, should be working now. I hope that Wien doesn't make a big problem with its size though. Thanks for the report!