kraina-ai / quackosm

QuackOSM: an open-source Python and CLI tool for reading OpenStreetMap PBF files using DuckDB
https://kraina-ai.github.io/quackosm/
Apache License 2.0
178 stars 10 forks source link

invalid pointer error #122

Closed aborruso closed 2 months ago

aborruso commented 3 months ago

Hi, first of all thank you for this great tool.

I am taking the first steps and I started with this command:

quackosm --geom-filter-geocode "Roccafiorita"

It works but at the end I have

free(): invalid pointer
[1]    3139 IOT instruction  quackosm --geom-filter-geocode "Roccafiorita"

What am I doing wrong?

THank you

image

RaczeQ commented 2 months ago

Hello @aborruso, sorry for the late reply.

From the logs you have provided, it seems like the operation worked properly (from the Python perspective), but there might be some internal problem with either Python installation or DuckDB installation on your machine. The free() call might be related to either C or Rust based dependencies and I have never encountered this error (library is tested on Ubuntu, Windows and Mac OS). It probably didn't release memory properly during operation, but logs indicate that everything went fine.

aborruso commented 2 months ago

hello @RaczeQ in output I have this folders/files

image

Does that sound like a correct output? What debugging could I do?

Thank you

RaczeQ commented 2 months ago

Yes, these are properly generated files:

If there was some error, you wouldn't see the third .parquet file, but since it exists, it means that operation finished successfully.

Here is my output:

~/development/quackosm# quackosm --geom-filter-geocode "Roccafiorita"
100%|████████████████████████████████████████| 191M/191M [00:00<00:00, 753GB/s]
⠇ [   1/32] Reading nodes • 0:00:11
  [   2/32] Filtering nodes - intersection 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 918/918 • 0:00:18 < 0:00:00 • 55.12 it/s
⠸ [   3/32] Filtering nodes - tags • 0:00:01
⠋ [   4/32] Calculating distinct filtered nodes ids • 0:00:00
⠸ [   5/32] Reading ways • 0:00:12
⠸ [   6/32] Unnesting ways • 0:00:06
⠼ [   7/32] Filtering ways - valid refs • 0:00:03
⠸ [   8/32] Filtering ways - intersection • 0:00:01
⠙ [   9/32] Filtering ways - tags • 0:00:00
⠋ [  10/32] Calculating distinct filtered ways ids • 0:00:00
⠼ [  11/32] Reading relations • 0:00:02
⠴ [  12/32] Unnesting relations • 0:00:02
⠹ [  13/32] Filtering relations - valid refs • 0:00:00
⠋ [  14/32] Filtering relations - intersection • 0:00:00
⠋ [  15/32] Filtering relations - tags • 0:00:00
⠋ [  16/32] Calculating distinct filtered relations ids • 0:00:00
⠋ [  17/32] Loading required ways - by relations • 0:00:00
⠋ [  18/32] Calculating distinct required ways ids • 0:00:00
⠇ [  19/32] Saving filtered nodes with geometries • 0:00:01
⠋ [20.1/32] Grouping filtered ways - assigning groups • 0:00:00
⠦ [20.2/32] Grouping filtered ways - joining with nodes • 0:00:04
⠋ [20.3/32] Grouping filtered ways - partitioning by group • 0:00:00
  [  21/32] Saving filtered ways with linestrings 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1 • 0:00:02 < 0:00:00 •  
⠋ [22.1/32] Grouping required ways - assigning groups • 0:00:00
⠸ [22.2/32] Grouping required ways - joining with nodes • 0:00:04
⠋ [22.3/32] Grouping required ways - partitioning by group • 0:00:00
  [  23/32] Saving required ways with linestrings 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1 • 0:00:02 < 0:00:00 •  
⠹ [  24/32] Saving filtered ways with geometries • 0:00:01
⠋ [  25/32] Saving valid relations parts • 0:00:00
⠋ [  26/32] Saving relations inner parts • 0:00:00
⠋ [  27/32] Saving relations outer parts • 0:00:00
⠋ [  28/32] Saving relations outer parts with holes • 0:00:00
⠋ [  29/32] Saving relations outer parts without holes • 0:00:00
⠋ [  30/32] Saving filtered relations with geometries • 0:00:00
⠋ [  31/32] Saving all features • 0:00:00
  [  32/32] Saving final geoparquet file 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4/4 • 0:00:00 < 0:00:00 •  
Finished operation in 0:01:21
files/b7463ec27e10f79cb256329f51a95e6d622f6c5090e4cc7267465be80943aa3e_nofilter_compact.parquet

Mine result is around 73 KB, so quite close to yours 93 KB (probably because of the QuackOSM version difference).

aborruso commented 2 months ago

thank you