Open PavlosDem99 opened 10 months ago
@PavlosDem99 Thanks for your interest. Our bucket only contains the first OMF release, the 07/26 release.
We will no longer provide new OMF data from our end because, with our help, OMF is now using Sedona to generate GeoParquet files. However, in OMF Nov and Dec releases, they didn't correctly partition data based on spatial proximity so the spatial filter is extremely slow. W are helping OMF fix this issue in the next release.
In fact, s3a://overturemaps-us-west-2/release/2023-12-14-alpha.0/theme=places/type=place
works in our Sedona container and I just tried. But it is extremely slow because of the reason mentioned above.
@PavlosDem99 Took me about 5 min to download places data in Bellevue, WA
@PavlosDem99 Took me about 5 min to download places data in Bellevue, WA
Thank you @jiayuasu for your reply I appreciate it a lot. You are doing amazing job you and your team! Thanks I am waiting for the next release!
I am trying to download OFM data using the new release 12-14 but I cout an error. The error is: [PATH_NOT_FOUND] Path does not exist: s3a://wherobots-public-data/overturemaps-us-west-2/release/2023-12-14-alpha.0/theme=buildings/type=building.
When i use the 07/26 release all are working perfectly.
Also, when I use this code from here Overture Map Data , also does not work.
from sedona.spark import *
config = SedonaContext.builder().config("fs.s3a.aws.credentials.provider", "org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider").getOrCreate() sedona = SedonaContext.create(config)
df = sedona.read.format("geoparquet").load("s3a://overturemaps-us-west-2/release/2023-12-14-alpha.0/theme=places/type=place") df.filter("ST_Contains(ST_GeomFromWKT('POLYGON((-122.48 47.43,-122.20 47.75,-121.92 47.37,-122.48 47.43))'), geometry) = true").show()