search

h-a-graham / sfurrr

Attempts to parallelise some {sf} functions...
Other
3 stars 0 forks source link

Error with dependency "Error: 'write_geoarrow_parquet' is not an exported object from 'namespace:geoarrow'" #3

Open nickbond opened 1 year ago

nickbond commented 1 year ago

I'm getting the error Error: 'write_geoarrow_parquet' is not an exported object from 'namespace:geoarrow' when trying to run the examples. This obviously looks to be associated with the dependency on the arrow package, but thought I would post here first to see whether you had come across it or any suggestions.

Thanks for your efforts. Definitely helpful to be able to deploy more cores on some of these larger spatial join problems!

Regards

Nick

h-a-graham commented 1 year ago

Thanks for this, got to admit I haven't looked at this for some time but it actually might be useful for me again so I'll take a look next week. Cheers

h-a-graham commented 1 year ago

Actually, this reminds me, in case you haven't seen it and you're after a more efficient way to make these types of operations much faster, check this out: https://dewey.dunnington.ca/post/2022/profiling-point-in-polygon-joins-in-r/

nickbond commented 1 year ago

Great, thanks for that. Amazing how much faster one can make things run with a bit of tinkering under the hood. I started to look at trying to solve the problem but didn't get far sorry. Much appreciated.

cheers!

h-a-graham commented 1 year ago

Okay so I've made some updates - hopefully the issue should be solved - I've also changed the way that you set the number of processes to use as shown in the updated readme - it turns out you can very ofen get a good performance increase if you tweak the number of processes for the task at hand - hope this helps!