Closed x4d3 closed 2 years ago
@keithdoggett @BuonOmo let me know what you think
I don't have much knowledge over the rgeo-proj4 codebase, but already have one question: if there is a perf improvement, how much do we gain?
Yeah good remark @BuonOmo
require "bundler/inline"
gemfile do
source "https://rubygems.org"
gem 'rgeo-proj4', github: 'x4d3/rgeo-proj4'
# gem "rgeo-proj4", "~> 3.1"
end
puts "preferred_native_interface #{RGeo::Geos.preferred_native_interface}"
puts "rgeo-proj4 version #{RGeo::Proj4::VERSION}"
puts "RGeo::Geos Info #{{supported: RGeo::Geos.supported?, proj4_supported: RGeo::CoordSys::Proj4.supported?, version: RGeo::CoordSys::Proj4.version}}"
# https://epsg.io/4326
ws84_definition = "proj=longlat +datum=WGS84 +no_defs +type=crs"
# https://epsg.io/2154
paris_lambert_definition = <<~EOS
+proj=lcc +lat_1=49 +lat_2=44 +lat_0=46.5 +lon_0=3 +x_0=700000 +y_0=6600000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs +type=crs
EOS
ws84_proj4 = RGeo::CoordSys::Proj4.create(ws84_definition)
raise "invalid ws84_definition" unless ws84_proj4
paris_lambert_proj4 = RGeo::CoordSys::Proj4.create(paris_lambert_definition)
raise "invalid paris_lambert_definition" unless paris_lambert_proj4
ws84_factory = RGeo::Cartesian.factory(srid: 4326, proj4: ws84_proj4)
paris_lambert_factory = RGeo::Cartesian.factory(srid: 2154, proj4: paris_lambert_proj4)
GEOMETRY_WKT = <<~EOS
MULTIPOLYGON (((784020.1897824854 6722964.293806808, 784037.326115887 6722975.889766449, 784054.4124898597 6722987.4862504555, 784071.3253728803 6722999.833471736, 784087.9890762889 6723013.43201526, 784104.3515158326 6723028.032402792, 784120.0308682051 6723043.388058192, 784134.7067634402 6723058.252599143, 784148.0652009461 6723072.12915206, 784159.8500073946 6723084.270231129, 784170.4972132337 6723095.42169378, 784180.7147581738 6723106.57680778, 784191.2547962071 6723119.22810715, 784202.6811317664 6723133.870442587, 784214.8759815434 6723150.754536163, 784227.4632711428 6723169.134420681, 784240.181080104 6723188.76220126, 784252.653331406 6723208.891610913, 784264.4439969584 6723228.777091963, 784275.221213894 6723248.171643657, 784284.4947434054 6723265.830027274, 784291.9327259483 6723281.505641706, 784298.2966828645 6723295.441536529, 784304.5985643048 6723309.128137981, 784311.9039028198 6723323.305754013, 784321.400262764 6723338.96387628, 784332.5380750942 6723356.106875415, 784344.6172699368 6723373.491695552, 784356.4424154337 6723390.378932787, 784367.2498671266 6723406.275734495, 784377.3693615372 6723421.178898737, 784388.1183614915 6723436.076913168, 784400.5524059066 6723451.709970812, 784415.821212109 6723469.317506157, 784433.52871879 6723488.153421866, 784452.4295334334 6723507.478738604, 784470.8843282094 6723526.058582454, 784487.5093193215 6723542.155666076, 784502.368705208 6723556.2690860145, 784516.2118965737 6723568.392673294, 784529.8525010797 6723579.019155646, 784544.3039663313 6723588.6392626595, 784559.558425686 6723597.503182013, 784575.0420814231 6723605.116047981, 784590.2174818617 6723611.732249156, 784604.44088523 6723616.857614208, 784617.4667343315 6723620.993872215, 784629.4128124788 6723623.890301049, 784640.2433945219 6723626.046546745, 784650.1783096731 6723627.461040881, 784659.2496563977 6723628.383231848, 784668.174745364 6723628.557302385, 784677.295429135 6723628.229868538, 784687.1470397501 6723626.897138264, 784697.8694634972 6723624.557523007, 784708.9673427406 6723621.7151745595, 784720.1830052689 6723618.622103866, 784731.0168432707 6723615.282357047, 784741.2968676833 6723611.447574695, 784750.6875940204 6723607.62045002, 784759.2168748197 6723603.550825318, 784766.4850260548 6723599.242197314, 784772.6897646365 6723594.44276483, 784777.9509994292 6723589.151908539, 784782.5863519495 6723583.1167083, 784786.961288117 6723575.834815708, 784791.1957108364 6723567.304911988, 784795.0519330226 6723557.778938195, 784798.2701595139 6723547.259102273, 784800.6684082886 6723535.497029851, 784802.1252754641 6723523.493236891, 784802.3182664857 6723509.751437822, 784801.0738921975 6723495.02257034, 784798.3037245563 6723478.308201662, 784793.7843121368 6723460.359494223, 784788.127298233 6723441.421170887, 784781.5909778869 6723422.49032073, 784774.7676320602 6723405.060836441, 784767.8870786584 6723389.130664893, 784760.0136832165 6723373.958494364, 784750.6308550239 6723357.550046889, 784738.7908438097 6723338.914192832, 784724.2353522843 6723317.053543477, 784707.9263733109 6723293.459249182, 784691.3040151886 6723270.616981534, 784678.976061013 6723254.533349478, 784675.7020984327 6723250.264152037, 784662.0126688856 6723233.892305451, 784649.2837293687 6723220.010606209, 784635.9831146887 6723205.883744532, 784620.9147737296 6723189.523616808, 784603.066757042 6723169.439897113, 784583.8285903189 6723146.869681756, 784564.9023020411 6723124.546537075, 784548.1255658776 6723104.703434426, 784535.1219723738 6723089.075222734, 784524.7717814728 6723076.4223131, 784515.8089956392 6723064.756996578, 784506.6315053658 6723051.344558031, 784496.0017910417 6723035.196530362, 784484.1696548788 6723016.310690788, 784471.7552316559 6722995.930974005, 784459.5206660905 6722975.549628993, 784448.0539926562 6722956.1609422555, 784437.0890500348 6722937.0180126205, 784425.7444064592 6722917.87821009, 784413.581909265 6722897.7459744355, 784399.9057362932 6722875.878055966, 784384.9121032218 6722853.021850328, 784369.1711865914 6722830.4220163785, 784353.2531560475 6722809.3224135395, 784337.6104054413 6722791.218522449, 784322.5484418896 6722775.607805061, 784307.9452370005 6722762.2414774075, 784293.907960408 6722749.619903008, 784280.3167058371 6722737.744000924, 784266.7832813218 6722725.617786431, 784252.6367154259 6722714.246345208, 784237.1790581255 6722702.636025109, 784220.0227432546 6722691.040340341, 784201.2640659683 6722680.207635438, 784181.8080638304 6722669.630677254, 784162.7381309723 6722659.799900687, 784144.7316082873 6722650.4597286945, 784127.9904608845 6722641.8582658805, 784112.1870717863 6722634.24811731, 784097.0116845404 6722627.631815419, 784082.0367635784 6722622.263114142, 784066.5107740257 6722617.898179469, 784049.562281599 6722614.294895784, 784030.169966921 6722611.21202143, 784007.4124334736 6722608.407665409, 783982.0348416284 6722605.375553562, 783956.031994045 6722602.848596293, 783931.7777657158 6722600.306678827, 783911.3326558475 6722597.982212025, 783895.3261667631 6722595.869746654, 783882.5713613448 6722594.229390163, 783871.9370069948 6722592.82083497, 783862.0705454329 6722592.405137684, 783852.3881866969 6722592.48741768, 783842.9662439888 6722593.816489142, 783833.6163697117 6722595.394868925, 783824.5126754063 6722597.720719233, 783815.7372182243 6722601.042863869, 783807.1558584626 6722604.863200681, 783798.5045521498 6722608.6839319905, 783789.9195677032 6722613.253662269, 783781.408773581 6722618.322403844, 783773.4617833871 6722623.885996045, 783766.5224957406 6722630.440308503, 783761.0905172229 6722637.981195592, 783757.2257988388 6722646.507948043, 783754.5607560001 6722656.273610394, 783752.9012929698 6722666.78019027, 783751.6257758558 6722677.783149326, 783750.6642590121 6722689.282982047, 783750.5205963242 6722701.775348378, 783751.4403425761 6722714.758220045, 783753.9994167396 6722728.976165743, 783758.3734330768 6722743.92825117, 783764.2483896465 6722759.117402907, 783771.1140676134 6722773.298952002, 783778.5244429724 6722785.726926091, 783786.4731507224 6722795.652316113, 783795.3626167289 6722804.570228123, 783806.1106181358 6722813.472546359, 783819.4393294791 6722823.851759641, 783835.784780275 6722836.453625396, 783855.1490952031 6722851.528146315, 783877.4038769861 6722868.077029753, 783902.287207964 6722885.852680436, 783929.6171073932 6722904.6068237815, 783958.7619492523 6722924.095006279, 783989.1600531361 6722944.072279992, 784020.1897824854 6722964.293806808)))
EOS
geometry = paris_lambert_factory.parse_wkt(GEOMETRY_WKT)
starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
1000.times do
RGeo::CoordSys::Proj4.transform(paris_lambert_proj4, geometry, ws84_proj4, ws84_factory)
end
puts "Processed in #{Process.clock_gettime(Process::CLOCK_MONOTONIC) - starting}"
With current master: 1.76
with that version: 1.22
so it's a (1.76-1.22)/1.22 x 100%
44% increase in performance !
(https://stackoverflow.com/a/53563651/1107536)
This was done on my machine, and of course it's more visible with a geometry with loads of points, but it does apply to my use case where I have to convert use geometries, with tons of points :)
Also something worth mentionning, is that CRSStore.get
is using a mutex to avoid multi-threading issues. Therefore the improvement of perf would be very much more visible in a multithreaded example.
Thanks for benchmarking that! Definitely a nice speed up for sure. I'll take a further look at this tomorrow, but I think it's a worthwhile change for sure and maintaining backwards compatibility is great.
As an aside, I just realized that your last change set us up to use this function under the hood for transforms https://proj.org/development/reference/functions.html#c.proj_trans_array. This would presumably help us speed up transformations for geometries other than points and since you got rid of the blocker of recomputing the crs_to_crs each time, it might be possible to implement this now.
@BuonOmo done !
Released in 3.1.1
The reason for this refactor is to avoid having to call
CRSStore.get
for every points when converting a geometry with a lot points, therefore I thought it would be a good idea to move the methods intoCRSToCRS
while keeping the backward compatibility.The goal, if this is merged, is to also modify
Feature.cast
in rgeo to useRGeo::CoordSys::Proj4.transform
directly for all the types of geometries and not only for the points.https://github.com/rgeo/rgeo/blob/master/lib/rgeo/feature/types.rb#L191-L229