samvera-deprecated / rubydora

Fedora Commons version 3 REST API ruby library
Other
27 stars 17 forks source link

Upgrade to rest-client >= 1.7.1 slows writes to Fedora by over 30% #82

Open acurley opened 10 years ago

acurley commented 10 years ago

N.B. Unfortunately I don't have an answer yet to this issue and cannot make a pull rquest but felt to add it should others experience the same problem. When a solution is authored, either in this gem or rest-client, I'll be sure to link to this issue

I recently ran bundle update on my application and among the various gems that updated was rest-client, which upgraded from 1.6.8 to 1.7.1 and was released three days ago. rest-client is a dependency of rubydora whose .gemspec file does not put a limitation on the version of rest-client. I noticed a significant performance decrease when running my specs and successfully isolated the cause to rest-client.

Below is my git diff on Gemfile.lock and some time trials to prove problem. I even tried creating objects both with and without updates to Solr to ensure that the problem was with the write speed to Fedora. I create 10 AcquisitionProject objects (an ActiveFedora object) using FactoryGirl in four circumstances and use Ruby#Benchmark to give time results.

[10:25:15 curley@server:/opt/apps/andrew/ike] $ git diff Gemfile.lock
diff --git a/Gemfile.lock b/Gemfile.lock
index cda372b..c696b6a 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -220,6 +220,7 @@ GEM
     multi_json (1.10.1)
     net-http-persistent (1.9)
     net-ldap (0.5.1)
+    netrc (0.7.7)
     nokogiri (1.6.2.1)
       mini_portile (= 0.6.0)
     nom-xml (0.5.1)
@@ -309,12 +310,10 @@ GEM
       rdf (~> 1.1, >= 1.1.4)
     rdf-xsd (1.1.1)
       rdf (~> 1.1)
-    rdoc (4.1.1)
-      json (~> 1.4)
     ref (1.0.5)
-    rest-client (1.6.8)
-      mime-types (~> 1.16)
-      rdoc (>= 2.4.2)
+    rest-client (1.7.1)
+      mime-types (>= 1.16, < 3.0)
+      netrc (~> 0.7)
     rolify (3.4.0)
     rsolr (1.0.10)
       builder (>= 2.1.2)

Under rest-client 1.6.8

irb(main):001:0> ENABLE_SOLR_UPDATES=false
=> false
irb(main):002:0> Benchmark.realtime { 10.times { FactoryGirl.create(:base_acquisition_project) }}
=> 3.20260709
irb(main):003:0> Benchmark.realtime { 10.times { FactoryGirl.create(:base_acquisition_project) }}
=> 2.818982106
irb(main):004:0> ENABLE_SOLR_UPDATES=true
=> true
irb(main):005:0> Benchmark.realtime { 10.times { FactoryGirl.create(:base_acquisition_project) }}
=> 2.905722773
irb(main):006:0> Benchmark.realtime { 10.times { FactoryGirl.create(:base_acquisition_project) }}
=> 2.967691368

Under rest-client 1.7.1

irb(main):006:0> ENABLE_SOLR_UPDATES=false
=> false
irb(main):007:0> Benchmark.realtime { 10.times { FactoryGirl.create(:base_acquisition_project) }}
=> 3.989939417
irb(main):008:0> Benchmark.realtime { 10.times { FactoryGirl.create(:base_acquisition_project) }}
=> 3.729743344
irb(main):009:0> ENABLE_SOLR_UPDATES=true
=> true
irb(main):010:0> Benchmark.realtime { 10.times { FactoryGirl.create(:base_acquisition_project) }}
=> 3.913787696
irb(main):012:0> Benchmark.realtime { 10.times { FactoryGirl.create(:base_acquisition_project) }}
=> 3.761060276

I can find nothing on the rest-client issue tracker about others experiencing the same kind of slowness so perhaps the problem is somewhere in rubydora and it's use of the RestClient class.

acurley commented 10 years ago

It strikes me that the obvious answer is to amend the .gemspec file to rest-client 1.6.8 but that seems premature and doesn't solve the problem. But if that is the consensus, I'll be glad to submit the pull request.