search

qbwc / quickbooks_api

A Ruby wrapper for the Quickbooks XML API
20 stars 10 forks source link

Determine a method to cache a loaded parser #9

Closed rposborne closed 11 years ago

rposborne commented 11 years ago

We need to decrease this things load time...

That or do the xml parsing during gem construction or something.

skryl commented 11 years ago

The parsing itself is fairly quick (~ 1 second), it's the wrapper class construction that takes time. These wrapper classes aren't strictly necessary. The reason I added them in the first place was to have some place to put qbxml constraints which aren't expressed in the xml structure itself (many of the the qbxml constraints are just comments in the qbxml schema file). I never got to that point in development though. The classes do improve runtime performance since they act as a sort of xml fragment cache during conversion with an obvious cost at load time.

An older iteration of the gem actually wrote all of the dynamically generated classes to disk via ruby2ruby but the sheer number of classes made load times almost as bad as just re-parsing.

I never profiled this code so I'm sure there are many obvious performance bottlenecks just waiting to be discovered. Since modifying/testing this code doesn't require a running QBPOS environment, I can probably do some refactoring, this time with tests :) in the near future. It would help if you guys could provide me with some common qbxml queries so I could use those as the beginnings of a test suite.

rposborne commented 11 years ago

As for Example qbxml queries anything from here is a good start, http://www.consolibyte.com/wiki/doku.php/quickbooks_example_qbxml

Hm, what about passing them off to memcached or something, where a pool of apps could pull the parser? ( I am starting to hit heroku's 30 second boot time window).

I will say any input would be greatly appreciated. Or if there is anything I could do just let me know.

skryl commented 11 years ago

New parser project started at http://github.com/skryl/qbxml. I decided to rewrite this thing from scratch and was able to keep it down to ~200 loc for the basic conversion/validation functionality by leveraging more Nokogiri and ActiveSupport builtins. Now I just need to add some tests to prove that this thing works the same as the old one. It doesn't have the same performance bottleneck since this one just uses Nokogiri's (much faster) in memory representation to do all the heavy lifting.

rposborne commented 11 years ago

Nice, I'll have work on QBWC to allow you to select parser of your choice.

skryl commented 11 years ago

K, I finished up with the backwards compatibility and ran some preliminary benchmarks.

Load Times quickbooks_api => 9.79 seconds qbxml => 0.11 seconds

Runtime Performance (300 large response conversions) quickbooks_api => 31.30 qbxml => 21.80

It can use some cleanup and could probably benefit from another set of eyes but seems to be winning so far :) I'll get a qbwc branch going with the qbxml gem swapped in but will need you guys to help do some live testing when the time comes.

Cheers,

Alex