QuickBooks Online: ChangeDataCapture service errors when returning Invoices with "Custom Fields"

[drmagu]

We used your quikbooks-devkit (as found on GitHub), and basically ran several tests successfully. We then incorporated this as part of a WordPress plugin for a client (of ours) to retrieve invoice information. We tested this successfully on our development server. However, after migrating to the client, errors started cropping up. After a lot of to-and-fro .. the only difference we could see is that we have the php.ini memory limit set to 192MB. The client has 128MB. When we tested at 128MB .. yep we get an internal server error (500). We have advised the client they need to upgrade their hosting plan. However, the question to you is this, since the only thing we need to do is connect (no problem) and retrieve invoices (crash), is there any config setting so the devkit runs with a smaller footprint?

Thank you Willy

* SEE COMMENTS * Although memory may be a factor, there is something else going on. We got the error log from the client's server, and see SimpleXML warnings.

[drmagu]

There may be more to it than just memory issues .. The query crashes on the client's server but not on our server. We use the QuickBooks_IPP_Service_ChangeDataCapture class to retrieve invoices that have changed "since" ...

    if ($Context = $IPP->context()) 
    {
        // Set the IPP version to v3 
        $IPP->version(QuickBooks_IPP_IDS::VERSION_3);
        // we will use the "ChangeDataCapture" service
        $CDCService = new QuickBooks_IPP_Service_ChangeDataCapture();

        //date since .. 
        $day = date('Y-m-d',$since);
        $time = date('H:i:s',$since);
        $timestamp = $day.'T'.$time.'Z';

        // What types of objects do you want to get? 
        $objects = array( 
            'Invoice', 
            );
        $cdc = $CDCService->cdc($Context, $realm, 
            $objects,
            $timestamp);

        // return object ... 

Here are the errors ..

20140619T113001: mytreeservice.com/wp/wp-admin/admin-ajax.php PHP Warning: SimpleXMLElement::__construct(): namespace error : Namespace prefix xsi for type on CustomField is not defined in /hermes/bosnaweb06a/b1975/sl.mytreese/public_html/wp/wp-content/plugins/zc -quickbooks-job-maps/lib/qbo-lib/QuickBooks/IPP/Object.php on line 53

PHP Warning: SimpleXMLElement::__construct(): <CustomField xsi:type="StringTypeCustomField"> in /hermes/bosnaweb06a/b1975/sl.mytreese/public_html/wp/wp-content/plugins/zc -quickbooks-job-maps/lib/qbo-lib/QuickBooks/IPP/Object.php on line 5

20140619T113002: mytreeservice.com/wp/wp-admin/admin-ajax.php PHP Fatal error: Error Num.: 2006 Error Msg.:MySQL server has gone away SQL:
SELECT
*
FROM
quickbooks_oauth
WHERE
app_username = 'DO_NOT_CHANGE_ME' AND app_tenant = '12345'
Database: /wrd_b66i4nhmcf in /hermes/bosnaweb06a/b1975/sl.mytreese/public_html/wp/wp-content/plugins/zc-quickbooks-job-maps/lib/qbo-lib/QuickBooks/Driver/Sql/Mysqli.php on line 379

Any ideas?

[consolibyte]

This doesn't look like a memory footprint problem at all - it looks like a MySQL time-out.

Can you tell where specifically in your code this error is getting triggered? Are you processing the list of objects and doing other stuff after you fetch them? Can you post the rest of your code? Can you tell how many objects you're getting back?

[drmagu]

You are right it isn't a memory issue. There were a lot of red herrings. We actually did find several issues with the client's server configuration, one of them being insufficient memory. However, there is an issue with the devkit, and we can now reproduce it.

When the invoices at QBO contain one or more "CustomField" items, the following PHP errors are triggered, multiple times for each invoice received from QBO ..

PHP Warning: SimpleXMLElement::__construct(): namespace error : Namespace prefix xsi for type on CustomField is not defined in /blahblah/QuickBooks/IPP/Object.php on line 53

In fact as I stated, a multitude of such warnings are generated for every invoice. These warnings, or the effect thereof, cause the client's server to crash. Our development server does not (crash), but we see the PHP warnings.

We created about 10 invoices in a test company, without custom fields, and everything processes just fine. We then activated the "Custom Fields" on the invoice template, and reimported the same invoices and indeed,we get warnings, and our client's server crashes.

[consolibyte]

Can you tell how many objects you're getting back?

Can you post some of the XML from $Service->lastResponse() that has the custom fields in it, so we can see what Intuit is returning?

Are you doing a lot of heavy processing of the data you get back? Queries that take a long time, etc.? Can you post your code?

[drmagu]

One object in this test .. No heavy processing. The time-out is only observed on the client's server .. over which I have very little control :). As far as I am concerned that is a red herring. The root cause, is the PHP errors. We did in fact cause a server crash (not on ours :) ) with a single invoice retrieval. It's the "CustomField" thing.

Here is the request ..

Request [GET https://quickbooks.api.intuit.com/v3/company/1216388085/cdc?entities=Invoice&changedSince=2010-01-01T00:00:00Z HTTP/1.1 Content-Type: text/plain Authorization: OAuth realm="", oauth_signature_method="HMAC-SHA1", oauth_signature="DP1FsKPVEN0TYO81SCSiLuYfRK0%3D", oauth_nonce="8FIhl", oauth_timestamp="1403707004", oauth_token="lvprd5Gw3SRffmT6faFU6G6uhKq9F7T6qiiNhVAJT0CY7xTd", oauth_consumer_key="qyprdcq5e1IEF0FczVknHAmP4HOJBi", oauth_version="1.0"

]

This the response for the Invoice WITH "Custom Fields"

Response [HTTP/1.1 200 OK Date: Wed, 25 Jun 2014 14:36:44 GMT Content-Encoding: gzip intuit_tid: 9d9b349f-9bf1-4be4-9520-cbbc11109648 Via: 1.1 ipp-gateway-.net Content-Type: application/xml;charset=UTF-8 QBO-Version: 74.210 Content-Length: 856 Cache-Control: max-age=0, no-cache, no-store, must-revalidate, private Expires: 0

102014-06-18T17:43:33-07:002014-06-18T17:43:33-07:00NoticeStringType10012014-06-18USD1135.00SalesItemLineDetail20351NON35.00SubTotalLineDetail13Willy Tester803 Edgewick CtSugar Land, TX 77478 USA29.6185208-95.60900092803 Edgewick CtSugar LandUSATX7747829.616688-95.59392632014-07-1835.00falseNotSetNotSet35.000falsefalsefalsefalse]

This is for a SINGLE invoice WITH a CustomField .. again, the PHP error (repeated 33 times) is:

Warning: SimpleXMLElement::__construct(): namespace error : Namespace prefix xsi for type on CustomField is not defined in /var/www/vhosts/m3pcs.info/subdomains/mytreeservice/httpdocs/wp/wp-content/plugins/zc-quickbooks-job-maps/lib/qbo-lib/QuickBooks/IPP/Object.php on line 53

And again, if I turn "Custom Fields" OFF in the company settings, the error does NOT occur. Just to be complete ... here is the SAME invoice .. with Custom Fields disabled:

Response [HTTP/1.1 200 OK Date: Wed, 25 Jun 2014 14:46:07 GMT Content-Type: application/xml;charset=UTF-8 Content-Length: 821 intuit_tid: 59f28766-bc73-4222-9f7b-457e14f1a8d2 Content-Encoding: gzip Via: 1.1 ipp-gateway-.net QBO-Version: 74.210 Cache-Control: max-age=0, no-cache, no-store, must-revalidate, private Expires: 0

102014-06-18T17:43:33-07:002014-06-18T17:43:33-07:0010012014-06-18USD1135.00SalesItemLineDetail20351NON35.00SubTotalLineDetail13Willy Tester803 Edgewick CtSugar Land, TX 77478 USA29.6185208-95.60900092803 Edgewick CtSugar LandUSATX7747829.616688-95.59392632014-07-1835.00falseNotSetNotSet35.000falsefalsefalsefalse]

Hope this helps. Willy

[consolibyte]

I think GitHub mangled your XML. Can you re-post, or post somewhere else and link to so we can see the XML?

[drmagu]

Or I don 't know what I am doing :). I agree it does not look like XML, but it is what is returned from QBO. You asked: "Can you post some of the XML from $Service->lastResponse() that has the custom fields in it, so we can see what Intuit is returning?"

OK .. this is what I did .. and it shows proper request / response .. no?

(...) // we will use the "ChangeDataCapture" service $CDCService = new QuickBooks_IPP_Service_ChangeDataCapture(); (...)

print('Request [' . $CDCService->lastRequest() . ']');

print('Response [' . $CDCService->lastResponse() . ']');

Not sure how to dump the XML ... or where that would be.

[consolibyte]

View > Source in your web browser.

[drmagu]

Here is the response .. it is in a GIST https://gist.github.com/drmagu/5b64014d7e3a374b9f74

[drmagu]

Did the XML response as documented in the GIST help?

[consolibyte]

Is there anyone experiencing this issue that could give me access to your QBO account so I can reproduce this? I tried reproducing with my QuickBooks Online account, but neither produces this error.

[drmagu]

@consolibyte .. I will email you an invitation to our test / development account. Just a thought. Could it be the version of the devkit library we are using?

[consolibyte]

I'm unable to reproduce these errors, even with your dev account (invitation from Greta).

Any chance you can IM me and we can get SSH/FTP to troubleshoot?

Or, go to line 53 in the Object.php class, and try adding this before it:

• libxml_use_internal_errors(true) And this after it:
• libxml_clear_errors();

And see if that supresses the errors.