search

dready92 / PHP-on-Couch

Data Access Library to access a CouchDB server with PHP.
http://dready.byethost31.com/index.php/display/view/192
GNU Lesser General Public License v3.0
246 stars 102 forks source link

Memory Leak #55

Closed nicolaisi closed 7 years ago

nicolaisi commented 10 years ago

Hi,

I'm facing this error

PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 64 bytes) in /home/paul/couch/lib/couch.php on line 152

I'm retrieving a huge amount of documents from my view.

MrTrick commented 10 years ago

This isn't specific to this couch library, it's just like any code. Holding objects in the program will consume your available memory. Too many at once, and you'll run out.

I'm going to assume that you don't need to have access to every single one of them at a time. Instead, you can process through them in batches.

Say you want to get every document in a view, you can do this;

  1. Fetch the view, limit=1000
  2. Process every result, noting the key and document id of the last result.
  3. Fetch the view again, start_key=$last_key, start_docid=$doc_id, skip=1, limit=1000
  4. Repeat step 2-3 until you fetch the view and less than 1000 results are returned.

You end up holding no more than 1000 documents in memory at once, and your app will be much 'appier.

It's discussed in more detail here; http://guide.couchdb.org/draft/recipes.html#pagination

nicolaisi commented 10 years ago

Yes, I don't have much idea with PHP, so I'm guessing it might has something to do with object allocation. The reason I thought it might work is due to my previous experience with python couchdb libraries, it never seems to be a problem to query a crazy amount of data at one time. So, if this has nothing to do with the library, then I think I can assume this is a restriction from the PHP.

Thanks for your explanation.