tomwalder / php-gds

Google Cloud Datastore Library for PHP
Apache License 2.0
164 stars 44 forks source link

Inconsistent results from select #150

Closed olalekan-agbaje closed 7 years ago

olalekan-agbaje commented 7 years ago

I seriously need help with GDS and PHP-GDS Library.

I am doing a fetch from outside google appengine using the fantastic php-gds library. So far the library works fine.

My problem is that my data fetch from GDS returns inconsistent results and I have no idea what the issue might be.

Please see code below:

<?php
    //...
    //...
    $offset = 0;
    do{
       $query = "SELECT * FROM `KIND` order by _URI ASC limit 300 offset ".$offset;
       $tableList=[];
       $tableList = $this->obj_store->fetchAll($query);
       $offset += count($tableList);
       $allTables[] = $tableList;
       $totalRecords = $offset;
    }while(!empty($tableList));

    echo $totalRecords; // i expect total records to be equal to the number of entities in the KIND.
                        // but the results are inconsistent. In some cases, it is correct 
                        // but in most cases it is far less than the total records.
// I could have a KIND with 750 entities and only 721 will be returned in total.
// I could have a KIND with 900 entities and all entities will be returned.
// I could have a KIND with 4000 entities and only 1200 will be returned.
    ?>

Also when I run the exact same query in the cloud console I get the right entity count.

Thanks.

olalekan-agbaje commented 7 years ago

Problem solved using Cursors as advised by @tomwalder

<?php
  $query = "SELECT * FROM `KIND`";
  $tableList=[];
  $queryInit = $this->obj_store->query($query);
  do{
      $page = $this->obj_store->fetchPage(300);// fetch page.
      $tableList = am($tableList,$page); //merge with existing records.
      $this->obj_store->setCursor($this->obj_store->getCursor());//set next cursor to previous last cursor
  }while(!empty($page)); //as long as page result is not empty.
?>