doctrine / orm

Doctrine Object Relational Mapper (ORM)
https://www.doctrine-project.org/projects/orm.html
MIT License
9.93k stars 2.51k forks source link

suggestion: this would be helpful with large data sets #5558

Open joshuayoung1380 opened 8 years ago

joshuayoung1380 commented 8 years ago

I'm just suggesting a new result format method for the query builder

would be useful when dealing with nested arrays and large data sets

purpose it to keep from executing the same query with different filters too many times

defined column values could be translated to a key in one of the arrays levels and the bottom level of the array would be a list of the entity objects returned

currently using straight sql and parsing it in this format to solve sluggish processing times when dealing with huge sets of data

this would just be super useful when needing to cut back on the number of query's so that we can filter the query builders results efficiently after the query has run

just suggesting this for doctrines query builder

a useful method for the query builder:

$querybuilder->getNestedResult(array('col1','col2'));

array would be a list of columns, to use for keys in nested levels of the result

getNestedResult would return the following:

$result = array(
    'col1:value' => array(
        'col2:value' => array(
            list of &$entityObjects
        )
    )
)

example use-cases:

foreach($result as $myCategory => $items){
    foreach($myCategory as $subCategory => $products){
        foreach($products as $productEntityObject){

        }
    }
}

or, of the more common needed use-cases:

foreach($myOtherRelatedTree as $myCategory => $items){
    foreach($myCategory as $subCategory => $specs){
        $products = $result[$myCategory][$subCategory];
    }
}
Ocramius commented 8 years ago

@joshuayoung1380 I updated the description to fit markdown syntax

Ocramius commented 8 years ago

@joshuayoung1380 I don't understand the question: are you looking for array_column(), by chance?

joshuayoung1380 commented 8 years ago

I did not mean to ask a question, just wanted to suggest the possibility of adding a new getResult method for the query builder.

Entity: reference

//what a normal sql query would return

id name foreign_id date
1 test1 3 2015_01_01
2 test2 3 2015_01_02
3 test3 7 2015_01_02

//what $queryBuilder->getResult() returns

array(
   $entityObject,
   $entityObject,
   $entityObject
)

//what i think would be useful as an addition to the query result formats

//$results['date']['foreign_id'] = array of entityObjects
$results = array(
    ['2015_01_01'] => array(
        [3] => array(list of entity objects)
    )
    ['2015_01_02'] => array(
        [3] => array(list of entity objects)
        [7] => array(list of entity objects)
    )
)

http://doctrine-orm.readthedocs.org/projects/doctrine-orm/en/latest/reference/dql-doctrine-query-language.html#query-result-formats

I had an instance where i needed to do the following cause i was working with a large data set and needed to filter the actual query’s Results instead of re-run the query with filters for every date and difference from the foreign key

i guess i could have removed my filters and took the results from the query builder and re-parsed the results to the formatting i needed. just thought that it would be useful if there was something built into doctrine to make this easy

$referances = array();

$q = 'SELECT id, name, foreign_id, date

      FROM reference

      WHERE date BETWEEN "'.$startDate->format('Y-m-d').'" AND "'.$endDate->format('Y-m-d').';

$ref = $conn->fetchAll($q);

foreach ($ref as $r){

    $referances[$r[‘date’]][$r[‘foreign_id’]] = $r;

}

foreach($datePeriod as $date){

    if(array_key_exists($date->format('Y-m-d'), $referances)){

        $dateReferances = &$referances[$date->format('Y-m-d')];

    } else{

        $dateReferances = null;

    }


    foreach($foreigns as $foreign_entity){

        //filter site impressions by date

        if($siteImpressions !== null && array_key_exists($date->format('Y-m-d'), $dateReferances)){

            $feDateReferances = &$dateReferances[$date->format('Y-m-d')];

        } else{

            $feDateReferances = null;

        }


        if($feDateReferances !== null){

            foreach($feDateReferances as $referance){
                
            }
        }
    }
}
billschaller commented 8 years ago

@joshuayoung1380 There are a lot of result set transformations like these that are possible and often useful but ultimately not great candidates for inclusion in the core of the ORM. I build trees and maps and graphs of resultsets on a regular basis, but they are very easy to build userland abstractions for.