Closed renta closed 2 years ago
Hi there,
I am the one who opened #944
I'm currently having the same use case as you (and surely many others). I'd also like to have this feature natively, however it could get very difficult to handle all use cases (=> you would need to configure filters several times depending on the behaviour you would like to apply, filtering subresources or not, maybe not all subfilters should filter subresources, etc., not very handy)
As @soyuka says (https://github.com/api-platform/api-platform/issues/424#issuecomment-343167947), you can actually achieve the behaviour with having a single sql statement by adding Doctrine filters https://api-platform.com/docs/core/filters#using-doctrine-filters . Up to you to determine if the filters should be enabled by default, then (dis|en)abled/configured on demand (by query string, specific custom action/event listener, etc.).
Maybe it's mostly a documentation problem, describing this common use case ?
Thank you for the answer @ymarillet. I've tried to solve this task with a custom filter and it would work for very simple use cases. But when I wanted to achieve the same behaviour as in SearchFilter for example, I had to copy-paste all its code, altered some blocks of Doctrine query builder, but my changes went not in the needed part of the result sql-query. Also, this way is related with copy-pasting of the original APIP filters (because I want SearchFilter "on steroids")...
Now I'm trying to deal with this task with a custom extension. Despite of custom filter, where you need to reinvent original APIP filters, extensions query builder in the applyToCollection method already has all the filter stuff - I just need to "monkey patch" it (copy filter condition to the part external part of the query). But it's also seems to be an ugly quirk (because I need to hardcode a filter value processing code there).
It would be ideal to have an ability of switching "on" and "off" FilterEagerLoadingExtension. Maybe there is a way to do it with service configuration or on events level?
Maybe it's mostly a documentation problem, describing this common use case ?
I'll be immensely glad to see an example of this problem workaround.
I'll be immensely glad to see an example of this problem workaround.
I unfortunatly don't have a clean way to do that for now, I find my solution pretty ugly atm, I'd need to think more about it to expose clean code covering most common use cases. My business use cases are: filtering subresource from an active/inactive status and/or validity dates (date start, date end).
For now, I have a class implementing ApiPlatform\Core\Bridge\Doctrine\Orm\Extension\ContextAwareQueryCollectionExtensionInterface
and ApiPlatform\Core\Bridge\Doctrine\Orm\Extension\QueryItemExtensionInterface
, introspecting into $context['filters']
to determine which Doctrine filters should be enabled.
Hi! I'm trying to solve the same problem here.
I'm new in api-platform and I'm using it for a GraphQL API, I think a good aproach would be to apply filter dependant of the nest level where we apply the filter. I'm going to try my best to explain it.
This example shows a list of products that only have stores with storeId = 3:
query test {
products(productStore_storeId: 3) {
edges {
node {
customCode
description
productStore {
edges {
node {
storeId
productId
state
}
}
}
}
}
}
}
This example shows a list of products and her stores, and filter the list by storeId = 1
query test {
products {
edges {
node {
customCode
description
productStore(productStore_storeId: 1) {
edges {
node {
storeId
productId
state
}
}
}
}
}
}
}
I think this aproach isn't viable for a REST API, but makes sense on a GraphQL API. To implement this aproach I don't know where to start, I have questions like, where are the filter parameters recollected.
At this moment I'm implementing ContextAwareQueryCollectionExtensionInterface
@ymarillet is there any way you could share here, not the file, but at least some tips about how to do it? I already know how to filter resources, but I've no idea about how to filter nested resources from a parent resource without creating a custom controller with custom methods. I'd like to use the api-platform REST structure and add my filters there and avoid creating methods for specific uses that should be easily done using the available framework utilities.
So.. If anyone can tell me some clues about how to filter subresources... I'll owe him/her a beer (or a coffee, I don't drink btw, but most developers love beer x_D)
Hi! I have a quick and dirty fix for this, you need to implement a custom search filter and overwrite the "filterProperty" function, here's my code, I think it's all to change:
protected function filterProperty(string $property, $value, QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, string $operationName = null, $context = [])
{
if (
null === $value ||
!$this->isPropertyEnabled($property, $resourceClass) ||
!$this->isPropertyMapped($property, $resourceClass, true)
) {
// If the property exist in the current entity, add it to the where, with this, we can filter lists on nest level
if(!$this->existPropertyInProperties($property, $resourceClass)) {
return;
}
}
$alias = $queryBuilder->getRootAliases()[0];
$field = $property;
if ($this->isPropertyNested($property, $resourceClass)) {
list($alias, $field, $associations) = $this->addJoinsForNestedProperty($property, $alias, $queryBuilder, $queryNameGenerator, $resourceClass);
$metadata = $this->getNestedMetadata($resourceClass, $associations);
} else {
$metadata = $this->getClassMetadata($resourceClass);
}
$values = $this->normalizeValues((array) $value);
if (empty($values)) {
$this->logger->notice('Invalid filter ignored', [
'exception' => new InvalidArgumentException(sprintf('At least one value is required, multiple values should be in "%1$s[]=firstvalue&%1$s[]=secondvalue" format', $property)),
]);
return;
}
$caseSensitive = true;
// If the property exist in the current entity, add it to the where, with this, we can filter lists on nest level
if (($pos = strpos($property, ".")) !== FALSE) {
$extractedProperty = substr($property, strpos($property, ".") + 1);
if(array_key_exists($extractedProperty, $this->properties)) {
$property = $extractedProperty;
$field = $extractedProperty;
}
}
if ($metadata->hasField($field)) {
if ('id' === $field) {
$values = array_map([$this, 'getIdFromValue'], $values);
}
if (!$this->hasValidValues($values, $this->getDoctrineFieldType($property, $resourceClass))) {
$this->logger->notice('Invalid filter ignored', [
'exception' => new InvalidArgumentException(sprintf('Values for field "%s" are not valid according to the doctrine type.', $field)),
]);
return;
}
$strategy = $this->properties[$property] ?? self::STRATEGY_EXACT;
// prefixing the strategy with i makes it case insensitive
if (0 === strpos($strategy, 'i')) {
$strategy = substr($strategy, 1);
$caseSensitive = false;
}
if (1 === \count($values)) {
$this->addWhereByStrategy($strategy, $queryBuilder, $queryNameGenerator, $alias, $field, $values[0], $caseSensitive);
return;
}
if (self::STRATEGY_EXACT !== $strategy) {
$this->logger->notice('Invalid filter ignored', [
'exception' => new InvalidArgumentException(sprintf('"%s" strategy selected for "%s" property, but only "%s" strategy supports multiple values', $strategy, $property, self::STRATEGY_EXACT)),
]);
return;
}
$wrapCase = $this->createWrapCase($caseSensitive);
$valueParameter = $queryNameGenerator->generateParameterName($field);
$queryBuilder
->andWhere(sprintf($wrapCase('%s.%s').' IN (:%s)', $alias, $field, $valueParameter))
->setParameter($valueParameter, $caseSensitive ? $values : array_map('strtolower', $values));
}
I check if the filter property exist in the related entity and add a where to the query
I could provide my solution for this problem, if anyone interested. It helps to filter subresources with GET parameter, but it require patch of the Api Platform core file (I did it with post-install script in composer.json - so it could be broken one day). I'm not really proud of my solution but it works as expected with minimum code changes (that's why I'm for the same-way functionality in the core framework).
@elboletaire : my solution involves doctrine filters. Make sure you understand what this means if you want to use mine.
The example down there is very specific and pretty dirty (up to you to modify depending on the complexity/genericity needed on your project).
Say you have two entities (configure Apiplatform and ORM annotations where needed...):
class Foo
{
private $id;
/** @var Bar[] **/
private $bars;
}
class Bar
{
private $id;
/** @var string */
private $status;
}
You will be able to filter Bar entities embedded in Foo entities through the query string.
i.e. :
/api/foos?bars.status=active
: means you want all Foo having at least one active Bar (which is the default meaning of such an ApiPlatform query string)
/api/foos?bars.status=active&filter_associations=true
: same as above but you only want active Bars in the result (ApiPlatform would return all Bars, active or not, without the doctrine filter)
Declare the doctrine filter config/packages/doctrine.yaml
:
doctrine:
orm:
entity_managers:
default:
filters:
exposable:
class: 'App\Bridge\Doctrine\Filter\ExposableFilter'
enabled: false # will be enabled per use case
Code it:
<?php
namespace App\Bridge\Doctrine\Filter;
use Doctrine\ORM\EntityManagerInterface;
use Doctrine\ORM\Mapping\ClassMetadata;
use Doctrine\ORM\Query\Filter\SQLFilter;
final class ExposableFilter extends SQLFilter
{
/**
* @var EntityManagerInterface
*/
private $entityManager;
/**
* @param ClassMetadata $targetEntity
* @param string $targetTableAlias
*
* @return string
*
* @throws \Exception
*/
public function addFilterConstraint(ClassMetadata $targetEntity, $targetTableAlias): string
{
$em = $this->getEntityManager();
if (!$em->getFilters()->isEnabled('exposable')) {
return '';
}
// this is a example of how you can select the entities affected by this filter
if (Bar::class !== $targetEntity->reflClass->getName()) {
return '';
}
if (!($value = $this->getParameter('status'))) {
return '';
}
return sprintf('%s.status = %s', $targetTableAlias, $this->getParameter('status'));
}
/**
* @return EntityManagerInterface
*
* @throws \ReflectionException
*/
private function getEntityManager(): EntityManagerInterface
{
if (null === $this->entityManager) {
$refl = new \ReflectionProperty(SQLFilter::class, 'em');
$refl->setAccessible(true);
$this->entityManager = $refl->getValue($this);
}
return $this->entityManager;
}
}
Code ApiPlatform extension:
<?php
namespace App\Bridge\Doctrine\ORM\Extension;
use ApiPlatform\Core\Bridge\Doctrine\Orm\Extension\ContextAwareQueryCollectionExtensionInterface;
use ApiPlatform\Core\Bridge\Doctrine\Orm\Extension\QueryItemExtensionInterface;
use ApiPlatform\Core\Bridge\Doctrine\Orm\Util\QueryNameGeneratorInterface;
use Doctrine\Bundle\DoctrineBundle\Registry;
use Doctrine\ORM\EntityManagerInterface;
use Doctrine\ORM\QueryBuilder;
class ExposableFilterExtension implements ContextAwareQueryCollectionExtensionInterface, QueryItemExtensionInterface
{
/**
* @var Registry
*/
private $doctrine;
/**
* @param Registry $doctrine
*/
public function __construct(Registry $doctrine)
{
$this->doctrine = $doctrine;
}
/**
* @param QueryBuilder $queryBuilder
* @param QueryNameGeneratorInterface $queryNameGenerator
* @param string $resourceClass
* @param string|null $operationName
* @param array $context
*/
public function applyToCollection(
QueryBuilder $queryBuilder,
QueryNameGeneratorInterface $queryNameGenerator,
string $resourceClass,
string $operationName = null,
array $context = []
) {
$this->enableFilters($context);
}
/**
* @param QueryBuilder $queryBuilder
* @param QueryNameGeneratorInterface $queryNameGenerator
* @param string $resourceClass
* @param array $identifiers
* @param string|null $operationName
* @param array $context
*/
public function applyToItem(
QueryBuilder $queryBuilder,
QueryNameGeneratorInterface $queryNameGenerator,
string $resourceClass,
array $identifiers,
string $operationName = null,
array $context = []
) {
$this->enableFilters($context);
}
/**
* @param array $context
*/
private function enableFilters(array $context): void
{
$filters = $context['filters'] ?? [];
if (empty($filters['filter_associations'])) {
return;
}
/** @var EntityManagerInterface $em */
$em = $this->doctrine->getManager();
// hardcoded !! change to your needs.
$status = $filters['bars.status'] ?? null;
if (empty($status)) {
return;
}
if ($em->getFilters()->isEnabled('exposable')) {
return;
}
$em->getFilters()->enable('exposable');
$em->getFilters()->getFilter('exposable')->setParameter('status', $status);
}
}
Configure the service:
services:
App\Bridge\Doctrine\ORM\Extension\ExposableFilterExtension:
tags:
- { name: 'api_platform.doctrine.orm.query_extension.collection', priority: 128 }
- { name: 'api_platform.doctrine.orm.query_extension.item', priority: 128 }
Ok, now that I'm filtering nested results... doctrine throws an EntityNotFoundException
because it can't find the just filtered entity... sorry, what? :joy:
That's probably because of https://github.com/doctrine/doctrine2/issues/4543
I workarounded that with a normalizer
<?php
namespace App\Serializer\Normalizer;
use Doctrine\Common\Persistence\Proxy;
use Doctrine\ORM\EntityNotFoundException;
use Symfony\Component\Serializer\Normalizer\NormalizerAwareTrait;
use Symfony\Component\Serializer\Normalizer\NormalizerInterface;
class ExposableNormalizer implements NormalizerInterface
{
use NormalizerAwareTrait;
/**
* @param Proxy $object
* @param string|null $format
* @param array $context
*
* @return mixed
*/
public function normalize($object, $format = null, array $context = [])
{
try {
return $this->normalizer->normalize($object, $format, $context);
} catch (EntityNotFoundException $e) {
return null;
}
}
/**
* @param mixed $data
* @param string|null $format
* @param array $context
*
* @return bool
*/
public function supportsNormalization($data, $format = null, array $context = [])
{
return $data instanceof Bar && $data instanceof Proxy;
}
}
I think this would be a nice feature to have, as there's clearly a lot of demand for it. But we need to think about how to do this.
Taking @ymarillet's example from above, perhaps it could be:
GET /foos?bars.status=active&prune[]=bars
or to prune everything:
GET /foos?bars.status=active&prune[]=*
Importantly, we also need to consider the semantics for conveying partial / filtered sub-collections in the various formats that we support.
WDYT @soyuka @dunglas?
I think @marcgraub's proposal for nested filters is also very interesting (and much more flexible), adapted for a REST API:
GET /products?pick[stores][id]=1
which means the filter id = 1
is applied on stores
.
(Naming inspiration: https://ramdajs.com/docs/#pick)
But I'm not sure if we could implement this without much difficulty.
To implement this sort of thing, I'm actually doing this.
I set force_eager to true on my entity Then I create a custom doctrine filter like this :
foreach ($queryBuilder->getAllAliases() as $alias) {
if (0 === strpos($alias, str_lower($ressourceClass))) {
$queryBuilder
->andWhere("$alias.$property = :value")
->setParameter('value', filter_var($value, FILTER_VALIDATE_BOOLEAN, FILTER_NULL_ON_FAILURE))
;
}
}
I've encountered the same problem with the date filter. I was trying to between dates in a subresource. This resulted in the resource being filtered but all subresources where returned.
I've traced the problem to the FilterEagerLoadingExtension
. This extension clones the query and then includes it in the werepart. In the process the wherepart on the original query is removed.
I'm not completely sure if this is on purpose and I couldn't come up with an issue when the wherepart is kept in the original query.
I could fix the issue by adding the following at the end of applyToCollection
in the FilterEagerLoadingExtension
:
$combinedWherePart = $queryBuilderClone->getDQLPart('where');
foreach ($wherePart->getParts() as $part) {
$combinedWherePart->add($part);
}
$queryBuilder->resetDQLPart('where');
$queryBuilder->add('where', $combinedWherePart);
@ymarillet your solution seems the one that works for me. Did you ever found a more generic solution for it? Or have anybody else did it?
Nope unfortunatly I couldn't spend more time on the issue as my solution was satisfying enough for my client(s).
It could be better to make 2+ separate HTTP requests though (the second being on the corresponding subresource with proper filters) if you don't want to be that hacky. I would do it that way today I think - the code I showed up previously is really 🤮 and can lead to problems as it's not flexible enough.
@ymarillet I was thinking the same, doing 2 calls. But I hope that ApiPlatform will offer a good solution for it, as it seems to be a very common use case.
Same problem here, can't find a pretty solution !
I would like some feedback on my Workaround :
My problem was to filter the contracts
of container
by their date.
The container were filtered but not the associated contracts.
I decided to hack the DQL Parts of the join with something like this :
$minCondition = "$contractAlias.startedAt <= :maxDate";
$maxCondition = "$contractAlias.endedAt >= :minDate OR $contractAlias.endedAt IS NULL";
$queryBuilder
->andWhere($minCondition)
->andWhere($maxCondition)
->setParameter('minDate', '2015-01-01')
->setParameter('maxDate', '2015-06-01')
;
$originalJoins = $queryBuilder->getDQLPart('join');
// We want to modify a join, so we will reset all, and recreate them, with 'container.contract' one modified
$queryBuilder->resetDQLPart('join');
// We keep all other joins as they are
foreach ($originalJoins as $alias => $originalJoinParts) {
// Skip the jointures of container (well (re)add those bellow
if ($alias !== $containerAlias) {
foreach ($originalJoinParts as $originalJoinPart) {
$queryBuilder->add('join', [$alias => $originalJoinPart], true);
}
}
}
// We (re)create the join of container.contracts
$contractJoinPart = new Join(
Join::LEFT_JOIN,
"$containerAlias.contracts",
$contractAlias,
Join::WITH,
$queryBuilder->expr()->andX($minCondition, $maxCondition)
);
// We add it to the joins associated to container
$queryBuilder->add('join', [$containerAlias => $contractJoinPart], true);
// Finally we add all other joins associated to container
foreach ($originalJoins[$containerAlias] as $joinPart) {
if ($joinPart->getAlias() !== $contractAlias) {
$queryBuilder->add('join', [$containerAlias => $joinPart], true);
}
}
You can also take a look at my solution by fixing it in the filter. This is where to issue starts because the 'where' part gets removed from the main query and is only included in the subqueries. The solution I posted above did solve it for us.
@HeinDR not sure to understand exactly where you did this ? Did you decorate FilterEagerLoadingExtension
?
No I just replaced it with a copy. like this:
api_platform.doctrine.orm.query_extension.filter_eager_loading: class: App\ApiPlatform\FilterEagerLoadingExtension
`App\ApiPlatform\FilterEagerLoadingExtension:
tags:
- { name: api_platform.doctrine.orm.query_extension.filter }
- { name: api_platform.doctrine.orm.query_extension.filter_eager_loading }`
@HeinDR Ok thanks, Overriding base of ApiPlatform
classes something i would like to avoid as much as I can.
@bastoune I get that. However, this issue came from a bug introduced in the FilterEagerLoadingExtension. It is mentioned on the original issue 944 . I think my solution does fix the issue at the source. I was hoping for some feedback and than maybe create a pull request for it. But I got no response so I'm not sure if there are objections to this solution.
not entirely sure, but doesn't this solve this issue?
https://symfonycasts.com/screencast/api-platform/relation-filtering
Correct me if I'm wrong, it worked for me
not entirely sure, but doesn't this solve this issue?
https://symfonycasts.com/screencast/api-platform/relation-filtering
Correct me if I'm wrong, it worked for me
It's filtering the ressource based on a subresource property, the problem here is when you also want to filter the subresources themself.
I think this would be a great feature, but I can image this is pretty difficult to implement.
Meanwhile, maybe we should add some explanation to the docs, since there are many questions about this behaviour:
It's actually easier to do since api-platform 2.6.4 and this fix: https://github.com/api-platform/core/pull/3525
For instance, on our side, we built a QueryBuilderHelper class we use to build the joins necessary to filter at the subresource level.
<?php
namespace App\StaticHelpers;
use ApiPlatform\Core\Bridge\Doctrine\Orm\Util\QueryNameGeneratorInterface;
use Doctrine\ORM\Query\Expr\Join;
use Doctrine\ORM\QueryBuilder;
class QueryBuilderHelper
{
public static function addJoinToRootEntity(
QueryBuilder $queryBuilder,
QueryNameGeneratorInterface $queryNameGenerator,
string $association,
string $condition,
array $parameters = [],
string $joinType = Join::LEFT_JOIN
): void {
$joinMethod = $joinType === Join::LEFT_JOIN ? 'leftJoin' : 'innerJoin';
$associationAsArray = explode('.', $association);
if (count($associationAsArray) !== 2) {
throw new \LengthException('In depth joins are not supported yet. Failing association: ' . $association);
}
$rootAlias = self::findRootAlias($associationAsArray[0], $queryBuilder);
if (!$rootAlias) {
throw new \LogicException('Join parent should be a root entity. Failing association: ' . $association);
}
$joinedEntity = $associationAsArray[1];
$joinAlias = $queryNameGenerator->generateJoinAlias($joinedEntity);
$queryBuilder->{$joinMethod}(
$rootAlias . '.' . $joinedEntity,
$joinAlias,
Join::WITH,
str_replace($joinedEntity . '.', $joinAlias . '.', $condition)
);
foreach ($parameters as $parameterName => $parameterValue) {
$queryBuilder->setParameter($parameterName, $parameterValue);
}
}
public static function addJoinsToRootEntity(
QueryBuilder $queryBuilder,
QueryNameGeneratorInterface $queryNameGenerator,
array $associationConditionMapping,
array $parameters,
string $joinType = Join::LEFT_JOIN
): void {
foreach ($associationConditionMapping as $association => $condition) {
self::addJoinToRootEntity(
$queryBuilder,
$queryNameGenerator,
$association,
$condition,
$parameters,
$joinType
);
}
}
public static function findRootAlias(string $entity, QueryBuilder $queryBuilder): ?string
{
$aliasesMap = array_combine($queryBuilder->getRootEntities(), $queryBuilder->getRootAliases());
$entityNameSpace = 'App\\Entity\\' . ucfirst($entity);
return array_key_exists($entityNameSpace, $aliasesMap) ? $aliasesMap['App\\Entity\\' . ucfirst($entity)] : null;
}
}
And, on the filter extension side:
<?php
declare(strict_types=1);
namespace App\Doctrine;
use ApiPlatform\Core\Bridge\Doctrine\Orm\Extension\ContextAwareQueryCollectionExtensionInterface;
use ApiPlatform\Core\Bridge\Doctrine\Orm\Util\QueryNameGeneratorInterface;
use App\StaticHelpers\QueryBuilderHelper;
use Doctrine\ORM\QueryBuilder;
class FooCollectionFilterExtension implements
ContextAwareQueryCollectionExtensionInterface
{
/**
* @inheritDoc
*/
public function applyToCollection(
QueryBuilder $queryBuilder,
QueryNameGeneratorInterface $queryNameGenerator,
string $resourceClass,
string $operationName = null,
array $context = []
): void {
if ($resourceClass !== Foo::class) {
return;
}
QueryBuilderHelper::addJoinToRootEntity(
$queryBuilder,
$queryNameGenerator,
'foo.bar',
'bar.someProperty = :someValue',
['someValue' => 'value']
);
}
}
@MariusJam Based on your solution, would it be possible to specify how to add the filters to the openAPI doc !
@MariusJam Based on your solution, would it be possible to specify how to add the filters in the openAPI doc !
Yes please!
to filter at the subr
Do you have an example about a URL filtering or did I miss something?
Thank you.
Hi! In Sylius, we've encountered a similar problem on a custom filter that sorts resources by nested and filtered resource. The case is we have to sort products by their first variant's price but excluding disabled variants (ProductPriceOrderFilter). Because the FilterEagerLoadingExtension
clones base query and wraps it, we lose what we filtered out in the subquery. So we disabled force_eager
in this particular operation. However, FilterEagerLoadingExtension
was applied anyway, and it is because we have eager loading on some associations. Therefore we decided to decorate this extension to perform early return in specific cases, but it's not a perfect solution for sure.
Anyway, I'm thinking about defining another extension like FilterExtension
with e.g. -18 priority to apply certain filteres after the FilterEagerLoadingExtension
was called. I think it may be another fine workaround :thinking:
I have a task to filter entities by their subresources. Original Api Platform filters (SearchFilter with exact strategy) work as follow: they are filter a main entity by the value of the subresource property, and then push a list of the main entity ids to the IN condition of a main doctrine query. This behaviour produce logic: you will get list of filtered main entities with all their subresources. And this is a logically right behaviour (we filter the MAIN entity by the SUBRESOURCE property).
However, seems to be logically right this filter behaviour produce a huge problem (especially, for collection of entities with several nested subresource levels): we show list of unfiltered subresources and to show them in a filtered state we need to give up an ability to retrieve subresources with one request and make additional requests for all of them. In our case it could lead to tens or even thousands requests in the case where we could be satisfied with a single one.
I've also found a class, that seems to create this behaviour and linked Github issues for it and my case. See class
https://github.com/api-platform/core/blob/master/src/Bridge/Doctrine/Orm/Extension/FilterEagerLoadingExtension.php
and issues:
https://github.com/api-platform/core/issues/944 https://github.com/api-platform/api-platform/issues/424
I think that filtering of subresources should be an optional ability for Api Platform filters. It will be a really performance friendly feature.