Paginator results can vary by property visibility

[truckee]

[With apologies for the duplicate of a post at SO] Describe the bug A Representative entity, mapped via joined inheritance to the User entity, in list view shows a count of 3 but displays only one. Representative is also in a ManyToOne relationship to the Nonprofit entity.

To Reproduce Unfortunately, I don't think this can be easily reproduced. It most likely requires a similar setup and data set. The issue does not appear in EA2, only in EA3.

Additional context Attached are listings copied from Symfony profiler of database queries from both EA2 & EA3. The significant difference is that EA2 queries all 3 nonprofits whereas EA3 queries only 1. There are no code differences between EA2 & EA3 for any entities.

Edit: Summary of db query sequence (staff is table name for Representative):

EA2, EA3 #1: Select properties from usertable joining mapped inheritances for id = 1 (admin)

EA2, EA3 #2: Select (EA2: properties; EA3: id) from staff  join usertable limit 15

EA2, EA3 #3:  Select properties from staff  join usertable for ids in array found in query #2.

EA2 #4: select properties from nonprofit join opportunities for id = 1
EA3 #4: select count of staff id join usertable

EA2 #5: select focuses for nonprofit id = 1
EA3 #5: select properties from nonprofit join opportunities for id = 3

EA2 #6: select properties from nonprofit join opportunities for id = 2
EA3 #6: select focuses for nonprofit id = 3

EA2 #7: select focuses for nonprofit id = 2

EA2  #8: select properties from nonprofit join opportunities for id = 3

EA2  #9: select focuses for nonprofit id = 3

EA2 #10: select count of staff id join usertable

EA2-rep_list_view.txt EA3-rep_list_view.txt

Snippets from the relevant entities and the Representative CRUD controller. Representative entity (snippet):

/**
 * @ORM\Table(name="staff")
 * @ORM\Entity
 */
class Representative extends User
{
...
    /**
     * @ORM\ManyToOne(targetEntity="Nonprofit", inversedBy="reps", cascade={"persist", "remove"})
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="orgId", referencedColumnName="id")
     * })
     */
    protected $nonprofit;
...
}

User entity (snippet):

/**
 * @ORM\Table(name="usertable")
 * @ORM\Entity
 * @ORM\Entity(repositoryClass="App\Repository\UserRepository")
 * @ORM\InheritanceType("JOINED")
 * @ORM\DiscriminatorColumn(name="type", type="string")
 * @ORM\DiscriminatorMap({"rep" = "Representative", "volunteer" = "Volunteer", "admin" = "Admin"})
 */
abstract class User implements UserInterface
{
...
}

Nonprofit entity (snippet):

class Nonprofit
{
...
    /**
     * @ORM\OneToMany(targetEntity="App\Entity\Representative", mappedBy="nonprofit", cascade={"persist", "remove"})
     */
    protected $reps;
...
}

Representative CRUD Controller:

class RepresentativeCrudController extends AbstractCrudController
{
    public static function getEntityFqcn(): string
    {
        return Representative::class;
    }

    public function configureCrud(Crud $crud): Crud
    {
        return $crud
            ->setPageTitle(Crud::PAGE_EDIT, 'Edit %entity_name%')
            ->setHelp('index', 'Locking staff deactivates nonprofit and blocks staff log in. Replacing also removes current staff.')
            ->setSearchFields(['id', 'roles', 'email', 'fname', 'sname', 'confirmationToken', 'replacementStatus']);
    }

    public function configureActions(Actions $actions): Actions
    {
        return $actions
            ->disable('new', 'edit', 'delete');
    }

    public function configureFields(string $pageName): iterable
    {
        $roles = ArrayField::new('roles');
        $password = TextField::new('password');
        $email = TextField::new('email');
        $fname = TextField::new('fname');
        $sname = TextField::new('sname');
        $lastLogin = DateTimeField::new('lastLogin');
        $confirmationToken = TextField::new('confirmationToken');
        $tokenExpiresAt = DateTimeField::new('tokenExpiresAt');
        $locked = BooleanField::new('locked');
        $enabled = BooleanField::new('enabled');
        $replacementStatus = TextField::new('replacementStatus');
        $initiated = DateField::new('initiated');
        $completed = DateField::new('completed');
        $nonprofit = AssociationField::new('nonprofit');
        $id = IntegerField::new('id', 'ID');
        $replace = TextareaField::new('replace')->setTemplatePath('Admin/replace_staff.html.twig');
        $fullName = TextareaField::new('fullName');
        $orgname = TextareaField::new('orgname');

        if (Crud::PAGE_INDEX === $pageName) {
            return [$replace, $fullName, $email, $orgname];
        } elseif (Crud::PAGE_DETAIL === $pageName) {
            return [$id, $roles, $password, $email, $fname, $sname, $lastLogin, $confirmationToken, $tokenExpiresAt, $locked, $enabled, $replacementStatus, $initiated, $completed, $nonprofit];
        } elseif (Crud::PAGE_NEW === $pageName) {
            return [$roles, $password, $email, $fname, $sname, $lastLogin, $confirmationToken, $tokenExpiresAt, $locked, $enabled, $replacementStatus, $initiated, $completed, $nonprofit];
        } elseif (Crud::PAGE_EDIT === $pageName) {
            return [$roles, $password, $email, $fname, $sname, $lastLogin, $confirmationToken, $tokenExpiresAt, $locked, $enabled, $replacementStatus, $initiated, $completed, $nonprofit];
        }
    }
}

[javiereguiluz]

I'm afraid I can't help here because I don't understand why this happens. Please, try setting any of the config options related to pagination (see https://symfony.com/doc/master/bundles/EasyAdminBundle/crud.html#search-and-pagination-options).

[truckee]

Options tried: ->setPaginatorUseOutputWalkers(true) and ->setPaginatorFetchJoinCollection(true)

Unfortunately, trying either or both of the paginations options in either the Representative or Nonprofit entity did not make a difference. In all cases only one of three possible entities was displayed; the same entity was displayed in each trial.

Edit: Created a fresh Symphony 5.1 full project & added only my entities & EA3. Copied the RepresentativeCrudController from the original project into the fresh installation. The same behavior obtains, thus eliminating nearly all of the original project's code as a source of error.

[globetrotdev]

Hi ! how are you? Can you help me. I have the same issue.

[truckee]

@belkoweb It would be useful if you could edit your post to show code - entities & controllers - producing the same issue.

[4lxndr]

I'm facing the exact same problem. Using xdebug leads to the problem that the method getPrimaryKeyValue returns always null. Therefore every time the same array key is overwritten with the latest entity.

EntityFactory.php

    public function createCollection(EntityDto $entityDto, ?iterable $entityInstances): EntityCollection
    {
        $entityDtos = [];

        foreach ($entityInstances as $entityInstance) {
            $newEntityDto = $entityDto->newWithInstance($entityInstance);
            $newEntityId = $newEntityDto->getPrimaryKeyValueAsString();
            if (!$this->authorizationChecker->isGranted(Permission::EA_ACCESS_ENTITY, $newEntityDto)) {
                $newEntityDto->markAsInaccessible();
            }

            $entityDtos[$newEntityId] = $newEntityDto;
        }

        return EntityCollection::new($entityDtos);
    }

EntityDto.php -> always throws the exception Property id does not exist

    public function getPrimaryKeyValue()
    {
        if (null === $this->instance) {
            return null;
        }

        if (null !== $this->primaryKeyValue) {
            return $this->primaryKeyValue;
        }

        try {
            $r = ClassUtils::newReflectionObject($this->instance);
            $primaryKeyProperty = $r->getProperty($this->primaryKeyName);
            $primaryKeyProperty->setAccessible(true);
            $primaryKeyValue = $primaryKeyProperty->getValue($this->instance);
        } catch (\Exception $e) {
            $primaryKeyValue = null;
        }

        return $this->primaryKeyValue = $primaryKeyValue;
    }

I assume that the primary key property is private in your abstract user? Making it protected solved the issue for me.

[truckee]

@4lxndr Absolutely correct. Solves the problem so I'm closing the issue.

[javiereguiluz]

@4lxndr thanks for the details. However, why does it matter that your primary key is private? We use PHP reflection and force the primary key to be accessible. (That's why I'm reopening this issue) I'm missing something here 🤔 Also, could we do something in our code to help people debug this problem? Thanks!

[truckee]

It appears the failure occurs at line 84 in EntityDto.

        try {
            $r = ClassUtils::newReflectionObject($this->instance);
            $primaryKeyProperty = $r->getProperty($this->primaryKeyName);    // line 84
            $primaryKeyProperty->setAccessible(true);
            $primaryKeyValue = $primaryKeyProperty->getValue($this->instance);
        } catch (\Exception $e) {
            $primaryKeyValue = null;
        }

Edit: removed incorrect analysis. What is true is that when the parent User entity property id is protected not private, line 84 does not throw the Exception. The difference is that $this->instance does not contain the property id when it is private.

protected $id; instance:

private $id; instance:

[javiereguiluz]

Thanks for the additional details.

This is really strange. I'm trying to reproduce this ... so I made my $id property private in the entity and removed the getId() method. The PHP reflection code worked as expected and it could extract the right property value:

[truckee]

Here's the equivalent in my app:

[truckee]

To expand on the issue: Representative is one of four entities defined in the User entity:

/**
 * @ORM\Table(name="usertable")
 * @ORM\Entity
 * @ORM\Entity(repositoryClass="App\Repository\UserRepository")
 * @ORM\InheritanceType("JOINED")
 * @ORM\DiscriminatorColumn(name="type", type="string")
 * @ORM\DiscriminatorMap({"rep" = "Representative", "volunteer" = "Volunteer", "admin" = "Admin"})
 */
abstract class User implements UserInterface

Each of those entities presents the identical problem when $id is private. Any suggestion on how to determine if this is relevant?

Edit: I'm wondering if we've lost site of the original issue. When the id is private, one and only one entity is rendered, which is not consistent with the id not appearing in any of the Reflection classes. That single entity's id is available with a template including {{ dump(entity.instance.id) }}.

Edit 2: While using xdebug trying to find out where the divergence happens between private & public id I stumbled on an exception. Changing private to protected resulted in

You cannot refresh a user from the EntityUserProvider that does not contain an identifier. The user object has to be serialized with its own identifier mapped by Doctrine

This does not occur when changing protected to private. But it is further evidence that the user entity (in this case admin, one of the three user types) does not contain an identifier.

Edit 3:

I have identified a difference that may be significant. At AbstractCrudController:119 there is $entities = $this->get(EntityFactory::class)->createCollection($context->getEntity(), $paginator->getResults()); I serialized each of the parameters of createCollection() and output them to text files. I then ran Windows fc to compare the files between protected and private runs. The $paginator->getResults() files were significantly different. Those files are attached. Interpreting the results are not in my skill set.

protected.txt private.txt

Edit 4: A close examination of these two files shows that the id of the Representative appears in a different sequence depending on whether the id is private or protected. For example:

protected:
N;s:36:" App\Entity\Representative completed";N;s:5:" * id";i:9;s:22:" App\Entity\User roles";a:1:{i:0;s:8:"ROLE_REP";}
private:
N;s:36:" App\Entity\Representative completed";N;s:19:" App\Entity\User id";i:9;s:22:" App\Entity\User roles";a:1:{i:0;s:8:"ROLE_REP";}

With protected $id; the id appears to be associated with the Representative entity while with private $id; it appears with the User entity. Why this is true I must leave for others to determine.

[truckee]

With apologies for my ignorance. The differences between private & public id properties are expected results from Doctrine. This I learned in a test outside of EA3. The question remains, though, is why there are differences in the display of users (which I believe comes from EA3).

[nico-dangerous]

Hi, I have the exact same problem here.

I got one entity Organization and 2 sub entities ClientOrganization and ResellerOrganization.

abstract class Organization
{
    /**
     * @var int
     * @Expose
     */
    private $id;

class ClientOrganization extends Organization {

    /**
     * @var integer
     * @Expose
     */
    private $id;

With the private $id, the ClientOrganization list show only 1 line and a correct counter (3 in my case)

When I change to protected $id, it's working, I can see all the values in the list.

My setup :

symfony/framework-bundle                v5.1.8
easycorp/easyadmin-bundle               v3.1.6
doctrine/common                          3.0.2

I hope this will help you.

[truckee]

Resolved by #3967