Scope class toArray() method takes too much time

[ehsanashar]

So I am trying to process around 15k records. Here is piece of code:

resource = new FractalCollection($this->items, $transformer);
$manager = new Manager();
$data = $manager->createData($resource)->toArray();

This toArray() method takes about 14 minutes processing 15k records. Is there anyway we can make this faster? @deefour

[patrick-radius]

What's the transformer itself actually doing? Did you try it with a dummy transformer that basically does nothing to see if the performance issue is actually from the library code?

[matthewtrask]

@ehsanashar what kind of work is the transformer doing?

[KorvinSzanto]

If I add this test to ManagerTest it completes happily so I think it's clear the performance concern is coming from your transformer:

    public function testHugeCollection(): void
    {
        $resource = new Collection(range(0, 15000), new class extends TransformerAbstract {
            public function transform($item)
            {
                return ['data' => $item];
            }
        });
        $manager = new Manager();

        set_time_limit(2);
        $time = microtime(true);
        $data = $manager->createData($resource)->toArray();
        $duration = microtime(true) - $time;

        $this->assertLessThan(0.01, $duration, "Took longer than 1/100th of a second to transform 15000 items.");
    }

On my machine it completes in 8ms with 15,000, 80ms for 150,000, etc.

[KorvinSzanto]

You can see my test run here in github actions. It's considerably slower since actions run in tiny containers but you can see it still gets through 15,000 records in 1/3rd of a second.