search

bfinlay / laravel-excel-seeder

Seed your database with Laravel using Excel and CSV files
Other
38 stars 8 forks source link

Performance metrics #24

Closed Shujee closed 10 months ago

Shujee commented 10 months ago

What kind of performance should I expect from the package? I'm using it with an Excel workbook containing 3 importable sheets with [171 rows/7 cols], [266 rows/3 cols] and [56 rows/3 cols] respectively and it takes about 15 seconds to completed, which seems a lot for that amount of data. Or is it normal?

bfinlay commented 10 months ago

In issue #3 you can see that it took about 1.15 minutes to seed 25k rows of data on my old machine.

On my current machine, it takes about 47s to seed 25k rows of data at steady state.

image

However the initial file read is slower.

I believe the speed limitation is the PHPSpreadsheet library.

I have some work in-progress to refactor to use Spout as the engine. It's drawback previously was that it did not support calculating formulas.

In your case, 15 seconds for that small number of rows does seem slow by comparison.

These results are also against an in-memory sqlite database.

bfinlay commented 10 months ago

In the latest github actions test runs, you can see the performance on the test spreadsheet:

image

In this test workbook there are 8 sheets. some of them only have a couple hundred rows, one has 3,000 rows. The entire book takes only 3.5s to load into MySQL.

This confirms that 15s for your data set is too slow. There is probably something else going on in your scenario.

The screenshot above is for PHP 8.0, Laravel 8.x. You can look at other configurations of PHP and Laravel versions in the github actions test results: https://github.com/bfinlay/laravel-excel-seeder/actions

I am going to close this issue, but feel free to comment if you think there is an issue in the software package.