search

bfinlay / laravel-excel-seeder

Seed your database with Laravel using Excel and CSV files
Other
38 stars 8 forks source link

Chunked file read #1

Closed bfinlay closed 4 years ago

bfinlay commented 5 years ago

The chunk functionality that exists is legacy holdover from laravel-csv-seeder. The PhpSpreadsheet library reads the entire spreadsheet into memory at once by default. This is fine for smaller spreadsheets but may be problematic for very large CSVs generated by another process.

PhpSpreadsheet does support chunk reading. This can be implemented when/if it is a problem.

See https://phpspreadsheet.readthedocs.io/en/latest/topics/reading-files/ and search for "chunk"

bfinlay commented 4 years ago

removed chunking from 2.1.0 and closed issue.

bfinlay commented 4 years ago

The PDO driver limits parameter bindings to 65535 parameters. 

Failed to execute the SQL statement: SQLSTATE[HY000]: General error: 7 number of parameters must be between 0 and 65535

This limits the number of rows that can be inserted into a single query.

bfinlay commented 4 years ago

This may help issue #3

bfinlay commented 4 years ago

batched inserts and chunked reading added in v2.1.8