simonhamp / laravel-nova-csv-import

The best CSV import component for Laravel Nova
https://novapackages.com/packages/simonhamp/laravel-nova-csv-import
MIT License
168 stars 76 forks source link

There were problems importing some of your data #10

Closed patricktsg closed 4 years ago

patricktsg commented 5 years ago

Getting this error: There were problems importing some of your data

Can't see any further details on my debug bar.

The CSV file is a direct dump from a table in the database.

simonhamp commented 5 years ago

@patricktsg Is there any chance that you can send a copy of the file you're trying to import?

patricktsg commented 5 years ago

Ahhhh, think I have some broken CSV data - I didn't see it previously but cspecial characters and html tags were busting the columns.

Any suggestions for importing cols with a lot of HTML tags?

barnomics commented 5 years ago

I'm having the same issue. My csv is super simple so I'm surprised it would be having any issues.

Does this tool print out an error log somewhere?

danielrhodeswarp commented 5 years ago

Having this issue on a super-duper simple CSV file which is surprising. I am sure that the same format file did indeed work previously as well... Would be good to get some insight on this issue...

danielrhodeswarp commented 5 years ago

I couldn't immediately find anything in logs (Laravel, PHP or web server) so I had to use the new Laravel dump-server and dump() out the $this->importer->failures() and $this->importer->errors() in the vendor/.../ImportController.php

Surely some easier way? Like a "verbosity" config setting for the package?

simonhamp commented 5 years ago

Thanks for reporting issues folks. Please note that this package is still pre-release (v0.1 only) so there are bound to be issues.

If you can possibly share the data files that you're experiencing issues with @barnomics and @danielrhodeswarp then I can investigate the problems and potentially come up with a fix for everyone.

Also, if you're keen to dig and support the ongoing development of the package, you're quite welcome to send in Pull Requests :) I don't have an awful lot of time to spend on developing this package for free.

barnomics commented 5 years ago

I actually dug into your code to log what was wrong and figured it out. My database has a constraint which was not included in my CSV ... so MYSQL was complaining about that (but it was not apparent to me)

For example my structure looks like ... name | value | contact_id (foreign key)

and my CSV looks like

name,value
joe,500
tom,1000

Possible improvements:

  1. Better error reporting.
  2. Ability to merge data with existing Resources instead of only adding new ones.

On the plus side, by digging into the code I discovered laravel-excel, which I was able to use to do exactly what I needed (quite easily I might add...)

Thank you!

simonhamp commented 5 years ago

Good sleuthing @barnomics. At the moment, this package will only bump into your Nova validation, so if your database has rules that are not encoded well at the application level then you're going to hit issues.

I agree that better error reporting is a must and that would definitely be necessary for v1.

Merging data is going to be trickier and require users to have a deeper understanding of their data, an enhanced user interface and a fully rounded import process.

simonhamp commented 5 years ago

This is effectively part of/a duplicate of #2

danielrhodeswarp commented 5 years ago

It's sometimes maatwebsite/excel (especially the DefaultValueBinder) doing weird things with the reading of the CSV file. Which then obviously causes issues when trying to import into the Eloquent backend data storage.