search

hejiheji001 / Web-Scraper-Plus

Web data extraction tool implemented as chrome extension with much more features
45 stars 18 forks source link

Data have not been saved to my MySQL database #6

Open ibraheemhsn opened 5 years ago

ibraheemhsn commented 5 years ago

Hi Data have not been saved to my MySQL database. I have configured ArrestDB and Database correctly. and added the correct storage details in the plugin options page, and then I run the following sitemap, but always nothing saved to data column in my database. I checked my table in MySQL database and I found that (id, sitemapid, content) have been saved correctly, but nothing saved in data column.

as well when I click (export data to CSV), it export empty csv file (only header exist). I made sure ArrestDB is working and it is already give me correct get response.

as well when I change plugin option to local storage or CouchDB , then everything saved correctly and going well !

this is the sitemap I used for testing:

{"_id":"sitemap7","startUrl":"https://github.com/hejiheji001/Web-Scraper-Plus","selectors":[{"parentSelectors":["_root"],"type":"SelectorText","multiple":true,"id":"linkssssss","selector":"span.css-truncate a.js-navigation-open","regex":"","datafilter":"","delay":""}]}

see image explaining correct get response but data field is still empty . correct get response

hejiheji001 commented 5 years ago

Hi, this error will happen when the data you're trying to save to MySQL is too long or in a wrong datatype.

Can you tell me the table structure of your mysql db?

ibraheemhsn commented 5 years ago

Hi, this is my data structure Capture3

BTW: only forth column (data) json type, is still empty after many scraping retries.

ibraheemhsn commented 5 years ago

is my db structure correct ? datatype here is JSON. also note I trying to save very little amount of data.