PromInc / organic-search-analytics

Import organic search analytics from Google Search Analytics (Google Webmaster Tools) and save to a MySQL database
36 stars 19 forks source link

Different charset for French and German keywords with apostrophes? #75

Open marowi85 opened 6 years ago

marowi85 commented 6 years ago

Hi Brian,

your tool is really great and I have only one question regarding French and German keywords with apostrophes. Currently, they were imported with special characters like "épilateurs électriques" instead of "épilateurs électriques".

I could imagine it has something to do with the UTF-8 charset and was wondering if latin1 would be better for these purposes?

Unfortunately, I have no idea where to change the charset, so that the keywords are imported correctly.

I would be very grateful for any help.

Best regards Manuel

PromInc commented 6 years ago

Manuel,

The charset is on a database table directly. This can be changed in MySQL - with a tool like MySQL Workbench, phpMyAdmin, or via command line with a direct alter query.

I believe you are correct in that changing the charset would resolve your issue but I'm not 100% positive without researching the differences in the charset options.

I'd suggest you make a backup of your current database before you make this change, but I believe this will be safe.

marowi85 commented 6 years ago

Hi Brian,

many thanks for your help.

I have changed the collation for "query" from "utf8_unicode_ci" to "latin1_general_ci" but there were still keywords with special characters imported.

If that is not so easy to change, then I will continue to correct the keywords with my set of SQL commands. That has always worked well for me.

PromInc commented 6 years ago

It could be a flaw in how the data is formatted prior to writing to the database.

If you call this URL (changing the domain references to your installation and the date as needed, you'll see all of the data captured from the Google API. This is a straight var_dump of the data returned from Google.

http://www.yourdomain.com/organic-search-analytics/data-capture-run.php?type=googleSearchAnalytics&domain=http%3A%2F%2Fwww.yourdomain.com%2F&date=2018-01-10&mode=return

The parameter mode=return outputs the data as opposed to writing it to the database.

The importGoogleSearchAnalytics function writes the data to the database. Maybe special characters need to be decoded from the Google response?

Being US based I don't have data that falls into this issue so it's a bit harder for me to track down the issue. But hopefully this helps to troubleshoot the source of the issue.

marowi85 commented 6 years ago

Hi Brian,

thanks again for your help.

The URL was very helpful. The keywords already come with special characters from Google.

Here is one example:

string(37) "meilleure brosse à dents électrique"

So do I have to decode them first before saving? If you could help me with that, I would be very grateful. Unfortunately, I do not have such good programming skills.