search

matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
https://matomo.org/
GNU General Public License v3.0
19.87k stars 2.65k forks source link

Plugin using GeoIP to accurately detect visitor countries, cities, and continent #5465

Closed mattab closed 10 years ago

mattab commented 16 years ago

GeoIp Plugin

Requirements

You need at least Piwik 0.2.33 to run the GeoIp plugin.

How to install?

How to apply the GeoIp to all your previous visits to fix your previous reports

You can apply GeoIP precise localization on your past data:

Authors

Changelog

Feedback

Please leave a comment if you have any feedback, suggestion, or bug report. Keywords: third-party-plugin

robocoder commented 13 years ago

Minor update in 0.16:

anonymous-matomo-user commented 13 years ago

There is a bug in the updateExistingVisitsWithGeoIpData() function:

$row = Piwik_FetchOne($query);
...
$count = $row['cnt'];

Piwik_FetchOne($query) returns the count direct:

$count = Piwik_FetchOne($query);
anonymous-matomo-user commented 13 years ago

Hi I started using the geoip v0.17.

I am using it on Uniserver - it works fine without the GeoIP plugin enabled. After enabling that I gte this

"500 Internal error - The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, apacherrors and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log."

[notice] EACCELERATOR(2672): PHP crashed on opline 12 of exec() at C:\Uniserver\www\piwik\libs\Zend\Db\Adapter\Pdo\Abstract.php:263

[crit] Parent: child process exited with status 3 -- Aborting.

anonymous-matomo-user commented 13 years ago

One another question - Is it possible to dump the csv into Mysql and query it from GeoIp? In that case, I guess there wont be any memory overhead or caching required - it might be a bit slower though

bavarianviking commented 13 years ago

Replying to lc1:

as you can see, loc data of Leonding is correct, but data of Vienna isn't. Has anybody an idea why? piwik is accumulating the longitude and altitude by archieving. but o don't know what i have to change...

Did you ever find out what you need to change to avoid that problem? I have the same problem, so you might have some advice ..

anonymous-matomo-user commented 13 years ago

Has anyone managed to display regionnames with 0.17 version ? e.marguin's solution doesn't work, and only thing I achieved was region codes. I can't force the codes to be converted into region names.

I feel it's stupid simple, but I'm too stupid to find the solution on my own :/

anonymous-matomo-user commented 13 years ago

I use Piwik 1.5 and latest GeoIP. Piwik website shows unknown city data but in database the location_geoip_city has the correct entrys. Country and continent data are correct - only city data are "unknown". Is there a patch to re-enable this feature?

robocoder commented 13 years ago

myam: disable eAccelerator -- it's not as stable as other opcode caches; yes, possible to query from a MySQL table but not planned for this plugin; for example, the new geolocation plugin in #1823 supports a table lookup for InfoDB's data

bavarian: I don't know where this is occurring; I'll see if it happens with the new plugin ...

controlc.de: make sure you're using the GeoLiteCity.dat (or GeoIPCity.dat); if you have mod_geoip installed, this plugin will use that instead, so make sure it's also using the city .dat file

anonymous-matomo-user commented 13 years ago

Are there plans to patch Live/Visitor.php to use the plug in? It is confusing at the moment seeing different country identification on the user log and the locations. There is a suggested post in forum http://forum.piwik.org/read.php?2,71788,page=1#msg-71788 and in the following entry.

robocoder commented 13 years ago

rafal: no because (1) GeoIP is not a core plugin, and (2) when we do move that functionality into core, it'll use the existing location_ fields.

anonymous-matomo-user commented 13 years ago

vipsoft, thanks for answering. What is the expected timeline for including a geo functionality in the core?

anonymous-matomo-user commented 13 years ago

Hallo, I would like to ask if there is a way to have the city names translated? ;)

timo-bes commented 13 years ago

In the upcoming release, there is an extension of the segmentation API which breaks the GeoIP plugin.

In Piwik_GeoIP::archiveDayAggregateVisits, udpate $labelSQL = 'log_visit.location_geoip_continent';

This will prevent the archiving process from failing.

I might add the new features soon, but maybe you want to have a look at the changes and do it yourself. If you want to see the way the new querying works, have a look at Piwik_ArchiveProcessing_Day::getSimpleDataTableFromSelect: you need to use $segment->getSelectQuery. Note that all fields have to use a table prefix.

anonymous-matomo-user commented 13 years ago

Thanks for the great plugin. Would it be possible to include displaying US/Canada regions (or all country regions) in the next release? I know this is available in the GeoLiteCity database, so why not use it. Regions would mean a lot more to me (in the USA) than cities. Thanks!

anonymous-matomo-user commented 13 years ago

This new plugin sounds promising. But I hope you are going to also keep the old browser language/country detection, maybe named as such. I personally consider that language display equally important as the IP location display.

Following scenario: I'm on a travel around the world, and have a travel blog. People accessing that blog are often people I have met on the trip, often still traveling. Now, when I see my Piwik logs, the IP location (which I currently check manually) is surely interesting, but what tells me more about a visitor is actually his browser language. If you check the IP address I am writing this from, you will see that it is Malaysian. How much do I have to do with Malaysia? Nothing. My browser language is German of Germany, which tells more. And the combination of the two IP location and browser country (i.e. the current detection) actually provides one more detail: the visitor is most likely a traveler or an expat. I can imagine website who interested in that marketing information.

You would not believe how many travelers roam the world this days. And I would say most of them use the often free WiFi (at their place of stay, bars and restaurants all over Southeast Asia) with their own devices: Laptops, Phones, Tablets, etc. It seems to be the new way of travel, with people sticking their noses into displays half of their time, with most of that time on Facebook.

P.S.: Since there are countries with several languages (Belguim, etc.), but also countries with common language (UK, US, etc.), maybe both, the browser country and its language could be shown (if provided by browser). Additionally to the IP location provided by this plugin.

anonymous-matomo-user commented 13 years ago

Replying to jawsmith:

This new plugin sounds promising. But I hope you are going to also keep the old browser language/country detection, maybe named as such. I personally consider that language display equally important as the IP location display. ...

Sorry, I apparently used wrong ticket. I copied my above comment to the ticket for inclusion of this plugin into core (#1823).

anonymous-matomo-user commented 13 years ago

Hi, first thank's for this great plugin but I've a problem with it.

When I looking for geodata of last week, month, year etc. I get the message "There is no data for this report." I checked whether the geoipUpdateRows the rows have been updated . All rows has been updated. Have any an idea why the plugin gives no information?

Thx a lot

Info. Piwik 1.6 GeoIP 0.18 (the versionnumber is not updated in plugin)

same problem with Piwik 1.5 GeoIP 0.17

anonymous-matomo-user commented 13 years ago

Hi,

I'm running the plugin since quite a while and figured out that on my hosted web space the GeoLocation very often was set to "Unknown". Today I figured out that there is a difference whether the data is added "live" or whether the data is added by running the geoipUpdateRows.php. Basically in "live" operation it about 80% of the time fails to add the data and using the geoipUpdateRows.php script it succeeds by regenerating the data. Investigating the GeoIP.php code I figured out that on my webspace function "public function logGeoIPInfo($notification)" fails to detect whether "getLocationInfoModGeoip" is supported properly. Basically the function is able to retrieve the country but most times fails to gather the other geoip data.

I could fix this by changing:

    public function logGeoIPInfo($notification)
    {
        $visitorInfo =& $notification->getNotificationObject();

        if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) )
     ...

into

    public function logGeoIPInfo($notification)
    {
        $visitorInfo =& $notification->getNotificationObject();

        if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) && !empty($_SERVER['GEOIP_CITY']) )
    ...

Maybe this could help someone who is running in the same problem.

sun

jowagner commented 13 years ago

Replying to matt:

From forum post GeoIP just show true country in "Locations & Provider" and it doesn't show true country in "Visitor Log", "world map" and "Live Visitors!"

Hi Matt, Can you please add this information in the top section "GeoIP Plugin" of the description and also add instructions for replacing respective widgets to the end of section "How to install?"? This will save people a lot of time troubleshooting. Maybe also cross-ref to #1823 to show users that integration into core is being worked on. Best regards, JJ

anonymous-matomo-user commented 13 years ago

I have a question regarding the 'Email Reports'. What if I want to include the city in the pdf reports. Is it possible to add that as an option.

Thanks!!

anonymous-matomo-user commented 13 years ago

I needed support for the MaxMind Organization database, so I have enhanced this plugin to allow for loading of multiple databases, &etc. This is based on 0.18 and tested under Piwik 1.6.

The diffs are straightforward. Apache mod_geoip support is not yet implemented, but forthcoming. I would like to get into the dev circle for this module so my enhancements may be included; I'd like feedback.

Where do I submit my changes for comments and feedback?

anonymous-matomo-user commented 13 years ago

I was going to post a patch, but this plugin in not in the piwik application repository. U would love to contribute...

anonymous-matomo-user commented 13 years ago

Hi,

I tried to install this plugin but when I want to activate the plugin in piwik, I have this error (piwik is installed via a lampp/xampp server) :

"Unable to load plugin 'GeoIP' because '/opt/lampp/htdocs/piwik/plugins/GeoIP/GeoIP.php' couldn't be found. You can manually uninstall the plugin by removing the line Plugins[] = GeoIP from the Piwik config file."

Backtrace:

0 /opt/lampp/htdocs/piwik/plugins/CorePluginsAdmin/Controller.php(28): Piwik_PluginsManager->loadPlugin('GeoIP')

1 function: Piwik_CorePluginsAdmin_Controller->index()

2 /opt/lampp/htdocs/piwik/core/FrontController.php(132): call_user_func_array(Array, Array)

3 /opt/lampp/htdocs/piwik/index.php(53): Piwik_FrontController->dispatch()

4 {main}

But I have a file /opt/lampp/htdocs/piwik/plugins/GeoIP/GeoIP.php...

gka commented 13 years ago

After installing the plugin (0.18) on Piwik (1.6) no more visitors were logged. Deactivating the plugin fixed the bug. Maybe I missed something?

anonymous-matomo-user commented 12 years ago

Replying to sun:

I could fix this by changing:

    public function logGeoIPInfo($notification)
    {
        $visitorInfo =& $notification->getNotificationObject();
    if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) )
  ...
> 
> into
> 
> 
public function logGeoIPInfo($notification)
{
    $visitorInfo =& $notification->getNotificationObject();

    if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) && !empty($_SERVER['GEOIP_CITY']) )
 ...
> 
> Maybe this could help someone who is running in the same problem.
> 
> sun

This works for me!!
Thanks !!!
anonymous-matomo-user commented 12 years ago

I could fix this by changing:

  public function logGeoIPInfo($notification)
  {
      $visitorInfo =& $notification->getNotificationObject();

      if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) )
     ...

into

  public function logGeoIPInfo($notification)
  {
      $visitorInfo =& $notification->getNotificationObject();

      if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) && !empty($_SERVER['GEOIP_CITY']) )
    ...

Maybe this could help someone who is running in the same problem.

sun

Don't work for me =( BTW: Piwik tell me that i have the versison 0.17, but i downloaded 0.18

anonymous-matomo-user commented 12 years ago

Replying to greg:

After installing the plugin (0.18) on Piwik (1.6) no more visitors were logged. Deactivating the plugin fixed the bug. Maybe I missed something? Same problem here. No logging at all since it was enabled.

Piwik plugin page says that the GeoIP plugin version is 1.17, but 0.18 was downloaded (from here) and installed. Seems that the version number was not updated (as stated in one of the comments above). (Or is the link/attachment wrong?)

  File: GeoIP.zip
CRC-32: 946fe9c0
   MD4: 097f8d10aaf2fec1a5a736eabda34445
   MD5: 071eef0ec054d7dbc9408652f43d09ad
 SHA-1: 266aee4fe6ff22a21298fd1ef45a5e4352ece920
anonymous-matomo-user commented 12 years ago

Need your help.

GeoIP is installed, and the visitor Country is shown to be correct under page Visitors -> Locations & Provider

But in Dashboard and Visitors -> Visitor Log the country flag is always the US flag.

This makes things quite confusing.

Is this by design?

I'm using Piwik 1.7.

anonymous-matomo-user commented 12 years ago

It seems this plugin is causing an error when archive.php is called in version 1.7.1

Any ideas?

anonymous-matomo-user commented 12 years ago

The last GEOip update breaks Piwik. This is the error I got Fatal error: Cannot redeclare geoip_country_code_by_name() in /home/piwik/public_html/plugins/GeoIP/libs/geoip.inc on line 347

It took some time to find this error. I changed back to an older backupped GEOip version. This solves the error.

anonymous-matomo-user commented 12 years ago

How to install GeoIP on high traffic website ? It would be nice to have a short documentation about it. High traffic documentation is something we usually find in other Piwik pages. I have 11.591.599 lines in piwik_log_visit, so the installation will take quite a long time. After pressing the "Activate" button, I setted the maintenance_mode and turned off record_statistics but it would have been nice to have a proper CLI.

anonymous-matomo-user commented 12 years ago

If you get a Fatal error: Cannot redeclare geoip_country_code_by_name() in /home/piwik/public_html/plugins/GeoIP/libs/geoip.inc on line 347 make sure geoip.so is disabled in php.ini This php setting conflicts with the function geoip_country_code_by_name

anonymous-matomo-user commented 12 years ago

When importing logs with the script provided in #703, IPs get geolocalized still the world map remains blank. Is there a way to make the GeoIP plugin work in way that is compatible with the World Map widget?

anonymous-matomo-user commented 12 years ago

I just tried to install the GeoIP plugin on Piwik 1.71 and received the following error message:

Mysqli prepare error: This command is not supported in the prepared statement protocol yet

Any one else experience this or know the solution for it?

anonymous-matomo-user commented 12 years ago

If mod_geoip is enabled and the Tracking API over HTTP with an individual IP (&cip=1.2.3.4) is used to create a new visit, then the IP-adress of the requesting server is used but not the individual IP. As a result, the GeoIP-location is always the location of the requesting server.

Workaround:

Change this

public function logGeoIPInfo($notification)
{
    $visitorInfo =& $notification->getNotificationObject();

    if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) )
...

into

public function logGeoIPInfo($notification)
{
    $visitorInfo =& $notification->getNotificationObject();

    if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) && $visitorInfo['location_ip'] == ip2long($_SERVER['REMOTE_ADDR']) )
...
timo-bes commented 12 years ago

The plugin does not use $archiveProcessing->shouldProcessReportsForPlugin() in archiveDay and archivePeriod like all core plugins. When using segmentation, this causes serious performance overhead. Could you fix this?

anonymous-matomo-user commented 12 years ago

Hi

I don't know if it's a bug in this plugin, but for me the cities in Piwik are not ordered with the correct countries. (See attached screenshot) [[Image(http://www.imagehousing.com/image/1016342)]] And I don't know where to look for the bug. - Any suggestions?

Best, MC

anonymous-matomo-user commented 12 years ago

the version number in the zip is 0.17 ( should be 0,18)

anonymous-matomo-user commented 12 years ago

The location fields do not seem to be updating, the fields are not populated I i force an update using the ./plugins/GeoIP/misc/geoipUpdateRows.php the do get populated.

Cheers Marco

anonymous-matomo-user commented 12 years ago

Setup GeoIP as instructed with Piwik 1.7.1 and mod_geoip. Visitors are logged but always with City Unknown. (See attached screenshot). [http://i.imgur.com/aiSMJ.png] Please advise, Thanks.

anonymous-matomo-user commented 12 years ago

Hi,

I followed the below procedure for enabling GeoIP for my website. 1.Downloaded GeoIP plugin, extracted it and copied in the folder piwik/plugins/ on the server. 2.Downloaded GeoLiteCity.dat file,extracted it and copied in the folder piwik/plugins/GeoIP/libs/ 3.And in the Piwik website I activated the GeoIP plugin by going to Settings-->Plugins.

Is that enough for enabling the GeoIP plugin and tracking the country?

For me,its showing that continent,country and city are "Unknown".

Is there any code to be embedded in the php page which I'm using for tracking? If so,how to make API calls from my PHP page to the GeoIP API ( API.php in GeoIP plugin )

It is saying "Requested report GeoIP.getGeoIPCountry for Website id=XXX not found in the list of available reports. "

Thanks, pol.

anonymous-matomo-user commented 12 years ago

Plugin not working on the reports page after upgrade to Piwik 1.8

For more information please follow to this forum topic [http://forum.piwik.org/read.php?2,89689] for more details.

anonymous-matomo-user commented 12 years ago

Plugin not working on the reports page after upgrade to Piwik 1.8

I'm having the same issue as pol and Globulopolis. I followed all the instructions and I'm still getting "Requested report GeoIP.getGeoIPCountry for Website id=XXX not found in the list of available reports."

Anybody working on this? Thanks, Kenegade

anonymous-matomo-user commented 12 years ago

I am having some trouble with the GeoIP after the update to Piwik 1.8.2

The total visitor count in Piwik doesn't reflect the same total count in the GeoIP plugin.

Also, is there an update to this plugin to support IPv6 now?

Thanks

robocoder commented 12 years ago

Reported in #3212: the GeoIP plugin does not support row evolution.

gka commented 12 years ago

I patched GeoIP.php to get it working with the new world map widget. The only thing I changed was to add a column for the GeoIP region code that is returned by the API but wasn't stored in the Piwik database. However the map needs this region code to fill those nice country region maps with colors :)

anonymous-matomo-user commented 12 years ago

I just installed the latest version. While it is working for new visitors, the update script geoipUpdateRows.php is not working. I tried it in the browser but there i only get a white page after a few seconds. When using a shell and using the command "php geoipUpdateRows.php" i get the following error:

PHP Parse error: syntax error, unexpected T_OBJECT_OPERATOR in ***/piwik/plugins/GeoIP/misc/geoipUpdateRows.php on line 30 Content-type: text/html

<br /> <b>Parse error</b>: syntax error, unexpected T_OBJECT_OPERATOR in <b>***/piwik/plugins/GeoIP/misc/geoipUpdateRows.php</b> on line <b>30</b><br />

Line 30 is: Piwik_FrontController::getInstance()->init();

Any suggestions?

anonymous-matomo-user commented 12 years ago

Hi! Thanks for this great plugin.

I'm having similar issues with the updated GeoIP.php file. Once updated, I tried running geoipUpdateRows.php through my browser and got the following message:

Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[42S22]: Column not found: 1054 Unknown column 'location_geoip_region' in 'field list'' in /home/samtqcca/public_html/stats/libs/Zend/Db/Statement/Pdo.php:228 Stack trace: #0 /home/samtqcca/public_html/stats/libs/Zend/Db/Statement/Pdo.php(228): PDOStatement->execute(Array) #1 /home/samtqcca/public_html/stats/libs/Zend/Db/Statement.php(300): Zend_Db_Statement_Pdo->_execute(Array) #2 /home/samtqcca/public_html/stats/libs/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array) #3 /home/samtqcca/public_html/stats/libs/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('???????UPDATE p...', Array) #4 /home/samtqcca/public_html/stats/core/Db/Adapter/Pdo/Mysql.php(220): Zend_Db_Adapter_Pdo_Abstract->query('???????UPDATE p...', Array) #5 /home/samtqcca/public_html/stats/core/PluginsFunctions/Sql.php(68): Piwik_Db_Adapter_Pdo_Mysql->query('???????UPDATE p...', Array) #6 /home/samtqcca/public_html/stats/core/PluginsFunctions/Sql.ph in /home/samtqcca/public_html/stats/libs/Zend/Db/Statement/Pdo.php on line 234

I ran it before updating the GeoIP.php file and all past visits were updated to the correct regions / cities, but the world map wasn't updated. That's why I updated the GeoIP.php file and tried to run the geoipUpdateRows.php script again.

Any ideas?

anonymous-matomo-user commented 12 years ago

Replying to cgray@:

I was going to post a patch, but this plugin in not in the piwik application repository. U would love to contribute...

Was this ever posted?

Or has the suggested update to the provider plugin that checks the database for organization identity from IP prior to using the gethostbyaddr() method been implemented?

anonymous-matomo-user commented 12 years ago

GeoIP.zip v0.18 is a corrupted zip file