YetiForceCompany / YetiForceCRM

Our team created for you one of the most innovative CRM systems that supports mainly business processes and allows for customization according to your needs. Be ahead of your competition and implement YetiForce!
https://yetiforce.com
Other
1.74k stars 750 forks source link

Geocoding #7693

Closed apruszenski closed 6 years ago

apruszenski commented 6 years ago

I am trying to install google Geocoding for mapping and auto addresses. I add the API in settings but nothing happens. Do I need to do anything else?

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/62274520-geocoding?utm_campaign=plugin&utm_content=tracker%2F9760560&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F9760560&utm_medium=issues&utm_source=github).
mariuszkrzaczkowski commented 6 years ago

2018-08-13_12-24-39

apruszenski commented 6 years ago

Goal: Have Maps and Auto address working.

Action: I installed GeoCoding Codes from both Google and OpenCage on the Software Configuration/ Integration / Maps {Google & OSM} / Settings page

global configuration

Expected result: When I went to the Leads page or Account page and clicked on map, I expected to see the location of a lead on the MAP or be able to do auto addressing

Actual result: Only a global MAP is shown not pinpointing to the map

Software: Yetiforce CRM v4.4.0 What other settings to I need to allow this to work?

PercyP commented 6 years ago

Same problem for me with OpenStreetMaps

ldgbc commented 6 years ago

Is your CRON working? That how they pull the Coordinate for maps.

If you can see the search address when you contact/accounts then it mean it partial working. Just manually run your CRON and you should get map pinpoint.

The weakness of OpenCage is it current inaccurate for our country. I email them asking to fix up the city and postcode but haven't heard anything from them.

PercyP commented 6 years ago

@ldgbc

thanks for that. I ran cron a few times but it still doesn't display for me. Do you know if there are any additional settings we need to set up?

ldgbc commented 6 years ago

Here is how I set up mine: 1) Get a API key from OpenCage 2) Go to setting, add OpenCage and API. Save 3) Now when I edit Contact/Account/AnyModuleThatHaveAddress I see this: image 4) I can type in the address to search or manually type it in each box. 5) I ran CRON (all of them), but only important one is: 20 and 22 (not sure if 22 is important) image

6) Now I go into map and it should work: image

As you can see, it is not very accurate maybe because I used the Street Numbers in Street (that the convention our country use) and it also partly OpenCage inaccurate data. The pinpoint should be in Melbourne.

And the default zoom still can't be change yet, so by default you looking at the whole State or country depending on how big/small your country is.

PercyP commented 6 years ago

@ldgbc I was missing the API key and now the search box appears.

I agree, there are major problems with the addresses, we use house and street numbers in the street address field, but even if I put the house number in the building number it doesn't seem to find it. Maybe it can only find business addresses and public buildings? It doesn't seem to do residential addresses - at least not for me.

If I searched for Perth Arena Western Australia it shows on the map, if I search for a home address it doesn't.

Is it the same with the Google Geocoder? Have you tried it?

ldgbc commented 6 years ago

@PercyP Hi PercyP,

I have been talking with OpenCage developer to fix up the postcode and city for my country and he said he will look into it. The result I get are close within range (postcode was usually 2 digit incorrect). The developer believe that his data is accurate and have yet to verify my result with the database and the source I provide is considered not as trust worthy. OpenCage used data source provided by Government and OpenStreetMap, if it from Government it should be accurate, it just that I think OpenCage database copy/paste a few row wrong so the Postcode and City is mixed by a few row (hence the 2 digits incorrect).

I haven't use the search since it is inaccurate but Yetiforce have yet to allow me to remap the information (see issue: https://github.com/YetiForceCompany/YetiForceCRM/issues/7290 ). Rather than use Building Numbers, Office Number, Street Number, Street Address, we usually just (Street Address, which include all of that), Suburb STATE Postcode (and sometime city). For example it would be: 100 City Road, Melbourne VIC (Victoria) 3000 (street number) (street name) (street type), (city name) (state, shorten to abbreviation) (postcode).

Right now Yetiforce is linking City/Village (Melbourne in the above example), to "District", which our country don't really use. So that issue I tried to hardcode it so relinks, but it doesn't work which it should. Maybe it is using some cache files.

With regard to Google Geocoder, as far as I know Google want you to sign up with an account, enable the API, but also open a Trial Account which require Credit Card information. For the sake of privacy of our database, me being the admin decide not to use Google Geocoder at all and work toward OpenCage/OpenStreetMap improvement.

Last thing is, OpenStreetMap doesn't deal well with Street Numbers, it can only search street name. In major city is might work but outer area of city (residential area) OpenStreetMap don't know if the left, top, right, bottom is Street Number 0 or Street Number 1000, etc.

Good lucks.

--- Update: Initially I was thinking of creating and possible learning to build my own offline Postcode + City lookup. I have done that easily with Excel but Yetiforce ability to create Picklist is too much of a hindrance and it cause a major freeze and lag if I decide to use the "picklist relation" features.

PercyP commented 6 years ago

@ldgbc

I think we have exactly the same issues.

In UK (as in Australia) we put the full number and street details in one string, then the Town/City, County, Postcode are separate. We also require Australia format so as I say have exactly your issue.

We don't use building number etc.

Like you I tried to integrate my own lookup for address fields using picklists (somewhere on here is my thread) and like you it didn't work as just hung - loads of problems.

Yes the new Google charging has impacted on every business that uses a map on their website. What puts me off is that they as you say require your credit card, but they also don't allow you to set it to stop when you reach their daily limit. I read an article somewhere saying that competitors and unscrupulous people can exploit this by using some automation tool to continuously open your maps and could basically bankrupt your business (waking up to huge charge to your credit card) I cannot however verify if this is truly the case - but is enough to put many off. I haven't tried yet, but maybe we can drag the icon to the correct position like you can with Google maps?

Just as I feel I am progressing, I feel I am taking major leaps back :(

PercyP commented 6 years ago

I noticed too that when OpenCage search does find an address it doesn't populate the street - even though that is in there. Try:

9 Port Jackson Parade Quinns Rocks WA 6030, Australia

Then if I add the street manually it does eventually display the map

ldgbc commented 6 years ago

@PercyP Interesting discover you did. Here are my result of my testing, it seem that you if play your dice correctly you might get the right result. If only the result is combine properly then this feature would be considered functional.

In this image, I picked the first result, highlight in Red. As you can see the result fill in Street name (perfect), too bad since I picked first result there is no Number. Locality Region, Quinns Rocks, which is correct for some country but ours country it would be in City/Village. Township is good, although we rarely use them very often. We probably will change that to "Council" using language editor. (en-AU) State, is perfect. (off topic but we probably find a way to use a Shorten abbreviation forms in future, Initial). map 1

In this second image, I picked the 2nd option, highlight in red.

As you can see, the numbers get fill in this time (hurry), but it under Building Numbers, that fine I suppose, but if mapping (see issue: https://github.com/YetiForceCompany/YetiForceCRM/issues/7290) work then I wouldn't care what the defaults is. But no Street Name (consider this a bug). No Locality Region (well this got moved to City/Village for this result). No Township. City/Village: Well at least it what I wanted this time.

Overall 2nd result give you Street Number (Building Number) and City/Village Name. map 2

Also please note that Parade = Pde (short form). Like Street = St, or Road = Rd. What I'm saying is, both those address are the same location (generally).


Now to the topic of coordination / code / map (longtitude and latitude). You can't manually move the coordinate, there is also no way to quickly view and edit the numbers either. It hidden somewhere. I haven't gotten the chance find where it being stored yet.

But I recently checked some of the newer address of data we added, when I look at the Map there was no "Pin", this can only mean that during my update (and manual update), I may have broken it. So that the update contain bug that break it. As not enough information is provided by me, for the time being we consider it My Configuration that causing it.

PercyP commented 6 years ago

@ldgbc

well spotted. So the address you select in the list determines what is posted to the fields in Yetiforce. In your first selection you chose the one without the building number and it populates the street. In the second selection you make it populates the building number but not the street.

For our needs we want to be able to select the second option and it populates the street field in yetiforce as a full string that includes the building number and the street.

So I am guessing this is not an Opengraph issue, but instead a field mapping issue in Yetiforce? If there was a possibility of having a mapping option that recognised the number and street as a 3rd option it would be perfect. I wonder if one of the yetiforce team could explain the logic of how the fields are populated and whether it is feasible to do something along the lines of the above? Or at the very least when you select the option 2 it knows to split the building number from the street address and fills in all fields.

ldgbc commented 6 years ago

@PercyP Based on my logical thinking it actually both.

Yetiforce just send search result to OpenCage using the API. "Search for this address please, here the API"

OpenCage return back the result with something like this. "Here is 3 result. Result 1 is: AddressLevel1 = Port Jackson Parade AddressLevel2 = Quinns Rocks AddressLevel3 = WA AddressLevel4 = 6030 AddressLevel5 = Australia " Result 2 is: AddressLevel1 = 9 AddressLevel2 = Port Jackson Pde AddressLevel3 = WA AddressLevel4 = 6030 AddressLevel5 = Australia

Result 3 is: AddressLevel1 = AddressLevel2 = Quinns Rock AddressLevel3 = WA AddressLevel4 = 6030 AddressLevel5 = Australia

You as the user click on: Result 1. It will auto field in as coded in this file: https://github.com/YetiForceCompany/YetiForceCRM/blob/developer/app/AddressFinder/OpenCageGeocoder.php#L59

So who at faults here?

On OpenCage side, you could say, well it Yetiforce fault because Level2 shouldn't 'addresslevel5' => $row['components']['city'] ?? $row['components']['village'] ?? '', Level5 should be County.

On Yetiforce side, you could say, well OpenCage result are inconsistant, for the same address it mapping result is different. Some result have 8 level, some only have 6 level. Some level are switched.

On my side, why can't I remap it to something like this?

                        'addresslevel8' => $row['components']['house_number'] ['road'] ?? '',

It also doesn't help that the "restrict or priority my country result first" last time I tried in v4.3.

https://opencagedata.com/api

countrycode   Used only for forward geocoding. Restricts the results to the specified country or countries. The country code is a two letter code as defined by the ISO 3166-1 Alpha 2 standard. E.g. gb for the United Kingdom, fr for France, us for United States. Please note, many non-country territories have their own code, for example pr for Puerto Rico. You can specify multiple codes by supplying a comma separated list. For example countrycode=fr,bl,gf,gp,mf,mq,nc,pf,pm,re,tf,wf,yt would be all French territories. Non-two letter countrycodes are ignored.
TomaszQr commented 6 years ago

In the file /config/modules/AddressFinder.php there are two functions that are used to map data from the service to fields in the CRM system, the REMAPPING_OPENCAGE variable specifies what the default mapping is and the variable REMAPPING_OPENCAGE_FOR_COUNTRY specifies the mapping for individual countries.

Fixed via https://github.com/YetiForceCompany/YetiForceCRM/pull/8601