search

opencaching / opencaching-pl

The source code of Opencaching.PL (and some other domains)
https://opencaching.pl/
GNU General Public License v3.0
22 stars 33 forks source link

Google Maps changes as of June 11th #1520

Closed andrixnet closed 5 years ago

andrixnet commented 6 years ago

I have just received this email from Google regardiung Google Maps API service:

Hi, Today we are announcing important changes, including our new name - Google Maps Platform, a simplified product structure, pay as you go pricing for all, and more. Please take a few minutes to review the announcement to familiarize yourself with the upcoming changes. We would like to highlight a few updates that may impact your implementation. Beginning June 11th, we are launching our new pricing plan and providing all users access to support. We’ll continue to offer a free tier — all developers will receive $200 of free monthly usage of our core products. In addition, this change will require you to enable billing and associate it with all of your Google Maps Platform projects. Creating a billing account helps us better understand your usage so we can continue developing helpful products. It also allows you to scale easily with less downtime and fewer performance issues if your product grows beyond the $200 of free monthly usage. For additional visibility and control you can set daily quotas or billing alerts.

How does this affect your current projects?

Project: API Project (xxxxxxxxxxxxxxxx)

Based on your project usage over the last 3 months and our new pricing plan, we estimate that your new cost will be less than $200 a month and will be covered by our $200 monthly free credit. This estimate does not include usage from other projects or keyless implementations. We want to support you along the way. Please read our Guide for Existing Users to understand how these changes may impact your account and what steps you need to take. Thank you for using Google Maps Platform.

Likely, each OC node admin (on whose account the Google Maps API key is registered) received a similar notification.

Beginning on June 11, 2018, you’ll need to enable billing with a credit card and have a valid API key for all projects.

The bad news is that this change requires billing information to be entered by the API key owner and likely Google will automatically bill any excess usage.

Also, in June 2016 we announced that we would stop supporting keyless usage, meaning any request that doesn’t include an API key or Client ID. This will go into effect on June 11th, and keyless access will no longer be supported. At that time, keyless calls to the Maps JavaScript API and Street View API will return low-resolution maps watermarked with “for development purposes only.” Keyless calls to any of the following APIs will return an error: Maps Static API (including Static Street View), Directions API, Distance Matrix API, Geocoding API, Geolocation API, Places API, Roads API, and Time Zone API.

This is a point when alternatives such as OpenStreetMap API and also alternate data sources may become of significant importance.

Possibly affected pages / sections:

andrixnet commented 6 years ago

On the pricing page there is an info that some level of control is allowed the user such as:

$200 free monthly usage For most of our users, the $200 monthly credit is enough to support their needs. To protect against unexpected increases, you can also set daily quotas, maximum daily billable limits, or maximum daily usage limits.

andrixnet commented 6 years ago

OpenStreetMap API: https://wiki.openstreetmap.org/wiki/API

rapotek commented 6 years ago

It is unclear to me what in fact is charged: the whole API usability or the map layer ("maps tiles") download. In case the whole API usability is charged, maybe we should think about moving to OSM overpass API and Leaflet JS library. I am not an expert in the matter so I cannot estimate the effort needed to replace the resources like cachemap or MyNbh map with the new solution.

kojoty commented 6 years ago

@rapotek G. wants to charge us for every load of MAPS API - dynamic map will costs 0,007$ per view (if you open page with dynamic map this is "one view" - next actions like scroling, zooming, maps switching are for "free").

@andrixnet for OCPL this is critical issue because it seems that we will have to pay several hundred $ monthly.

My first idea is to write to G that their new price politic is a significant change for our service, that it is not nice that after years they decided to start charging so much for maps, (announcing it one month before the change!) and we are really sed that we need to stop using their API - but their do not give us a choice. Maybe it helps something - but I don't have a good feelings.

The question is what alternative we have now - this is really important to choose well because this is significant workload to switch such API in future.

(For me this issue is a highest priority right now.)

andrixnet commented 6 years ago

I seriously doubt such a letter would have any effect.

OSM API is one choice. (I remember I played a bit with an older version some years ago, but only basic). There are others, as well.

Of course, the biggest question is: can a new API be implemented within the given timeframe?

kojoty commented 6 years ago

I seriously doubt such a letter would have any effect.

Sure, but let's try to scrounge something :)

Of course, the biggest question is: can a new API be implemented within the given timeframe?

hmm, this is a case of main map of caches (aka v3) - if we switch it to other API usage of G. API should decreased to "free limit".

Additionally we can temporary disable map in "my neighbourhood" - or make it accessible "on demand" - after user click it - then for sure we will be in a free limit - next maps we can migrate in future...

andrixnet commented 6 years ago

There are also the maps and routing features under My Routes. Not sure about GeoPaths ...

andrixnet commented 6 years ago

And a very important detail: for Google API to continue to work, you MUST add billing and credit card information into your Google account ...

rapotek commented 6 years ago

@kojoty I just made a simple test in MyNeighbourhood and you are right :worried: . Even If you set mapTypeName to OSM in DynamicMapModel and disable loading native maps in dynamicMap.tpl, the Maps JavaScript API loads still increases on reload. I wonder if a change to dynamic refresh data in MyNbh map and sections would decrease the loads counter? But the effort to check seems to be not worth the result while we search for alternatives anyway.

harrieklomp commented 6 years ago

I am not a expert but could Bing maps be a solution?

kojoty commented 6 years ago

@harrieklomp, this is one of possible alternative - but we need to take a look at all options to choose solution which will fit for our needs.

kojoty commented 6 years ago

possible alternatives list (I will edit this list):

Rather not an option for OC:

rapotek commented 6 years ago

I think what we should focus on is support for already used map layers by OC nodes and possibilities/solutions for dynamic updates of markers etc.

andrixnet commented 6 years ago

I agree with @rapotek about bing API. It is another corporation controlled API.

One part is the API features (same or equivalent to those) used by OC nodes - function for users. Another part is licensing. Definitely the open-source APIs appeal the most. Last but not least, is the datasources. Both from API support perspective and usage licensing. (otherwise said, it would be great to use an open-source API, but if displaying satelite layer requires payment brings us back to where we started)

andrixnet commented 6 years ago

Interesting to observe: Groundspeak itself implements maps with Leaflet: https://www.geocaching.com/about/maps.aspx

kojoty commented 6 years ago

yeah, it seems that leaflet is the most used free map api now - I think we need to use it - I'm going to research at this this evening.

rapotek commented 6 years ago

I just noticed: while we use only tiles from OSM, not the whole data set (paths etc), introducing overpass API is unnecessary - it was my mistake. The leaflet itself should be enough.

One more thing: in leaflet the WMS to image translation seems to be included (L.TileLayer.WMS), the function and configuration options we use in the code for this purpose can be omitted for leaflet, I suppose.

kojoty commented 6 years ago

I think leaflet can ba a good option for us at all - it takes some time but we can get many improvements.

I'm thinking also about resignation from OKAPI way for map overlay generation - maybe we can do it without static tiles - but I don't know now - it also needs some perf. tests.

Maybe it is a good occasion to write new OC map from scratch...

rapotek commented 6 years ago

@kojoty Have we enough time to write everything from scratch? I think we should concentrate on dynamic map for now, because it is an easier part for refactoring. There can be the visual inconsistency between cachemaps and dynamic maps f.ex. in MyNbh but the google API load increment should slow down then and give us time to modify the more difficult parts like OKAPI.

kojoty commented 6 years ago

@rapotek (I answer ni Polish, because this is side discussion)

To jest najważniejsze pytanie - oczywiście ze skupiamy sie na mapie dynamicznej i w tej sytuacji (nie jestem pewien na 100%) ale chyba trzeba będzie napisać ten kod na nowo (w sensie sam skrypt wyświetlający mapę - to co dziś nazywa się cachemap*).

Oczywiście, że nie możemy robić wszystkiego od razu - ale zobaczymy jak to będzie wyglądać w praktyce.

Myślałem tylko, żeby tak pisać ta nową mapę, żeby ewentualna zmiana podejścia w kwestii tego jak na mapę nakładamy informacje o keszach mogła być w miarę łatwo zmieniona.

Ja mam świadomość naszych (zwłaszcza własnych) ograniczeń. :)

rapotek commented 6 years ago

@kojoty ja jestem jak najbardziej za tym, żeby napisać to od nowa, tylko trzeba to przemyśleć na spokojnie. Nie orientuję się do końca, ale OKAPI chyba keszuje sobie całe obrazki map z nałożonymi na nie skrzynkami, zgadza się? Jeżeli tak, to np. dla OSM jest to nie do końca legalne i np. to trzeba będzie zmienić.

Zgadzam się z testami wydajności. Trzeba będzie sprawdzić czy przeniesienie wszystkiego na stronę klienta (javascript) nie spowoduje efektu narzekania przez użytkowników, że mapy im działają bardzo powoli.

Można spróbować wydzielić jakieś warstwy abstrakcji w ramach mapy (częściowo to już jest), które będą miały wspólne elementy dla różnych podejść do nakładania informacji o keszach, natomiast różnice będą implementowane już w osobnych klasach/szablonach. Innego sposobu na razie nie widzę.

kojoty commented 6 years ago

Nie orientuję się do końca, ale OKAPI chyba keszuje sobie całe obrazki map z nałożonymi na nie skrzynkami, zgadza się? Jeżeli tak, to np. dla OSM jest to nie do końca legalne i np. to trzeba będzie zmienić.

Nie, OKAPI keszuje dane o lokalizacji wszystkich skrzynek tak by miec szybkie powiazanie nr-kafla - lista keszy z kafla, do tego keszuje chyba same wygenerowane kafle-obrazki, które są jak przezroczysta folia, na której są tylko ikonki keszy - one sa warstwą nakładaną na kafle z właściwą mapą - nie ważne jaką OSM czy Google czy topo.

Dalej poniewaz kafle OKAPi to obrazki wiec nie sa klikalne wiec uzyskania jakiejs informacji o keszu wymaga obsługi klika na mapie i serwisu zwracającego info o tym jaki kesz jest pod kliknietymi kordami :)

Do tego dochodzi obsługa "cache-setów" - czyli wyników wyszukiwań keszy prezentowanych na mapie - to sa w ogole dość skomplikowane tematy - ale wydaje mi się, że można by to zrobić "lepiej" - ja kiedyś juz gadałem o tym z wryglem i on się tez zgadzał, że można inaczej - ale na razie były zawsze pilniejsze sprawy.

Myślę, że w pierwszej kolejności musimy odtworzyć "stan obecny" ale warto by pomysleć, jaki byłby nasz model docelowy...

Na razie popatrzmy dokładnie na ten leaflet i opcje jakie dla nas niesie - ja im dłużej patrze tym jestem bardziej optymistą :P

andrixnet commented 6 years ago

Is Leaflet adequate for "My routes" module as well?

andrixnet commented 6 years ago

Note: Cachemap v3 has (IMO) an excellent feature: it shows in top-left corner the current coordinates (under cursor) in DD MM.mmm format. It would be nice to keep this functionality with the next API.

rapotek commented 6 years ago

Note: Cachemap v3 has (IMO) an excellent feature: it shows in top-left corner the current coordinates (under cursor) in DD MM.mmm format. It would be nice to keep this functionality with the next API.

There are plugins for leaflet which provides similar functionalities. If not, we can write our own plugin, it seems to be not so hard to do.

Update: I was playing with it a little bit, just out of curiosity. The effect is in screenshot from working example, with compass image and inner style copied from our cachemap3 display. coordinates

deg-pl commented 6 years ago

IMHO Openlayers is a better solution than leafletjs. Please remember about new MyNeighbourhood configuration mode - we cannot disable maps here. New API should support to draw and edit transparent circles on the map (Openlayer does).

kojoty commented 6 years ago

@andrixnet I think that we are able to get full previous functionality without Google api maps.

@deg Sure there is no problem in drawing on the map.

kojoty commented 6 years ago

I made some experiments with leaflet and it seems to work great.

Now I try to prepare some experiment with openlayers - i would like to compare them - especially I'm interesting if there is a difference on how speed of both solutions in our use-case.

if it gets any result I'll share my code on branch.

Moris299 commented 6 years ago

I tried to randomly put 25k caches into leaflet map - it lagging on my mobile phone. What if I want to see map of whole Polish OC with 34 834 caches? my experiment with 34834 caches:

http://moris.pw/opencaching/map/test/map.php?c=34834

(GET value editable, it will change caches count)

kojoty commented 6 years ago

Hi Moris299,

I tried to randomly put 25k caches into leaflet map - it lagging on my mobile phone. What if I want to see map of whole Polish OC with 34 834 caches?

I think that at least at the beginning we stay with additional layer generated by OKAPI - same as before - then we don't need to put all caches on the map at once.

BTW. map can also present all caches - included archived and in-service - what gets 50k+ markers, and markers have different styles so it probably also affects perf.

kojoty commented 6 years ago

I made very simple test here https://github.com/opencaching/opencaching-pl/pull/1523 where you can see openlayers and leaflet in action.

Both are quite simple, both works great. Leaflet is a little bit faster in this but at my env. openlayers seems to scroll better (more smoothly).

I don't know which one to choose.

andrixnet commented 6 years ago

IMO the grouping done by leaflet in @Moris299 example isn't such a good idea

kojoty commented 6 years ago

@andrixnet, I think we will use current solution where cache icons are generated as a transparent overlay for map tiles

andrixnet commented 6 years ago

Still TODO:

kojoty commented 6 years ago

Still TODO:

map in My Routes map in GeoPaths

sure, I'm working on this - list is a little longer - all "dynamic" maps except main map are in state "todo" now

rapotek commented 5 years ago

My routes works on google.maps.DirectionsService which should be replaced too. What about openroute service? I've checked it out a little bit and it seems to work ok.

kojoty commented 5 years ago

@rapotek - If you manage to update MyRoutes you will be my hero of the month! Even better if you would like to refactor it completely - this code is really hard to maintain...

rapotek commented 5 years ago

@kojoty I will have to analize it thoroughly first, then decide if I am able to get into it.

kojoty commented 5 years ago

I think we can close this issue - Google Maps are removed from OC (almost - geocoder and MyRoutes stil need refactor. but this is another issue...)