Showing State/Province of Location?

[chibongho]

Is there a way for the app to show the State/Province information when displaying location? When I look up the weather for Portland, it's showing all 4 results as "Portland, US". See Screenshot.

If it's not a feature yet, would you consider adding an option to show that info under settings? I'd be happy to take a stab at implementing it.

[robinpaulson]

Are you not getting a map with the list of locations? That will help most people.

That said, I think this is worthwhile, go for it. Do ask if you have questions about the code; I may not be able to answer, but there are others who can.

[robinpaulson]

I wonder if OWM provides this information directly, or if you'll have to use something like the OSM nominatim service to figure it out? It could get messy, form what I recall OSM has 8 possible levels of administrative areas for any given location. I assume nowhere uses all 8, but some may well use 4 or 5, how will you select which information to display?

[robinpaulson]

https://wiki.openstreetmap.org/wiki/Tag:boundary%3Dadministrative might be useful. There are 11 administrative levels across the world. I'd want to be sure that the code selects an appropriate admin level in every country. Sure, states in the USA are fine, but what about in Senegal, in Japan, in Argentina?

[robinpaulson]

This might be useful, in the nominatim documentation: https://nominatim.org/release-docs/develop/api/Reverse/

OWM will return a lat, lon pair for each city it shows in the search return box (i.e. the image you included above)

[chibongho]

You're right, I'm not getting the map location pictures shown. I completely overlooked that. I'll try to dig into why that in.

Yeah, I can see this isn't implemented yet. I'll take a look at the nominatim doc you sent.

[chibongho]

I did some research and here's what I found.

The nomainatim API will not serve our needs. It is unable to reverse lookup too many lat/lon coordinates of cities I tried:

Rio de Janeiro, BR: https://nominatim.openstreetmap.org/reverse?lat=-43.2075&lon=-22.902781&format=geojson&zoom=5 San Francisco, US: https://nominatim.openstreetmap.org/reverse?lat=-122.419418&lon=37.774929&format=geojson&zoom=5 Paris, FR: https://nominatim.openstreetmap.org/reverse?lat=2.35236&lon=48.856461&format=geojson&accept-language=zh

OWM has supported cities listed here. There are 213000 of them: http://bulk.openweathermap.org/sample/ (linked to from https://openweathermap.org/current)

Although undocumented, I'm pretty sure the OWM CityIds are actually Geonames Ids, and Geonames has an API to look up State/Province info by Geonames Id:

Rio de Janeiro, BR San Francisco, US Paris, FR

IMO, the ISO3166-2 code in the returned XML is what we should display as the state/province information. https://en.wikipedia.org/wiki/ISO_3166-2

More about Geonames:

• Calls to geonames /get API is rate-limited to 1000 per hour and 20000 requests per day . (http://www.geonames.org/export/ and http://www.geonames.org/export/credits.html). For comparison, the free tier of OWM (if that's what we are using) has a rate limit of 60 per minute.
• Their data is under the CC Attribution 4.0 license. (http://www.geonames.org/ and https://creativecommons.org/licenses/by/4.0/), so we can technically just download the ISO3166-2 code of all 213000 cities and store it somewhere (and provide attribution somewhere in the app). The uncompressed size of that data would be roughly < 2MB. However, somebody will be responsible for curating that data, and make it either downloadable or bundled with the app.

I also submitted a question to OWM customer support requesting ISO3166-2 code to be returned for city queries. I'll wait for a reply from them.

Let me know if you have advice/opinion on how to proceed.

[robinpaulson]

Excellent research, please do proceed as you think best. It's disappointing that nominatim doesn't work, but geonames is fine. I am OK with distributing the data with the package, nice to see owm reusing their codes.

[robinpaulson]

I think curating the data could become part of the app maintenance, could you document your findings somewhere? Here would be fine.

[robinpaulson]

I'm interested which data set you were looking at. I checked out the http://download.geonames.org/export/dump/ page, but not sure which of the dumps you thought we might use.

[robinpaulson]

IMO, the ISO3166-2 code in the returned XML is what we should display as the state/province information. https://en.wikipedia.org/wiki/ISO_3166-2

This looks like a fine suggestion.

[chibongho]

I'll use the allCountries.zip file. It's huge (1.5GB unzipped) but it has everything we need. The file only contains FIPS code and not ISO_3166-2 code, but somebody else made another parser to convert from one to the other: https://github.com/Tigrov/geoname-divisions

I'll proceed to write the parser to generate the ISO_3166-2 code of all OWM cities. I'll put it in a different github repo.

[robinpaulson]

I'll proceed to write the parser to ...

Excellent, good news.

[FridoDeluxe]

You're right, I'm not getting the map location pictures shown. I completely overlooked that. I'll try to dig into why that in.

Just to confirm: I noticed the same behavior a few weeks ago, but haven't investigated the reason.

[robinpaulson]

A guess: I wonder if we have exceeded our quota for map tiles? Is there a quota?

[FridoDeluxe]

From my experience OSM seems to throttle the serving of tiles, but I think this is more done on an IP range basis. But they also evaluate the user agent, here is the policy: https://operations.osmfoundation.org/policies/tiles/. I haven't checked all points and I'm not sure if Forecastie meets the policy.

The missing map bug is probably a candidate for its own issue.

[chibongho]

Ok, here's my update.

• It turns out ISO_3166-2 codes are not human friendly for all countries. It works for places like US or GB, but some countries that code is just one letter long (Greece), and some countries use other abbreviations that is different from that code (Philippines)
• it turns out geonames has a reverse geocoding API similar to nominatim's to give the "Country subdivision" (state/province name) based on latitude + longitude. I overlooked it when researching geonames API last week: http://www.geonames.org/export/web-services.html#countrysubdiv

The API gives the full names of divisions (not abbreviations) and support localization into different languages. That API is also rate-limited to 1000 per hour and 20000 requests per day. I think that's acceptable as users should only be using it while searching for cities. What do you think about using it instead of storing the data within the app?

[robinpaulson]

@chibongho good work doing the research, that sounds fine, lets go with geonames. I doubt we'll hit that number, as you say it's mostly only for the first time a person installs/goes to a new location.

[robinpaulson]

Hi @chibongho how did you get on with this?

[chibongho]

Hi, sorry for be unresponsive. I did some on it some but got busy these past couple months. If you have time to work on it, you should probably take over.

[robinpaulson]

For those who were interested in the bug which caused the maps to not display, that has since been fixed.

[robinpaulson]

@chibongho No problem, we all have other priorities. It's beyond my skills, but maybe someone else will pick it up. You've posted some good research here as a start point.

[ghost]

Dropping in to note that this could probably be considered an accessibility issue, seeing as not everyone can physically see the maps, even when they do load correctly.

[robinpaulson]

this could probably be considered an accessibility issue

This is a valid point. I'm not 100% sure what you mean though. Could you expand please?

[ghost]

this could probably be considered an accessibility issue

This is a valid point. I'm not 100% sure what you mean though. Could you expand please?

Some people use screen readers on Android. Navigating and viewing a map to recognize your location is significantly more difficult if you can't see well, if there are areas in a country with the same name (for example, the many Decaturs in Georgia, Illinois, and Alabama, all just showing up as Decatur, US).

Text that can be read by a screen reader to say "[city], [state/province], [country]" is a lot more accessible in this regard, as well as just nicer.

[robinpaulson]

Text that can be read by a screen reader to say "[city], [state/province], [country]" is a lot more accessible in this regard, as well as just nicer.

i understand. thanks, this is great, we will do as you suggest