search

osmlab / appledata

114 stars 5 forks source link

Data Improvements in Brazil #126

Open awisemanapple opened 5 years ago

awisemanapple commented 5 years ago

Task Description

This task outlines work to improve OSM data in Brazil, such as the road network, coastlines and inland water, and land use -- remove duplicates, correct geometry, fix incorrect tags and other similar things.

Improvements that will be addressed include the following:

Mapping Guidelines

The team will follow OSM and local policy, along with any other guidelines as appropriate.

For water and coastlines, the team will follow OSM coastline guidelines, waterways policy such as water and natural=water and local policy along with any other policy as appropriate.

Some coastlines are connected to administrative boundaries, in which case we will adjust both if appropriate.

Where we see them, we will remove the source=PGS tag, which was from an earlier coastline import that needs to be cleaned up. We received a message on the India coastlines project requesting this, and it is commonly done in other coastline improvement edits.

The team will use the imagery and data sources specified below as needed for mapping in the country.

When in doubt, the team will use the existing tags and data which are used locally. The history in JOSM (ctrl-H) identifies what previous mappers added and their notes and/or explanations regarding changes made. This history can also show if a previous mapper has visited that area or has expert knowledge or sources. We also reached out to the talk-br email list and the OSM Telegram for Brazil for feedback and suggestions.

Our team uses the hashtag #adt (for Apple Data Team) for our edits.

Area of Focus

Throughout the country.

Tools

The team will use JOSM for completing the task. JOSM has validation warnings for road networks and other possible issues that are important to prevent improper changes.

Sources

The team will apply image offsets as needed.

Ferry data

Apple is releasing data on ferry terminal and line information missing from OSM in Brazil. This contribution is made available for use under the OpenStreetMap ODbL license. The data includes ferry terminals with names of ferry lines, whether the ferry is for cars, and the direction of the ferry, if applicable. We will confirm that the ferry exists and the terminal location is correct, and if missing we will add those to OSM and estimate the ferry route. The data includes starting and ending points at ferry terminals but not the specific routes between them. When we use the data we will cite it in the OSM changeset source tag.

Changeset Comments

The team will provide changeset comments that are in compliance with OSM changeset guidelines.

Error Detection

The team will check for errors visually and using JOSM validation warnings prior to committing the changeset.

In addition, the team will review its work in accordance with validation guidelines such as the OSM Wiki and LearnOSM validation guidelines

Things you can do

If you'd like to help, here are a few ideas:

Contributors

For more information

Our community lead is Teddy Ahlvin and our editing leads are Jon Westlake and Erica Olson. To get in touch with Teddy, you can email or contact him via OSM. All of our leads can be reached via email.

OSM Wiki

ftrebien commented 3 years ago

In the last 8 days, there were more than 1000 changesets made by ADT in the metropolitan region of Porto Alegre. They all have comments in English. This has never happened before, many local mappers are not fluent in English. I saw several mistakes, including messing with the classification of highways not following local practices or the wiki, all described but never explained as has been required of local mappers for a long time. Not long ago I've also seen changes made by ADT based on old data that did not reflect recent changes observed on the ground. What is happening? Why the sudden change?

awisemanapple commented 3 years ago

@ftrebien hello ftrebien, I also responded on the Brazil Telegram. If you can share changesets you have questions about, or if you wanted to comment on them on OSM, that would help us understand your questions. We always try to follow the local highway guidelines.

ftrebien commented 3 years ago

@awisemanapple What I have so far:

awisemanapple commented 3 years ago

Thanks for the examples, and for the conversation on Telegram. Most of our edits were focusing on adding missing roads and using Mapillary and OpenStreetCam/Kartaview to fix one-way issues in this area, with smaller numbers of edits to classifications with the goal of following the local guidelines or addressing gaps or sudden jumps. I understand from our conversation that the guidelines are not fully accepted by everyone and can be subjective, so we will plan to not make many adjustments to classifications, especially for tertiary and above, and will make sure to take the surrounding classifications into consideration as well when adding missing roads. I also see your point about the main streets in neighborhoods such as https://www.openstreetmap.org/#map=17/-30.03977/-51.14162, we can change those back to tertiary.

You should also see fewer mappers and fewer changesets going forward in the area, as well as more explanation in the changeset comment, and the team will focus more on things that are objectively inaccurate, such as missing roads and things added by brand new mappers or just plain mistakes.

Thanks again for your suggestions and feedback, it’s very useful.

ftrebien commented 3 years ago

@awisemanapple Thanks for your consideration. You are right, we are still looking for rules that would make the classification system both satisfactory and less subjective. Unfortunately, in Brazil, conditions vary significantly from place to place, so it is difficult to find objective conventions that apply satisfactorily everywhere. Many turned out to be problematic in practice.

This change did not affect the map much and is within acceptable limits depending on personal opinions, so there is no rush to undo it, only that providing a justification is important to help the local community advance its street classification standards. The street in question is in better condition than the nearby parallel streets (although in absolute terms its conditions are not optimal), the density of both the the tertiary network around it is low (particularly in the area bounded by urban arterials / secondary roads), and the density of the secondary network is also relatively low, so there is room to choose one main way that likely functions as a collector for that residential area. Usually it would be two main ways that are approximately perpendicular to one another and that divide the arterial-bounded area in more or less equal sub-areas, connecting to the arterials on most or all sides, but street grids come in many configurations and this often requires adjustments and sometimes just doesn't make any sense considering other desirable qualities (such density and complexity, or the physical conditions of the street relative to other collectors nearby). This type of reasoning is mentioned in the city plans of some other municipalities in Brazil, it is what some planners do when trying to adapt the old system of existing streets to the American-inspired functional system required by federal law. This also tends to reduce zigzag-routing through local streets in poorer conditions, which is useful. The city plan of Porto Alegre is updated every 10 years, which is about to happen anytime soon, I hope this time it will include a collector network that will help make this decision less subjective.

ftrebien commented 3 years ago

classification of highways ... all described but never explained as has been required of local mappers for a long time.

Yesterday I realized that user popnlock, which formerly worked for ADT (like most users mapping for ADT), has "reviewed" the street classification of many cities in Rio Grande do Sul. There are dozens of changesets changing the urban classification all across the state without explanation and ignoring city plans. Satellite imagery was given as a source for the change, but satellite imagery does not present street function or, in the case of emergent networks, usage, which is not captured by imagery. Apple is consistently ignoring the courtesy rule of the community. This opens a precedent for anyone to change road classification without providing any explanation, including in ways contrary to the ideals of the local community or Apple.

List of changesets I've discovered with this issue so far: 108252469, 108234157, 108613793, 107835364, 107901837, 107874495, 108097831, 107952259, 107897031, 108602284, 108652389, 108470821, 108620108, 108294931, 107902986, 108613069, 108652487, 108217170, 108217923, 108652389, 108652276, 108520264

These also have this issue and refer to previous classification work made by other local mappers that closely followed city plans: 107946681, 107946979, 107949008, 108524180, 108523708, 108523780, 108523822, 108523893, 108524248, 108524285, 108524330, 108524354, 108524430

I expect to find more changesets like these. I've seen changes made to cities without an official city plan where classification had been done as a consensus of multiple alternative sources, some of these places I know personally and, based on my experience visiting them and reviewing city plans, I don't entirely agree with the changes.

ftrebien commented 3 years ago

Changeset 105020085 redoes changeset 99380529 which was undone in changeset 99660422 after the Brazilian community reached unanimous consensus on Telegram, with 5 people voicing an opinion against the change made by Apple: @willemarcel, @nighto, @AntMadeira, @Playzinho, and me; and none in favor. Please undo the change and keep it that way.

awisemanapple commented 3 years ago

Hi @ftrebien, thanks for the comment. Our goal is always to follow the local guidelines such as https://wiki.openstreetmap.org/wiki/Brazil/Classificação_das_rodovias_do_Brasil for everything, we don't have our own goals or anything like that for highway classifications. We are doing our best to follow the local guidelines as they are written on the OSM Wiki so that things are consistent, and when we change something it is because we think that better fits the guidelines. From the recent conversations on Telegram I understand that some cities have their own guidelines which are based on the city or state government’s guidelines, so we will make sure to follow those.

When we cite imagery in the changeset source, it means that is the imagery we are looking at when editing. We use the guidance from the wiki and community feedback and examine imagery to try determine the function, connectivity, and overall purpose of roads and highways before making changes. Our goal is to only make changes that fall within those wiki guidelines.

Also in general we will be doing fewer classification changes going forward and will ask in advance before we make classification changes. We will also review those changsets you cited and reach out for further clarification. I hope this helps answer some of your concerns.

Also we saw your comment on 105020085 and are looking into it.

awisemanapple commented 3 years ago

@ftrebien Hello Fernando. I took a look at the first set of 12 changesets that you originally mentioned and agree with you that a lot of these secondary downgrades to tertiary were either unnecessary or changed the balance between the two road classes in smaller cities. I can assign a team member to restore the previous classification in each of the twelve changesets you provided from popnlock. Fully reverting changesets may cause additional data issues from an edit one month ago but we could also go that route to accomplish the classification restoration if you prefer. Let me know if this works and we will get started early tomorrow. Also we will start to look at the new list that you updated. Thanks.

ftrebien commented 3 years ago

@awisemanapple Hello Andrew. Reverting to the previous classification is better in my opinion. It is not a final result, it is open for discussion with the local community; if the local community thinks it's too high in some places, we will improve it in successive iterations. Anyone can join the discussion, although they tend to be time-consuming. If you encounter editing conflicts, perhaps a better (but more laborious) method is to simply set the previous classification manually, without using revert scripts.

adrianojbr commented 3 years ago

Recentemente reverti o changeset 109893347 que alterou, em 8/18/21, a classificação de trecho da rodovia SC-390.

adrianojbr commented 3 years ago

No changeset 107084965, de 6/28/21, foram alterados para tertiary estrada que não tem passagem de carros. Inclusive, há obras no local para assim então permitir a passagem de veículos (https://omunicipioblumenau.com.br/transito-em-trecho-da-ligacao-dos-bairros-velha-e-garcia-sera-interrompido-em-blumenau/).

awisemanapple commented 3 years ago

@adrianojbr Obrigado pelas mensagens. Concordo que https://www.openstreetmap.org/way/353531912 não deve ser terciário. No entanto, pode haver uma classificação melhor se for usado para o tráfego local entre cidades? Por exemplo, a faixa não é mencionada em https://wiki.openstreetmap.org/wiki/Brazil/Classificação_das_rodovias_do_Brasil, mas em https://wiki.openstreetmap.org/wiki/Tag:highway%3Dtrack diz que a faixa não deve ser usada para estradas que são usadas na rede de rodovias de conexão principal e https://wiki.openstreetmap.org/wiki/Pt:Tag:highway%3Dtrack sugere não classificado em vez de estradas secundárias conectando "pequenas povoações, como aldeias, vilarejos ou habitações isoladas". Talvez Highway = unclassified fosse melhor, combinado com outras tags para denotar que é uma estrada que não permite a passagem de carros, como narrow = yes, surface = , lisura = ou tracktype = *?

E para a estrada secundária em https://www.openstreetmap.org/changeset/110428719 nós a rebaixamos para terciária com base nas diretrizes de https://wiki.openstreetmap.org/wiki/Brazil/Classificação_das_rodovias_do_Brasil em que esta estrada não atuar como uma rota ideal entre centros urbanos (com mais de 2.000 habitantes), ou formar uma rota completa conectando-se a outra rodovia de alto nível. Embora tenha uma etiqueta ref e uma superfície pavimentada na maior parte da rota, ela não levou a uma cidade grande ou a um marco significativo, por isso a rebaixamos.

Deixe-me saber o que você acha disso, ou se há outras diretrizes que devemos usar. Também estamos bem em deixá-los como você os alterou, mas gostaríamos de explicar nosso pensamento. Obrigada.

adrianojbr commented 3 years ago

@awisemanapple, inicialmente eu que agradeço a resposta.

A minha intenção é mostrar que não podem ser aplicadas as orientações da wiki indiscriminadamente, sem levar em conta o conhecimento do local.

A Trilha do Gato (https://www.openstreetmap.org/way/353531912) é um projeto de ligação entre bairros de Blumenau (Velha e Garcia), mas está intransitável para veículos, bicicletas e pedestres. Somente passam, atualmente, veículos 4x4. Recentemente foram iniciadas obras para melhorar o estado da via. Inclusive nunca mais tentei passar por lá devido a essas condições. Quando eu puder, passarei por lá para verificar a situação e atualizar o mapa.

Já para o trecho da rodovia SC-390 (https://www.openstreetmap.org/changeset/110428719) concordo que a classificação dela é questionável, tanto que coloquei no comentário do changeset 110428719 (com destaque): "reversão do changeset 109893347 que alterou classificação de rodovia, sem prévia discussão e que eu considero indevida (a alteração)". O critério que usei na época para classificá-la como secundária (https://www.openstreetmap.org/changeset/66856740) foi, principalmente a hierarquia entre as vias próximas, já que essa é a única estrada asfaltada no interior de Capão Alto e Lages, tornando-a rota preferencial. Concordo que isso é discutível, mas que se faça a discussão no local adequando (fórum, lista de emails, Telegram...) antes de alterar o mapa.

Abraço.

awisemanapple commented 3 years ago

@adrianojbr Obrigado, agradeço as explicações. Concordo, faremos o possível para entrar em contato se tivermos alguma dúvida no futuro.

Gustavo22Soaresh commented 3 years ago

@awisemanapple estou encontrando muitas mudanças indevidas de Residencial para Unclassifield https://www.openstreetmap.org/way/495768066

awisemanapple commented 3 years ago

@Playzinho Obrigado pela nota, estamos investigando e iremos restaurar a classificação residencial dentro das cidades do Estado do Maranhão se encontrarmos alguma. Nossa equipe não faz mais esse tipo de mudança entre Residential e Unclassified.