Node has disappeared

[Svenyo]

@1000miles cc @holgerd I edited the phone number of a place that was visible on https://wheelmap.org/nodes/-152854733 11 days later, I check up on the place again. The URL shows a blank page and the entry has disappeared from Wheelmap while it's still visible on OSM. I also get a result when typing the name of the place in the Wheelmap search bar.

[holgerd]

My hunch is that the sync stopped working again.

[holgerd]

Strange: https://wheelmap.org/nodes/-152854733 now shows up.

[holgerd]

@1000miles please check created_at and updated_at of this node.

[1000miles]

Even though https://wheelmap.org/nodes/-152854733 now shows up we need to investigate this strange behavior.

[1000miles]

As discussed yesterday here are the required infos from the database:

+------------+---------------------+---------------------+
| osm_id     | created_at          | updated_at          |
+------------+---------------------+---------------------+
| -152854733 | 2017-01-02 10:48:19 | 2017-01-02 10:48:19 |
+------------+---------------------+---------------------+
1 row in set (0.00 sec)

[holgerd]

Thanks @1000miles . In the node history on OSM I see:

• The wheelchair info was added by @Svenyo on Jan 2, 10:28am UTC.
• Before that, she added tags to the place on Dec 22nd in this changeset (as she said in in her initial comment of this issue).
• 3 years ago the key tourism=chalet was added to that place (changeset).

Question remains why this node was not visible on Wheelmap on Jan 2nd in the morning.

Since it works now and we don't have any clues that this might be a broader issue, I close this now.

[holgerd]

@schultyy suggested to investigate further so I reopen this.

[schultyy]

I already invested some time in #446 where we closed it as Won't fix because we couldn't find out what's causing the deletion. This one is a similar case. I didn't find any DELETE statement in the MySQL slow.log file also no hint in any other log file.

What we could do is to enhance the planet_reader which is performing the import into MySQL to write logs with Ids it created or deleted to see if this happened during import.

[ghost]

Reopened bc similar case was reported @Svenyo will attach screenshots.

[Svenyo]

From the experience of the user http://www.openstreetmap.org/user/Ronos:

1) I have the same problem after I tagged accessibility by using wheelmap iOS and wheelmap webpage. It happens 30% of cases. Sometimes the places disappear after tagging not immediately, but after around 2-5 days.

The tagging wheelmap problem is only with the places that appear in the another plataform, openstreet, like AREAS. In openstreet, there r two ways to add places (example: restaurant): By using point or by using AREA. For understanding better please, go to openstreet, press edit. After that, it's gonna appear the option to add things. Three options: point, line and area. I added a picture for u to understand what bottoms I'm talking about.

2) till now the problem doesn't exist when I add place by using wheelmap, iOS or open street. The problem is ONLY AFTER TAGGING by using wheelmap browser or iOS. We don't have android at home, so I don't know if it happens the same there.

Example: https://www.openstreetmap.org/way/465233475

However, I'm not sure, if the tagging of the accessibility (wheelchair=limited) really caused the problem. From the changeset it seems that the last edit Ronos did was adding the tag wheelchair:description.

[Svenyo]

Some more examples from the same user

1) Please compare: https://wheelmap.org/map#/?lat=-12.075119780588828&lon=-77.08077818155287&zoom=19 https://www.openstreetmap.org/edit#map=20/-12.07489/-77.08108

2) Please compare: https://www.openstreetmap.org/edit#map=20/-12.07453/-77.08215 https://wheelmap.org/map#/?lat=-12.074389312271338&lon=-77.0822212100029&zoom=19

[holgerd]

nest step: check in the logs if we have deleted it.

[holgerd]

Hunch: Maybe this happens when you attach a wheelchair-tag to a building or way.

[Svenyo]

@holgerd cc @Hoverbear Further example of a user confirming the idea that this is somehow connected to "ways" in OSM:

Ted writes:

I have found some problems with editing (and adding photos to) shops etc. that are marked on OpenStreetMap as multi-nodes = ways/areas, rather than single nodes.

• However, as shown below, I have now found a solution to this issue

Examples in Reigate:

http://www.openstreetmap.org/way/374832869 Way: Toney's Tattoos (374832869) http://www.openstreetmap.org/way/47153166 Morrisons Supermarket

Before I added the wheelmap data for the Supermarket: http://www.openstreetmap.org/changeset/35593558 It had appeared as a node on Wheelmap, and on the Android App, but then it suddenly disappeared.

I had the same "Disappearing" problem with http://www.openstreetmap.org/way/307696808 - Reigate Baptist Church, but this has reappeared again on Wheelmap for no obvious reason. https://wheelmap.org/nodes/-307696808

Having written the above, then looked at the two links again, I have managed by playing around, found that I can edit the Tattoo Shop via Firefox by using the

• OSM + way/[reference]
• as a wheeelmap + node/-[reference]

Interesting! Perhaps it will reappear on the OSM app once I have saved it in this format?

• Yes it did !!!

Will wok this solution on other places that I have had issues tagging

Regards

Ted

PS: Was going to ask but now have the answer to: "What is the right way to edit these (and other examples) so I can bring them back as Wheelmap visible places without causing them to have double entries on OSM, both as a single node, and as way/building?"

[Hoverbear]

Hmm it sounds like this might have to do with the pipeline for how we deal with ways...

[Svenyo]

And some more information:

Now checked what I got for Morrisons.

With both Firefox (on the Linux laptop) and Chrome (on this Android phone) I get the error page "sorry, page not found" if I use the "-" prefix for place. https://wheelmap.org/en/nodes/-47153166 Marked as a way on OSM.

However, using the "-" works with Tony's Tattoo & the Church listed below. Also marked as ways on OSM

(...)

Regards Ted

[schultyy]

• 465233475: Logs don't say anything about this one
• -307696808: Logs don't say anything about this one
• -47153166: Logs don't say anything about this one

Either these haven't been deleted by the importer or it happened on a point in time which is not covered by the current log file.

I checked the logs again for POI -374832869 (http://www.openstreetmap.org/way/374832869):

Creating POI -374832869
Deleting POI -374832869
Creating POI -374832869

Unfortunately none of the entries do have timestamps so I can't say when these actions happened. Also note that they didn't happen consecutively. They only show up in this order because I searched for this exact Id, omitting everything else in between.

This one got deleted during the import at some point. The importer deletes a node if one of the criteria is not met:

• Tags are not empty
• node_type id is set
• is not private (tags.access != 'private', tags.access != 'no')
• Is not abandoned (tags.disused != 'yes', tags.abandoned != 'yes')

EDIT:

I checked the database for -374832869:

• It was deleted at: 2017-02-27 15:21:17
• It was created again on 2017-02-27 15:33:10

[schultyy]

I checked the changeset for -374832869:

• The website was changed (before it was www.tonystattoos.co.uk, now it is http://www.tonystattoos.co.uk)
• It got a description tag

[holgerd]

@Hoverbear can you please test the idea we just discussed today to learn more about this issue?

[Hoverbear]

I tested on https://staging.wheelmap.org/nodes/465233475, will see if it appears over the next couple days.

[Hoverbear]

The node did not reappear. :(

[Hoverbear]

Putting this on hold as we talked about at the meeting last week.

[Hoverbear]

I was grepping the update logs in /home/osm/log/update.log today and noticed this error:

Wed Apr 26 14:50:01 CEST 2017
lockfile exists
Processing: Node(91k 0.0k/s) Way(13k 0.01k/s) Relation(430 0.49/s)insert_rel failed: ERROR:  value "37945" is out of range for type smallint
(7)
Arguments were: 7066589, 37945, 37945, {4731083058,...}, {...},
Error occurred, cleaning up
osm2pgsql SVN version 0.81.0 (64bit id space)

error in osm2pgsql, restoring old state file

This error seems to be caused by something residing outside of the range of smallint.

We noted that both of the nodes in #600 (and one from this issue) were not present in the OSM clone, so perhaps it's the case that these failures are causing us to lose nodes?

This may be related to https://github.com/openstreetmap/osm2pgsql/issues/713. I note that was fixed in March this year, and we're currently using 0.81.0 (released on 14 Sep 2012 ) which is much older than 0.90.x series that include the fix.

[Hoverbear]

After a discussion with @Xylakant today we don't really have a good way to test updating this tool.

This is partially due to the fact that none of the other instances of wheelmap we are familiar with are having the same problem.

Updating on production is very risky (since the current version is so old), so it's likely a good idea to avoid trying to do that.

[holgerd]

@Hoverbear What are reasonable next steps here?

[Hoverbear]

@holgerd Well, we really need an instance of wheelmap that isn't production which has this issue as well. This has proven to be a problem as production and staging both use the same OSM database, and the new infrastructure is not having this problem.

There is a big risk to upgrading the osm2pgsql on production since we'd be jumping over 5 years in versions. Doing this would require (at least) some downtime and any issues could result in an extended (at least a week) downtime while we rebuild the entire OSM database.

In my opinion the best step to take would be to try to use the version we're currently using (0.81.0) on one of the instances of the new infra and see if the issue is reproduced.

[holgerd]

Plan: wontfix on old infrastructure. When moved to the new infrastructure, test this again