Autosolve does not always work

[daganzdaanda]

Look at https://www.openstreetmap.org/changeset/118560111 and see that the linked notes are still open. Sadly, I did not run with the --debug flag. What I recall doing: I downloaded a lot of notes with josm to look for things I could work on. I have set JOSM to also download closed notes for 500 days, and a limit of 1000 notes downloaded at once. I may have had more than 1000 notes downloaded. I looked at a few open notes and saw https://www.openstreetmap.org/note/2442094 which I commented on and set to "remember". Then I worked on https://www.openstreetmap.org/note/2980584 and https://www.openstreetmap.org/note/2695620, also adding a comment and setting them to "remember". This one https://www.openstreetmap.org/note/3087172 I clicked on "remember" but then decided it should remain open and clicked "forget". I also added a comment there.

I'm not 100% sure whether I first added the comments and afterwards pressed "remember" or if it may have been the other way round, too. But I did similar stuff when I tried to replicate #11 and I never got a problem then...

I also can't say if the comments were uploaded by NoteSolver or if it happened afterwards as I pressed JOSM's "upload changed notes" button.

Here's the JOSM status report after I saw that the notes were not closed.

[daganzdaanda]

Some wild guesses for a possible reason:

• Could there be a problem with just too many notes, since I dl'd a big area full of them?
• Or does "forget note" sometimes clash with comments added through JOSM?

What's clear compared to my previous issue #11: This has nothing to do with interrupting the upload. This time I got a warning about "fixme" tagged things, but I continued the upload.

[daganzdaanda]

Here's the --debug output with a test that did go wrong somehow, I think... but I'm getting confused right now, too many variables... 😕 😖

[daganzdaanda]

It happened again: https://www.openstreetmap.org/changeset/118756500 should have closed three issues. No --debug info, sorry.

This time there were not a lot of notes dl'd, so probably not a memory problem. I also did not click "forget note". I added a comment to two of the notes and after that marked them as "remember". I continued mapping and when I wanted to upload everything, the autosolve dialogue came up where I chose yes. Then the validator had some warnings so I cancelled the upload to fix these. Again, upload - autosolve - yes - another validator warning - cancel. So again, fix warnings - upload - autosolve - yes - and now at the third try there were no warnings that I could fix anymore. I continued to add a few words to the changeset comment and finished the upload. I did not get the message that the notes were being closed / uploaded, but also no error message.

The two comments I added to the notes have been uploaded, but I think I did that "manually" through JOSM (should be visible from the timestamp being different than the changeset timestamp?)

[daganzdaanda]

Wild guess number 3: Could adding something to the changeset comment lead to some issue here?

[daganzdaanda]

This time there was only one note to remember, and it did not get closed: https://www.openstreetmap.org/changeset/118797668 https://www.openstreetmap.org/note/2078000 Hopefully this gist of JOSM --debug output is helpful for you.

What I did this time: add comment to note > set to remember keep working try upload > autocomplete dialog > yes > validator issues > cancel repeat until satisfied then add a few words to prefilled changeset comment > upload to server. Result: changeset is uploaded, but the note is ignored - the comment is not uploaded, and the note remains open.

[daganzdaanda]

Again I tried unsuccessfully to reproduce the problem with a new JOSM home directory on master.apis.dev.openstreetmap.org Just adding a few words or characters to the changeset comment is not what makes the plugin forget to close the notes. I have no idea how to get closer to the problem..... 😢 It seems complicated, maybe there's some interference with other plugins?

[daganzdaanda]

It happened again: https://www.openstreetmap.org/changeset/118931559 JOSM status report JOSM debug output

[daganzdaanda]

This time I noticed that when I wanted to exit, I got the info from JOSM that there are changed notes that have not been uploaded. So I cancelled the exit to get a status report and the debug output. Then I clicked the upload notes button and afterwards I was able to exit without a reminder from JOSM. The comments that I added to the notes were uploaded, but the notes are still open (Makes sense, JOSM does not know I wanted to close them.)

[daganzdaanda]

And again: Changeset; Josm status; Josm debug This time I interrupted the upload to fix the validator's warnings again, and with the second try added text to the changeset comment and the changeset sources field.

Any idea what's keeping the notes from being closed automatically?

[daganzdaanda]

Another one: Changeset; Josm status; Josm debug This time I did not interrupt the upload.

[daganzdaanda]

@kmpoppe Do you see any plugin I'm using that might be interfering with noteSolver?

[kmpoppe]

Hey @daganzdaanda,

so one problem I'm guessing is that you make changes to the notes without uploading those first before you upload the changeset. That's something I have never tested as to how it might change the state of the note or even prevents closure. I will give that some thought. Maybe there are ways to detect non uploaded changes and start a "sync" on them first.

The problem with the upload being cancelled due to QA checks by JOSM was something I could never replicate creating weird results.

The exclude any other plugins interfering, could you please find yourself an area and a note, where no QA will interfere (you can check them before you hit the changeset upload) and where you do not add any comments to the note, and then do a simple change, remember the note for closure and upload it? If that gets closed properly on both ends we'd be one step further.

K

[daganzdaanda]

I just did this changeset, only one note remembered, no comment added to the note, only one change, no validation problems, no editing the changeset fields. Having all the same plugins as before installed. It worked as it should. I remember that I had some successful auto-closures earlier also with adding a comment to the notes before. So I'll try this again now.

[daganzdaanda]

Yeah, it is possible to add a comment to notes and have them still autoclosed: Changeset

[daganzdaanda]

After you told me in #11 how to do it, I did a whole round of tests on the master dev-server with a clean JOSM: https://master.apis.dev.openstreetmap.org/user/dgda2/history#map=14/80.0695/12.4836 I was even provoking validator issues there, and tried to systematically vary what I did to the notes in what sequence, trying to reproduce the issue but it never showed up...

And before that, I tried in a series of changesets on the normal server with my normal JOSM and plugins, see https://www.openstreetmap.org/changeset/118487490 https://www.openstreetmap.org/changeset/118487490 https://www.openstreetmap.org/changeset/118487620 https://www.openstreetmap.org/changeset/118487724 https://www.openstreetmap.org/changeset/118487885 https://www.openstreetmap.org/changeset/118488008 https://www.openstreetmap.org/changeset/118488517 https://www.openstreetmap.org/changeset/118488909 and the linked notes. Again, I was not able to get the problem in those "laboratory" conditions.

[daganzdaanda]

https://master.apis.dev.openstreetmap.org/changeset/234137#map=16/80.0685/12.5079&layers=ND The note https://master.apis.dev.openstreetmap.org/note/30268 was not closed. I had been adding plugins one by one, and merging nodes with "M" to check if I could get at #18 I did not upload unless the new plugin needed a restart of JOSM. The last plugin I installed was "public transport", before that in the same session (from latest to earliest) "pt assistant", "photo geotagging", "open qa", "opendata". Josm status Josm debug

[daganzdaanda]

The note is still set to "remember".

[daganzdaanda]

Still same JOSM session, no restart: After making another change I tried uploading again. I got the "close note automatically" again, and the validator errors again, but this time with the upload it actually did get closed. https://master.apis.dev.openstreetmap.org/changeset/234141

[daganzdaanda]

restarted JOSM continued installing plugins. https://master.apis.dev.openstreetmap.org/changeset/234146#map=16/80.0685/12.5079&layers=ND https://master.apis.dev.openstreetmap.org/note/30259 still open JOSM debug JOSM status

newest plugin was "tageditor" before came "scripting", "reverter", "reltoolbox". Using "M" to merge nodes was no problem in all of these tests.

[daganzdaanda]

closing JOSM right after this gives the "unsaved notes" reminder. I selected to "upload changes before closing". very interesting result in https://master.apis.dev.openstreetmap.org/note/30259 so the last two comments about new plugins were not uploaded!

[daganzdaanda]

next changeset without adding any more plugins, only one merged node, added 2 comments to note, no validation popup: note was solved and closed.

same session, added terracer and turnlanes plugins - needs restart, did not allow to close JOSM just right away. Tried some another merge operation, no problem. Then did the upload, this time with a validator warning (ignored), upload went fine, note is closed: https://master.apis.dev.openstreetmap.org/changeset/234162#map=16/80.0686/12.5078&layers=ND

[daganzdaanda]

changeset did not close https://master.apis.dev.openstreetmap.org/note/30274 josm debug josm status note looks like this in josm: first a comment was added, then set to remember, then the other comments. all comments were added before doing the upload. no new plugins were added. "m" merging was not a problem.

[daganzdaanda]

closed JOSM let it upload the changed notes, result:

1 comment uploaded, 4 comments forgotten. Very weird.

[daganzdaanda]

I had an issue with notes not uploading when exiting more than a year ago already, see JOSM ticket 20566. I didn't know of noteSolver at that time. So it this specific problem is likely no fault of noteSolver. But it may make finding the other issue(s?) more difficult.

[daganzdaanda]

debug status

Hi again... haven't been mapping a lot lately, but now this issue has shown up again. Only two notes, I did add a comment to both and uploaded the comments first. Only after that I used the "remember note" function. When I uploaded the changeset I did see the notice that the notes will be closed, and the info was in the changeset comment field. The changeset made it to the server: https://www.openstreetmap.org/changeset/121627394 but the notes are still open as I type: https://www.openstreetmap.org/note/3192635 and https://www.openstreetmap.org/note/3192633 ETA: in JOSM notesolver still shows "forget note" selectable on these notes, so it seems to know that they haven't uploaded.

Maybe you can find a pattern in all of the debug info I've collected? Even if it might have a JOSM bug as its root?

I really like this plugin, it's just a shame that it does not seem to work reliably on my computer...

[kmpoppe]

Hi @daganzdaanda, I published a beta (https://github.com/kmpoppe/noteSolver/releases/download/0.4.3-beta1/notesolver.jar) that I think addresses this and #18. Could you test this please? K

[risturiz]

Tested :-) ... Maybe my issue have relation with this... The plugin don't close the note when i create a lot of nodes ( highways ), but works when i edit few nodes.

[daganzdaanda]

Hi @kmpoppe

Thanks! I installed the beta version but didn't do any mapping for a while... Now I did use it on one note, with no good result 😢 changeset says it's closing the note, but the note is still open. https://www.openstreetmap.org/note/3171536 https://www.openstreetmap.org/changeset/123493273 status debug Hope you can find something in those logs...

You can see that I added a comment to that note. I thought I pressed the green arrow to upload the comment before I uploaded the changeset, but I seem to have mis-clicked: the arrow is still green and the comment is not yet visible on the website. I now pressed the green button again, and the comment got uploaded, and then I closed the note manually.

[daganzdaanda]

wait... the josm debug says I still have notesolver 0.4.2 ?? ARgh 😫 Sorry! Maybe the JOSM automatic plugin update has overwritten the local version? ...OK, now I've got + notesolver (0.4.3-beta1) installed again... I will report when I use it.

[daganzdaanda]

This time I did a few more notes and edits, and the result is this: https://www.openstreetmap.org/changeset/123494387 The notes are still remembered after the changeset has been closed: debug status

[daganzdaanda]

Hi, it's me again. Changeset https://www.openstreetmap.org/changeset/127223542 Should have closed https://www.openstreetmap.org/note/3328026 which coincidentally was opened by you :-) Status Debug This time I had saved the session in which I added a comment to the note. Then after restart of JOSM I loaded the session, marked the note to be remembered and went on to upload the changeset. I did get the notice that the note will be closed, and the changeset text was prefilled. I added my personal text, and completed the upload. The note remained open. I will manually upload the added comment and close the note.

[daganzdaanda]

https://www.openstreetmap.org/changeset/127695815 https://www.openstreetmap.org/note/3328655 Josm status Sorry, forgot the --debug this time.

This time I "remembered" the note, added 2 comments, started upload, okayed the auto-close-request, got some validator issues, stopped to fix them, started upload again, ok to the question, added personal text to changeset comment, uploaded. Changeset shows up, note still open.

[kmpoppe]

@daganzdaanda Thanks for your patience. I have released version 0.5.0 - I have now sometimes seen what you described but I'm still no closer to understanding why that happens. Please try this new version anyway and check whether the behaviour changed for you.

[ygra]

For me it's still the same: It sometimes doesn't close the note. Typical workflow for me is one long session, open notes from the web site, fix them, commit and close note. Remove all layers except for background imagery and start anew. Sometimes I get the feeling that if the Notes layer is still there with other notes it doesn't work, but that's no hard and fast rule, seemingly.

[daganzdaanda]

Thanks, I will try and see what happens :-)

[daganzdaanda]

Had a longer session now: https://www.openstreetmap.org/changeset/133841380 says it closed a lot of notes, but the screenshot says otherwise... debug status

[daganzdaanda]

I'm closing these notes manually now

[kmpoppe]

Next time, please check if you have the notes "in focus" - i.e. they are visible in the JOSM main window. Wonder if that has something to do with it ...

[daganzdaanda]

Hi! This time it's just one single note and it is still in view in the JOSM main window: The changeset: https://www.openstreetmap.org/changeset/134280429 The note: https://www.openstreetmap.org/note/3053957 JOSM debug JOSM status

I will close the note manually.

[daganzdaanda]

And another one... https://www.openstreetmap.org/changeset/134363309 https://www.openstreetmap.org/note/2975827 https://www.openstreetmap.org/note/2975826 I added a comment through JOSM to this note after "remembering" it and before closing the changeset. I did not upload the comment manually. NoteSolver did ask me whether I want to close/upload the two notes, but it didn't do so. Status Debug

...Oh, and the notes were both visible in JOSM when I did the upload.

[daganzdaanda]

So after the above, I tried something new. I uploaded the changes to the notes (the one comment on note 2975826). Then I moved two nodes of a building to have some new changes to upload. When I started the upload, again I confirmed I wanted the two notes closed. And lo and behold, this time they were closed! https://www.openstreetmap.org/changeset/134363502 debug status

[rohieb]

Yesterday I also had a whole JOSM session (about 9 notes) which were closed successfully on upload. They were always in view when I uploaded.

[daganzdaanda]

Yesterday I also had a whole JOSM session (about 9 notes) which were closed successfully on upload. They were always in view when I uploaded.

It's really nice to hear that! Sometimes I got noteSolver to work as it should, too. But most of the times it doesn't close the notes. I can only imagine that something in my JOSM setup might be different and not compatible. Argh.

[daganzdaanda]

Again: https://www.openstreetmap.org/note/3160043 https://www.openstreetmap.org/changeset/134918761 This time I added the comment to the note first and uploaded it (which I'm pretty sure took two clicks on the button "send changes to server", the first one did not do anything). Then I set the note to be remembered, and started the upload for the changeset, confirmed I wanted to close the note. Had to stop again, because of validator errors, fixed those, did the upload. The note is still open.

debug status

[daganzdaanda]

I had problems with JOSM uploading note changes before: https://josm.openstreetmap.de/ticket/20566 Like I said, this time I am pretty sure I hit the "Upload note changes" button once and nothing happened until I pressed it a second time. Maybe my JOSM always needs two "hits" to start the upload, be it button clicks or software calls? Will have to watch out for this...

[kmpoppe]

So ... are you always trying to add additional comments to a note before you try to close them via noteSolver? I can only imagine that the Plugin simply cannot detect whether the note needs a change and thus won't upload anything.

[daganzdaanda]

I just reproduced it: closed JOSM and restarted to new session. DL the area and notes. Click on "close note" button, add text. Click on "upload note changes to server" button. Nothing happens. Click a second time on that button: Note change is uploaded (commented and closed) status debug

[daganzdaanda]

Another test with a note around null island debug I needed 2 clicks again, but I can't interpret the debug output if it recorded the clicks or not.

[daganzdaanda]

I removed a lot of plugins and tried again with another note. This time everything worked with a single click! status debug

[daganzdaanda]

I added back the notesolver plugin. And now I had to double click the button again(!) status debug

The debug output also shows repeated lines again, was not looking like that last time, when I only needed one click to upload the changes.