developmentseed / observe

Cross-platform, offline, field mapping tool for OpenStreetMap
ISC License
66 stars 5 forks source link

Batching uploads into single changesets #29

Open batpad opened 5 years ago

batpad commented 5 years ago

Based on feedback from the OSM community, it seems like there is a broad preference to have a larger amount of changes grouped into changesets. A very related issue is "do not save immediately".

Opening this issue to start a discussion, and also make a proposal for how this could work.

In general, I wish OSM changesets would be atomic and this would be the norm, but this is not the place for that discussion, nor can we change that, so using this ticket to outline a path forward.

So, for the first issue, of the app "auto-uploading", it would likely be fairly straightforward to have a config parameter called AUTO_UPLOAD or so. When set to false, we turn off automatically uploading the changeset when the user saves, and just add it to the pending queue. A user would then be able to go to the User Contributions screen, review their changes, and either Upload All, or upload changes individually.

To the second issue of "batching features in a changeset - if we are not auto-uploading, changes would always be uploaded by explicit user action, and when doing an Upload All, we'd have a single user action where we can take the changeset comment, and upload all current pending edits into a single changeset. We could either have a config parameter like BATCH_UPLOADS_IN_CHANGESET, or make this the default behaviour.

I believe the slightly complicated thing here would be dealing with Conflicts. Of course, dealing with conflicts is a bit simpler with the current model as a single feature conflict does not affect other features. I think this should be solvable by opening a changeset and then uploading OsmChange files for features one by one - if we receive a conflict for one, we can tag that edit as a Conflict, but still go ahead with the other uploads and close the changeset. The user would then be able to resolve conflicts for conflicting features as they do now.

May have to think about this a bit more, but this seems overall doable - the way things are setup, it should not be too much of a change to just upload many changes to a single changeset and then close it during the Upload All action. Especially if we go with not automatically uploading changes in the background, this should be relatively straightforward. We will need to think about the Conflicts case a bit carefully - I would rather not block an entire set of changes because a single feature has conflicts, but I think we can definitely work it out so that is not the case.

It seems to me like the broad opinion is against the idea of single-feature changesets, so we can maybe give this a couple of days and see if that continues to remain the consensus, I can sketch this out a bit more in detail and see if I can give an initial implementation a shot.

geohacker commented 5 years ago

So, for the first issue, of the app "auto-uploading", it would likely be fairly straightforward to have a config parameter called AUTO_UPLOAD or so. When set to false, we turn off automatically uploading the changeset when the user saves, and just add it to the pending queue. A user would then be able to go to the User Contributions screen, review their changes, and either Upload All, or upload changes individually.

👍 I support this. cc @mojodna - what do you think?

I think this should be solvable by opening a changeset and then uploading OsmChange files for features one by one - if we receive a conflict for one, we can tag that edit as a Conflict, but still go ahead with the other uploads and close the changeset.

This also sounds good to me. In case of a conflict, once that's resolved, it could go into the next batch of changeset. Also, may be we can default to x number of edits per changeset, and also allow people to adjust that based on the use-case.

cc @sethvincent

mojodna commented 5 years ago

Sure, that makes sense to me.