Allow exports for anywhere in the world

[d3netxer]

I don't think there should be spatial restrictions of where exports occur. Data can be used for humanitarian purposes anywhere in the world.Are there limited computing resources to process exports?

[bgirardot]

We have discussed this a fair amount and there are resource and bandwidth issues to consider in making it cover the entire world. If there are humanitarian needs for data in places not currently covered by the export, please just let us know and we can add them quickly on an as needed basis as we did for the Balkans. Did you have some specific cases in mind?

[mataharimhairi]

I believe that the current server hosting the Export Tool can cope with supporting all regions @dodobas @bjohare please correct me if I am wrong!

This is something we considered during development, but thought it would be best to run by the Activation WG first. It is really easy to make the change through the admin tool, and again restrict it, say if there was a large scale humanitarian response and we want to focus on a particular area.

It is a shame to limit the areas of export to certain regions, as it is a great tool for getting OSM data out in various formats. Especially for those that don't have a lot of GIS experience but need to get their hands on some files. @bgirardot what do you think?

[PaulUithol]

@TylerRadford , is this something we could prioritize?

I've had a number of people ask me what the best way is to get raw/shapefiles of OSM data over the weekend. The export tool is the best method I know off all around, but in a lot of these cases pointing them to the HOT export tool only leads to disappointment as they won't be able to select the area they're interested in.

The change shouldn't be hard to make technically; the mask gets pulled from the region_mask table (https://github.com/hotosm/osm-export-tool2/blob/master/jobs/models.py#L252; which could either be disabled all together, or updated to a simple poly covering the whole earth).

[mataharimhairi]

@PaulUithol @TylerRadford

This is something we have been discussing in the Activation WG meeting [1], which I have personally been a little slow to action. Will do it this week, so we can review it in the meeting next Wednesday 5th October 2016 at 14:00 UTC. Please join us if you can, your input would be greatly appreciated and I would love to hear more about the comments/suggestions/requests from our users.

[1] https://trello.com/c/rBGVdeJV/48-reasons-to-open-up-all-regions-for-export-via-the-tool

[PaulUithol]

@mataharimhairi , great, didn't realize this was already moving. Afraid I can't make that meeting, will be at a workshop all day. But have added comments to that card, thanks!

[TylerRadford]

I just heard from ARC that they cloned/forked the export tool for use on POSM. They removed all regions. We could talk to Seth Fitzsimmons who did this work. From what I understand from talking to Drazen the only technical concern was that jobs are associated to regions to removing or changing regions would affect old jobs.

Tyler Radford Executive Director tyler.radford@hotosm.org @TylerSRadford

Humanitarian OpenStreetMap Team Using OpenStreetMap for Humanitarian Response & Economic Development web http://hotosm.org/ | twitter https://twitter.com/hotosm | facebook https://www.facebook.com/hotosm | donate http://hotosm.org/donate

On Tue, Sep 27, 2016 at 10:57 AM, Paul Uithol notifications@github.com wrote:

@mataharimhairi https://github.com/mataharimhairi , great, didn't realize this was already moving. Afraid I can't make that meeting, will be at a workshop all day. But have added comments to that card, thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hotosm/osm-export-tool2/issues/106#issuecomment-249887947, or mute the thread https://github.com/notifications/unsubscribe-auth/AJZbz2caqfrNWvYdy5o74Wdm5dgYSF4uks5quS7ugaJpZM4GxIWX .

[mataharimhairi]

Brian updated the tool so that removing existing region polygons through the admin export tool no longer deletes jobs associated with it, but simply removes any assigned region [1].

The wiki states that we can then reassign a region to them, although I've been playing with the admin interface and I can't seem to find this option. Currently there 755 jobs that would have to be reassigned.

[1] https://github.com/hotosm/osm-export-tool2/wiki/Add-a-HOT-Export-Region

[mataharimhairi]

To be honest though, would the exports need to be reassigned a region if there is only one polygon region 'global'?

[PaulUithol]

Not really I guess. Only page I can see the region being used is on the exports list (http://export.hotosm.org/en/exports/). As long as the map shows you where the exports are, don't think that will be too much of a loss.

[mojodna]

A secondary (and possibly more substantial) consideration is how much would need to be done to reconfigure the local Overpass instance to support this (assuming that it's currently dropping diff updates for areas not exposed through the Export Tool). By omitting the US and Europe, particularly data-dense areas don't need to be tracking.

We got around this (for POSM) by using the public Overpass API instance, which we deemed acceptable because the number of people using that Export Tool instance was assumed to be much lower than HOT's.

[bgirardot]

If we want to limit areas again in the future is that still possible?

We have no idea the impact on our services this has, if it does turn out to be an impact we can not afford, we need to be able to go back to region limited exports.

[mojodna]

Yes, but not with the existing system (which more more intended for associating exports w/ regions than limiting currently-available regions). I fully agree that we want the ability to be able to limit areas, but I think the approach should be simplified and more targeted to restriction than association.