Better source categorization

bhousel commented 8 years ago

per @nicolas17 #134

I think imagery layers (like satellite) should be here, external raster layers (like some government's open data) should be here, and OSM-based layers (like QA) should be here. But there should be a field in the JSON saying in what "category" the layer is, to allow filtering if wanted.

grischard commented 6 years ago

Why and when would this kind of filtering be wanted?

simonpoole commented 6 years ago

For example a website wanting to show aerial imagery but not maps ....

Klumbumbus commented 6 years ago

Marc-marc-marc commented 6 years ago

for the key name: @grischard proposes categoryin #266 that look like fine.

for the key value : @grischard proposes osmin #266 for render made from osm data. it would have the advantage of involving an (c) openstreetmap contributors. @stoecker proposes map and photo. it look like a good idea to also add those 2 values (map for not-osm-based map like official one, and photo for sat and drone imagery).

did we need a value for derivative works like QA ?

Klumbumbus commented 6 years ago

@stoecker proposes map and photo.

The disuccion in this ticket went a bit further

Marc-marc-marc commented 6 years ago

@Klumbumbus you're right. I forget to check it again before posting. From josm ticket, i like map, osmbasedmap, photo, other For historicmap, is the date field not sufficient to define the historical character or not which can vary according to the use? EDIT: historic for josm ticket seems to mean not-the-lasted. maybe it's better to add a key for that or change historic into not-the-lasted to avoid confusion with the usual meaning of the word historical.

question remain for QA : is it a good idea to put them into osmbasedmap ? a layer which displays for example the error points found by Osmose, it is not really a map, but still osm-based. I hope that the discussions on the two projects will continue to be mutually enriching in order to have a common vision that will facilitate everyone's life.

Marc-marc-marc commented 6 years ago

so what ? what did you ting if :

we add a key category with value map, osmbasedmap (or osmbased ?), photo, other ? qa ?
we add not-the-lasted=no to allow app to hide or group previous versions of a layer ?

Klumbumbus commented 6 years ago

category was added just today to josm https://josm.openstreetmap.de/ticket/16103#comment:28 with the following supported values: photo aerial or satellite photo, map a map, historicmap historic or otherwise outdated map, osmbasedmap map based on OSM data, historicphoto historic or otherwise outdated aerial or satellite photo, other any other type

Marc-marc-marc commented 6 years ago

yes but as I said on josm ticket, mixing category and not-the-lasted is not a good idea and in addition to that, historic is not a good value-prefix for "not the last update available" it's why I propose another word but that still allow a eli<>josm (easy) match the question remain for layer like osmose : it's osmbased but not a map. should it be category=osmbased ? other ? qa ?

bhousel commented 6 years ago

category was added just today to josm https://josm.openstreetmap.de/ticket/16103#comment:28 with the following supported values: photo aerial or satellite photo, map a map, historicmap historic or otherwise outdated map, osmbasedmap map based on OSM data, historicphoto historic or otherwise outdated aerial or satellite photo, other any other type

@Marc-marc-marc this is a great start! I saw on that ticket that it looks like there is some confusion about what historic means. Could this category field just be used to store the type of imagery and instead use start/end dates to capture the "historic" or "not latest" quality?

Marc-marc-marc commented 6 years ago

yes you can of course use only start_date/end_date to find the lasted layer but it's hard to find witch layers cover the same feature or not. for ex :

layer id foo 2016 and layer id bar 2017 are the same feature for the end user or not ? for a human, it's easy to have an idea, for an app, it's not easy (you need to compare shapes for ex)
layers id municipaly1 date 2016 layer id municipality2 date 2016 layer id municipality1+2 date 2017 if you want to use only the lasted, it's not easy for a query to find that the municipaly1and municipality2 layers can be dropped due the fact the last one cover the same shape+feature.
how to find that another layer show not hidden because the feature are not the same (infrared <> true color) that need a lot of keys, a lot of work to fill all key....or put the result in not-the-lasted when an old layer have the same feature, nearly the same shape than a new one

bhousel commented 6 years ago

@Marc-marc-marc I'm confused what you mean by "which layers cover the same feature".. The three bullet points all relate to comparing geometry, and this doesn't seem like a very challenging issue. In fact using the imagery index in the first place requires an application to do spatial queries around the place of interest to know which layers apply there.

My plan for dealing with this in iD is to just throw the whole index into which-polygon (which is itself based on an rbush index) and ask it which layers cover the area where the user is looking.

Marc-marc-marc commented 6 years ago

one of the possibilities of a not-the-lasted tag would be to be able to list all available layers while keeping only the last one for each place (for example for each country or region). if you just want to keep the last one for on lon&lat, in this case you don't need not-the-lasted let's move forward without

and for category value ? do we need to make a separate category for the different osm-based rendering layers (for ex osmbasedmap) <> qa tools (for ex qa?) or we put all into a osmbased imho it's better to split render (an "display only" app can get those) and qa layer (usefull as overlay when editing the map)

simonpoole commented 5 years ago

I would suggest using the category values from JOSM plus a value for qa layers (regardless of source), could we get buy in from the JOSM devs @don-vip for a common value for that?

grischard commented 5 years ago

Replicating josm's values sounds good to me. I'm ok with having a 'qa' category if JOSM devs agree to have it too.

So as far as I can tell, we need to:

[x] find consensus on 'qa' with team JOSM
[x] update schema
[x] update conversion scripts on the ELI side
[x] ~update geojson conversion script on the JOSM side?~

don-vip commented 5 years ago

I'm ok with 'qa' :)

Klumbumbus commented 5 years ago

Recent discussions in JOSMs trac in https://josm.openstreetmap.de/ticket/18172

grischard commented 5 years ago

I'm cool with 'terrain' too if JOSM decides to have it.

simon04 commented 4 years ago

FYI, JOSM has added elevation and qa in https://josm.openstreetmap.de/changeset/15658/josm

grischard commented 4 years ago

All right, now let's get #733 ready :)

grischard commented 4 years ago

So we now need to update the conversion scripts but also @rbuffat's strict check script.

@Klumbumbus will the JOSM ImageryCompare compare the categories?

don-vip commented 4 years ago

@Klumbumbus will the JOSM ImageryCompare compare the categories?

Yes: https://josm.openstreetmap.de/changeset/15692/josm/

grischard commented 4 years ago

This is now a long ticket, so to summarise what's left to be done:

[x] update ELI conversion scripts
[ ] update the strict check.py

simonpoole commented 4 years ago

This is now a long ticket, so to summarise what's left to be done:
* [ ]  update ELI conversion scripts
Didn't I already do that (at least the PR was merged :-))? It doesn't make sense to update the legacy format one, because ... legacy format that doesn't include the category (and I suspect nobody actually uses it any more).
* [ ]  update the strict check.py

grischard commented 4 years ago

~convert_individual.py and convert_xml.py would need to understand the 'category' attribute too.~

Edit: just the strict check.py then

simonpoole commented 4 years ago

A I mentioned in my last PR we currently don't have agreement on how to categorize IR/NIR/CIR imagery see also https://github.com/osmlab/editor-layer-index/issues/325 @don-vip @Klumbumbus how are you categorizing these?

don-vip commented 4 years ago

What are "IR/NIR/CIR" layers?

simonpoole commented 4 years ago

Infrared and Near Infrared

imagico commented 4 years ago

As mentioned in https://github.com/osmlab/editor-layer-index/pull/324#issuecomment-300546509 all infrared aerial imagery in the index is currently false color images with a green-red-nir band combination, most infrared satellite imagery is a red-nir-swir combination.

Other possibilities for images based on infrared data could be monochrome NIR images or visualizations of computed values like NDVI.

The main use of storing such information in differentiated form would be to allow editors to show users specific instructions how to interpret the imagery.

Klumbumbus commented 4 years ago

@Klumbumbus how are you categorizing these?

As photo or historicphoto. If a photo is true or false color is not covered by the current category values.

Hedaja commented 4 months ago

Another good category would be DSM/DEM/DTM for (lidar-scannend) surface models. These are extremely helpful when mapping in forested areas.

simonpoole commented 4 months ago

"elevation' already includes those (l'm not sure why this issue hasn't long been closed).

osmlab / editor-layer-index

Better source categorization #136