all relevant vector modules should have cats and where parameters

[landam]

Reported by mlennert on 14 Nov 2012 17:23 UTC A series of vector modules should have cats and where parameters allowing the user to apply the operation only on selected features. Having this options avoids having to create many intermediate maps. And for a more homogeneous user experience it would be nice to have them in all modules, unless irrelevant.

Here's a tentative list of those modules that don't have them today in grass7 and that could potentially profit of having them:

* v.select: both for ainput and binput
* v.to.rast
* v.to.rast3
* v.to.3d
* v.distance: both for from and to
* v.net.alloc: it might be interesting to have a where option to complement ccats for selecting the nodes
* v.net.iso: it might be interesting to have a where option to complement ccats for selecting the nodes
* v.overlay: both for ainput and binput
* v.delaunay
* v.voronoi
* v.perturb
* v.class: has where, but not cats
* v.report
* v.univar: has where, but not cats
* v.qcount
* v.normal
* v.out.dxf
* v.out.gps: has where, but not cats
* v.out.ogr
* v.out.pov
* v.out.svg
* v.out.vtk
* v.parallel
* v.sample
* v.split
* v.surf.bspline
* v.surf.idw
* v.surf.rst: has where, but not cats
* v.vol.rst: has where, but not cats
* v.drape: has where, but not cats
* v.extrude
* v.kcv
* v. kernel
* v.neighbors

GRASS GIS version and provenance

svn-trunk

Migrated-From: https://trac.osgeo.org/grass/ticket/1798

[landam]

Comment by glynn on 14 Nov 2012 20:18 UTC Replying to [ticket:1798 mlennert]:

A series of vector modules should have cats and where parameters allowing the user to apply the operation only on selected features. Having this options avoids having to create many intermediate maps. And for a more homogeneous user experience it would be nice to have them in all modules, unless irrelevant.

Would it be feasible (and desirable) to allow the creation of "views" (virtual maps which obtain their data from an underlying map), similar to v.external or raster "reclass" maps?

This could avoid the need to implement these options explicitly for every vector module.

[landam]

Comment by hellik on 14 Nov 2012 21:59 UTC Replying to [comment:1 glynn]:

Would it be feasible (and desirable) to allow the creation of "views" (virtual maps which obtain their data from an underlying map), similar to v.external or raster "reclass" maps?

IMHO having some kind of "views" would be a nice feature.

Helmut

[landam]

Comment by mlennert on 15 Nov 2012 08:12 UTC Replying to [comment:1 glynn]:

Replying to [ticket:1798 mlennert]:

A series of vector modules should have cats and where parameters allowing the user to apply the operation only on selected features. Having this options avoids having to create many intermediate maps. And for a more homogeneous user experience it would be nice to have them in all modules, unless irrelevant.

Would it be feasible (and desirable) to allow the creation of "views" (virtual maps which obtain their data from an underlying map), similar to v.external or raster "reclass" maps?

I don't know enough about the "feasible" question, but I agree that it might be worth looking at this option. At the same time, I really appreciate the flexibility of doing the "on-the-fly" selection that the cats and where parameters allow.

Imagine the case where you have 100 points and want to create individual buffers around each, i.e. you can't call v.buffer on all at once as this will fusion the buffers. With views you have the extra step of having to create a view for each cat before calling v.buffer, with the cats/where options you can just call v.buffer. But I'm aware that we're talking luxury problems here, and that views that can be called like normal maps would allow us not to have to worry about this for each new module (and especially addons).

Moritz

[landam]

Comment by mmetz on 18 Nov 2012 16:30 UTC Replying to [comment:1 glynn]:

Replying to [ticket:1798 mlennert]:

A series of vector modules should have cats and where parameters allowing the user to apply the operation only on selected features. Having this options avoids having to create many intermediate maps. And for a more homogeneous user experience it would be nice to have them in all modules, unless irrelevant.

Would it be feasible (and desirable) to allow the creation of "views" (virtual maps which obtain their data from an underlying map), similar to v.external or raster "reclass" maps?

Technically, this would be possible, but it would require quite a few changes to the vector libs. Essentially, constraints would need to be implemented, similar to OGR filtering options.

IIUC, these virtual maps should behave just like normal maps: from a user perspective, it could look like

r.extract -v (-v as new flag to create a view or virtual vector)
v.buffer

instead of

r.extract
v.buffer

Disk space consumption would be lower for virtual maps, but so would be processing speed when working with a virtual map. I would rather go for processing speed.

Markus M

[landam]

Comment by hellik on 18 Nov 2012 19:22 UTC Replying to [comment:4 mmetz]:

IIUC, these virtual maps should behave just like normal maps: from a user perspective, it could look like

r.extract -v (-v as new flag to create a view or virtual vector)
v.buffer

instead of

r.extract
v.buffer

AFAIU a virtual map could be a "permanent" selection by cats/where, store this virtual map in the mapset and use this virtual map as input for processing by the v.*-modules.

v.make.virtual input=vector output=virtualmap cat=1-10 where A < 10
v.* input=virtualmap

Helmut

[landam]

Comment by mmetz on 19 Nov 2012 08:15 UTC Replying to [comment:5 hellik]:

Replying to [comment:4 mmetz]:

IIUC, these virtual maps should behave just like normal maps: from a user perspective, it could look like

v.extract -v (-v as new flag to create a view or virtual vector)
v.buffer

instead of

v.extract
v.buffer

[Sorry, I meant v.extract]

AFAIU a virtual map could be a "permanent" selection by cats/where, store this virtual map in the mapset and use this virtual map as input for processing by the v.*-modules.

v.make.virtual input=vector output=virtualmap cat=1-10 where A < 10
v.* input=virtualmap

The advantage of a virtual map over using v.extract to get a "permanent" selection by cats/where is not clear to me because the result of v.extract can be processed faster than a virtual map.

Markus M

[landam]

Comment by mlennert on 19 Nov 2012 16:48 UTC Replying to [comment:6 mmetz]:

Replying to [comment:5 hellik]:

Replying to [comment:4 mmetz]:

IIUC, these virtual maps should behave just like normal maps: from a user perspective, it could look like

v.extract -v (-v as new flag to create a view or virtual vector)
v.buffer

instead of

v.extract
v.buffer

[Sorry, I meant v.extract]

AFAIU a virtual map could be a "permanent" selection by cats/where, store this virtual map in the mapset and use this virtual map as input for processing by the v.*-modules.

v.make.virtual input=vector output=virtualmap cat=1-10 where A < 10
v.* input=virtualmap

The advantage of a virtual map over using v.extract to get a "permanent" selection by cats/where is not clear to me because the result of v.extract can be processed faster than a virtual map.

The only real advantage I can see is if whenever you modify the original map, your virtual map gets modified also (just like a database view). You then don't have to run v.extract each time. Also, you might consider it useful to avoid clutter if you can list views separately from vector maps (i.e. g.list view).

But, again, I personally think that cats/where parameters do solve much of the issue and a virtual map then would only save you the typing of these parameters.

Moritz

[landam]

Comment by glynn on 19 Nov 2012 18:15 UTC Replying to [comment:6 mmetz]:

The advantage of a virtual map over using v.extract to get a "permanent" selection by cats/where is not clear to me Disk usage. A virtual map wouldn't require an extra copy of the coordinates and topology.

because the result of v.extract can be processed faster than a virtual map. In the simplest case, a virtual map shouldn't be any slower than providing cats=/where= on the command line; they both do the same thing. There may be a difference in the case where you use where= when creating the virtual map and again when processing it (i.e. you'd have to "compose" the where= queries).

[landam]

Comment by hellik on 19 Nov 2012 18:36 UTC Replying to [comment:7 mlennert]:

The only real advantage I can see is if whenever you modify the original map, your virtual map gets modified also (just like a database view). You then don't have to run v.extract each time.

IMHO that would ease heavy vector analysis a lot. I'm often using this approach in some other GIS software ;o)

Also, you might consider it useful to avoid clutter if you can list views separately from vector maps (i.e. g.list view).

good idea

a virtual map then would only save you the typing of these parameters.

at least me being lazy and wingrass-GUI-oriented would be happy

Helmut

[landam]

Comment by hellik on 19 Nov 2012 18:39 UTC Replying to [comment:8 glynn]:

In the simplest case, a virtual map shouldn't be any slower than providing cats=/where= on the command line; they both do the same thing.

that would be also my understanding of a view/virtual map.

Helmut

[landam]

Comment by mmetz on 20 Nov 2012 09:24 UTC Replying to [comment:8 glynn]:

Replying to [comment:6 mmetz]:

The advantage of a virtual map over using v.extract to get a "permanent" selection by cats/where is not clear to me Disk usage. A virtual map wouldn't require an extra copy of the coordinates and topology.

With regard to vector processing, processing speed and memory consumption is in my experience more of an issue than disk space.

because the result of v.extract can be processed faster than a virtual map. In the simplest case, a virtual map shouldn't be any slower than providing cats=/where= on the command line; they both do the same thing.

The idea of a virtual map is to avoid adding cats/where options to all vector modules. For a vector module that does not have cats/where options, you would need to either create a virtual map or run v.extract. Then you would run a vector module without providing cats/where options, and using the result of v.extract would be faster.

There may be a difference in the case where you use where= when creating the virtual map and again when processing it (i.e. you'd have to "compose" the where= queries).

That should not be a problem because the where constraints of the virtual map would need to be considered on library level whereas the where constraints provided as an option to the module would be processed on module level.

For the implementation of virtual maps, it should IMHO be avoided that every single vector module needs to be modified. Vector modules use the vector library to retrieve a feature (point, line, boundary, centroid, area, island, face, node) from a vector. The library would then need to check if the feature matches the filtering criteria. This will require quite a lot of modification to the vector libraries, taking special care of boundaries and islands. For example, region constraints are already built in for sequential reading of lines, but unfortunately with a bug because boundaries are skipped if they are outside the constraint bounding box, even if one of the areas formed by this boundary overlaps with the constraint bounding box, resulting in corrupted areas.

In short, I am not against virtual vector maps, but I am a bit intimidated by the estimated amount of modifications needed on library level, and because I am not sure if it can be avoided that every single vector module needs some modification.

Markus M

[landam]

Comment by mlennert on 21 Nov 2012 08:05 UTC Replying to [comment:11 mmetz]:

Replying to [comment:8 glynn]:

Replying to [comment:6 mmetz]:

The advantage of a virtual map over using v.extract to get a "permanent" selection by cats/where is not clear to me Disk usage. A virtual map wouldn't require an extra copy of the coordinates and topology.

With regard to vector processing, processing speed and memory consumption is in my experience more of an issue than disk space.

because the result of v.extract can be processed faster than a virtual map. In the simplest case, a virtual map shouldn't be any slower than providing cats=/where= on the command line; they both do the same thing.

Yes, but does it in an on-the-fly, through away the results way, while the other keeps the results. For me personally, I mostly need the on-the-fly way and this means that there is no advantage of virtual maps over cats/where parameters. I actually would have the extra step of creating and removing the virtual map after usage, i.e. 3 commands instead of 1.

When I need to use the same extraction of a vector map over and over again, I prefer to use v.extract.

The idea of a virtual map is to avoid adding cats/where options to all vector modules. For a vector module that does not have cats/where options, you would need to either create a virtual map or run v.extract. Then you would run a vector module without providing cats/where options, and using the result of v.extract would be faster.

I see two advantages of virtual maps:

1) When they are seen like views in databases, i.e. a filter allowing to view particular features of a constantly evolving vector map. In that sense they help you save having to type the cats/where option every time you need to work with the relevant features.

2) That(If) they could allow to do more than where/cats parameters if virtual maps can also be created through v.select equivalents, i.e. selecting not only based on cats and attributes, but also based on location relative to another vector map. Then again, maybe it would be fairly straightforward to include a "vector_filter' option next to cats and where ?

I've not encountered 1) very often in my personal work. 2) is a regular need.

For the implementation of virtual maps, it should IMHO be avoided that every single vector module needs to be modified.

That was the main idea behind Glynn's proposal IIUC.

In short, I am not against virtual vector maps, but I am a bit intimidated by the estimated amount of modifications needed on library level, and because I am not sure if it can be avoided that every single vector module needs some modification.

I cannot judge this, so I'll leave this reflection up to those who can. IIUC, you have some boilerplate code that can be inserted into vector modules quite easily to add cats/where parameters. If that is the case, and you can explain to me how to do it, then I can spend some time adding this to modules. We can then still discuss whether virtual maps are a good and feasible idea.

Moritz

[landam]

Comment by mmetz on 23 Nov 2012 19:54 UTC Replying to [comment:12 mlennert]:

IIUC, you have some boilerplate code that can be inserted into vector modules quite easily to add cats/where parameters. If that is the case, and you can explain to me how to do it, then I can spend some time adding this to modules. We can then still discuss whether virtual maps are a good and feasible idea.

Here is a template:

    /* define variables */
    struct cat_list *cat_list = NULL;

    /* parse options */

    /* new: set category constraints */
    if (field > 0)
    cat_list = Vect_cats_set_constraint(&In, field, where_opt->answer,
                                            cats_opt->answer);

    for (line = 1; line <= nlines; line++) {

        Vect_read_line(&In, Points, Cats, line);

        /* new: check if categories are within constraints */
        if (field > 0 && !Vect_cats_in_constraint(Cats, field, cat_list))
            continue;

        /* process line */
    }

    for (area = 1; area <= nareas; area++) {

        Vect_get_area_cats(&In, area, Cats);

        /* new: check if categories are within constraints */
        if (field > 0 && !Vect_cats_in_constraint(Cats, field, cat_list))
            continue;

        /* process area */
    }

I hope the template is clear enough. This is already implemented in v.buffer and v.hull.

Markus M

[landam]

Comment by mmetz on 15 Dec 2012 23:23 UTC Replying to [comment:13 mmetz]:

Replying to [comment:12 mlennert]:

IIUC, you have some boilerplate code that can be inserted into vector modules quite easily to add cats/where parameters. If that is the case, and you can explain to me how to do it, then I can spend some time adding this to modules.

v.to.rast got cats and where options in https://trac.osgeo.org/grass/changeset/54301.

Markus M

[landam]

Comment by lucadelu on 7 Jan 2013 08:25 UTC I would like also to see where option in v.dissolve, is it possible?

[landam]

Comment by mmetz on 7 Jan 2013 16:07 UTC Replying to [comment:15 lucadelu]:

I would like also to see where option in v.dissolve, is it possible?

First, v.reclass would need cats and where options, then these options can be added to v.dissolve as well.

[landam]

Modified by @landam on 12 May 2016 06:43 UTC

[landam]

Modified by @landam on 25 Aug 2016 15:51 UTC

[landam]

Comment by @landam on 27 Aug 2016 13:42 UTC Milestone renamed

[landam]

Comment by neteler on 26 Jan 2018 11:40 UTC Ticket retargeted after milestone closed

[landam]

Modified by neteler on 12 Jun 2018 20:48 UTC

[landam]

Comment by @landam on 25 Sep 2018 16:51 UTC All enhancement tickets should be assigned to 7.6 milestone.

[landam]

Comment by @landam on 25 Jan 2019 21:07 UTC Ticket retargeted after milestone closed