[WFS-Client] WFS client does not keep field type

[tbonfort]

Reporter: jlacroix Date: 2004/11/08 - 22:09

The WFS client read the GML returned by a GetFeature. It use OGR to read it.
Since we do not pass a schema with the GML, OGR try to build its own schema
file. For that it guess the field type. The problem is that it is not always
accurate.

I have some mapinfo file here. Those have attributes set to Char(2). Those
attributes contains different values, but most of them are simple numbers.
Others are like "9+" or "1." (without double quotes). My problem is that the
schema created by OGR treat them as integer.

My attribute named "Ct" contains "9+" (I isolated to have only one row) and the
gfs file contains:
    <PropertyDefn>
      <Name>Ct</Name>
      <ElementPath>Ct</ElementPath>
      <Type>Integer</Type>
    </PropertyDefn>

This can cause a problem since I want to filter my classes and styles on the Ct
attribute. Since I don't have the right values returned, I can't filter correctly.

One simple solution could be to have an option to return everything as string
(if it is possible). Or to return every fields that are not normal numbers as
string.

Whadayathink?

[tbonfort]

Author: dmorissette Date: 2004/11/17 - 00:14

Any idea on this Frank?

[tbonfort]

Author: fwarmerdam Date: 2004/11/17 - 02:51

JS / Daniel, 

I can tighten up the condition on what qualifies as a numeric field a bit.
The analysis of the field types is done in GMLPropertyDefn::AnalysePropertyValue()
in gdal/ogr/ogrsf_frmts/gml/gmlpropertydefn.cpp.  

Currently if a field is made up entirely of characters that can occur in 
numbers (including scientific format) then it is treated as numeric.  This can 
mean that various forms like "9+" or even a field that has only "e", or "d" in
it could be treated as numeric.  I could tighten up some of the scientific 
notation related stuff, including tests that somethink like a "+" only occurs
before an exponent, or that a "-" only occurs at the beginning or before an 
exponent.  However, this would still potentially identify some fields as 
numeric that aren't really intended to be numeric.  

On the other hand, if you know the specific schema that you will be dealing
with you can actually pre-associate a .gfs file with the file.  I gather that
your issue is when OGR is used to parse GML read from a WFS server in MapServer,
is that right?  This is a case that where it wouldn't be very easy to use
a .gfs file to preset the schema. 

[tbonfort]

Author: dmorissette Date: 2004/11/17 - 05:28

Is the .gfs file a regular GML schema? Would it be possible to use the GML
schema obtained using DescribeFeatureType as the .gfs file?

Maybe we could have a option in OGR where we tell the GML driver to treat all
attributes as strings? That would be good enough for all MapServer uses.

Julien: do you know if it would be enough to just improve the detection of
numeric fields?

[tbonfort]

Author: jlacroix Date: 2004/11/17 - 16:41

I'm not sure the improvement of the numeric field detection will worth the
effort. It depends on Frank and how much work it is. In the specific test case
that I had, there's was "9+", but there was also "1.". Do you think your
enhancement will also detect "1." as a string? If yes and if it's a quick fix,
then go ahead. Maybe trying to pass everything as a string will be easier...But
the cleanest way to fix this would be to pass a .gfs since the server is
suppossed to know the type of it's fields. Can MapServer do that? In the Vendor
Specific Capabilities?

[tbonfort]

Author: fwarmerdam Date: 2004/11/17 - 17:05

Julien, 

"1." is a numeric value, and would not be classified as a string.  The "9+" would.

Danmo, 

The .gfs file is not an XML schema.  It is a custom OGR specific XML file 
holding various information normally computed in OGR's first pass over the GML
file.  That is layer definitions, extents, feature counts, field types, etc. 

I could fairly easily allow all fields in an GML file to be treated as strings,
but I am not exactly sure how we would pass such information in.  Perhaps
via a config variable?  We could have mapogr.cpp always call
CPLSetConfigOption("GML_FIELDTYPES","ALWAYS_STRING").   If you are good with 
that let me know and I will implement it.   It will also (slightly) improve
GML reading performance when enabled. 

[tbonfort]

Author: dmorissette Date: 2004/11/17 - 17:47

CPLSetConfigOption("GML_FIELDTYPES","ALWAYS_STRING") works great for me... and
if that improves performance then that's even better.

I assume that this would apply only to the automated generation of the .gfs
file, and if someone provides their own .gfs file with a GML file (not WFS) then
it would take precedence.

[tbonfort]

Author: fwarmerdam Date: 2004/11/17 - 18:18

The change is in GDAL/OGR CVS now.  I have not made the corresponding change
in mapogr.cpp ... I will leave that to you Julien. 

[tbonfort]

Author: dmorissette Date: 2004/11/17 - 21:30

Thanks Frank. I'll add this to mapogr.cpp... if I can ever compile the latest
GDAL from CVS.

[tbonfort]

Author: dmorissette Date: 2004/11/17 - 21:47

I have committed the call in CVS. Julien will test with the dataset that caused
the problem.

[tbonfort]

Author: jlacroix Date: 2004/11/17 - 22:01

Thanks guys! it works great. :)