Features in the SJTSK/Krovak East North (ESPG 5514) coor. sst. are not shown in the map

[kamil-novak]

I have created a Feature Service that references points in the S-JTSK / Krovak East North coordinate system. This Feature Service is available at standard URL http://server/koop/rest/services/param/FeatureServer/0/query. Features have standard JSON structure and standard xy values for S-JTSK:

{"attributes": {"ID":1,"CISLO":1}, "geometry":{"x":-791364.2178699165,"y":-989431.1763396218}}

I have a problem that my features are not visible in the webmap. I see this features in attribute table but not in the map.

But if I set geometry filter filtersApplied.geometry = true in the Koop for this data, then I see my features in the webmap. In this case the problem is that any spatial URL query returns all features in the Feature Service.

Is there any way how to work correctly with S-JTSK / Krovak East North coordinate system?

Thanks

[rgwozdz]

Hello @wellbloud-cz - GeoJSON is suppose to use WGS84 spatial reference, and as a result Koop expects the GeoJSON delivered by your Provider to have WGS84 coordinates. Koop doesn't provide any on-the-fly translation if your provider data doesn't arrive as WGS84.

I'm not sure why filtersApplied.geometry = true results in data getting rendered on you web map. That parameter simply tells winnow that it doesn't need to apply the geometry filter, because it has already been applied.

What we generally suggest is that you convert your data to WGS84 in your provider before passing it into the getData callback.

[kamil-novak]

Hello @rgwozdz, thank you for your reply.

We have the opposite problem. Our data-provider arrives it as WGS84 and we would like to show it as S-JTSK. So I convert this data to S-JTSK before passing it into the getData callback function.

But I guess according to your reply that there is no way to do that?

[rgwozdz]

What web map client are you using? Does it accept data that has spatial reference other that WGS84 Web Mercator (Auxiliary Sphere)?

[kamil-novak]

I am using Map Viewer on ArcGIS Online. There I have created webmap with S-JTSK basemap.

Maybe one more thing is important. I have to set filtersApplied.projection = true, too. If filtersApplied.projection = false, there is the same problem. In summary, I need to set filtersApplied: { geometry: true, projection: true }, then I can display features in the map, but spatial query works incorrectly.

Without filters: but in map client...

With filters:

[rgwozdz]

Ok, that's helpful. So, yes, if you send your data as non-WGS84 you need filtersApplied.projection = true. The spatial query probably doesn't work because Koop is trying to apply it to dataset that it assumes is in WGS84.

So I think what you might have to do is re-implement the spatial filter in your provider so that it understands which projection the source data is in. You can pull winnow into the provider and use it do the filtering. Just be sure to set options.sourceSR to the appropriate wkid.

I'll admit this workaround is not very desirable. I'm going to discuss with colleague so options for streamlining.

[kamil-novak]

Thanks for reply. I tried to pull winnow into the provider and I tried to use winnow.query(features, options) method according to documentation. I can influence some parametres as where and so on. But I didn't find sourceSR in the documentation. So I tried to set projection: 5514 or projection: 102067 (S-JTSK), but in this case there is bad request error in webmap client.

[rgwozdz]

Yes, I need to PR the docs to add sourceSR, but look at the changelog here. I think you will want to use the projections WKT as the value of sourceSR.

[kamil-novak]

OK, I tried to set sourceSR, but it seems that this property is not implemented in my Koop / winnow. Koop doesn't read this property. I am using winnow in version 1.16.9.

[rgwozdz]

@wellbloud-cz - I experimented with this and have some good news to report. First, I think you don't have to bother pulling winnow into your provider. Just add these query parameter to your request object prior to the get data callback:

    req.query.sourceSR = '<WKT of the source data's spatial reference here>'
    req.query.outSR = '<WKT of the your desired output spatial reference>' // can be same as sourcSR

    callback(null, geojson)

With the req.query.sourceSR and req.query.outSR set, I was able to successfully use the geometry filter AND receive data in a non-WGS84 projection.

[kamil-novak]

@rgwozdz - thank you very much for your information. I tried many settings:

req.query.sourceSR = 'GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]]';
req.query.outSR = 'GEOGCS["WGS 84",DATUM["WGS_1984"SPHEROID["WGS 84",6378137,298.257223563,AUTHORIT["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIME["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree"0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORIT["EPSG","4326"]]';

or

req.query.sourceSR = 'GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]]';
req.query.outSR = 'proj4.defs("EPSG:5514","+proj=krovak +lat_0=49.5 +lon_0=24.83333333333333 +alpha=30.28813972222222 +k=0.9999 +x_0=0 +y_0=0 +ellps=bessel +towgs84=589,76,480,0,0,0,0 +units=m +no_defs");';

or

req.query.sourceSR = 5514
req.query.outSR = 5514

or

req.query.sourceSR =102067
req.query.outSR = 102067

or

req.query.sourceSR = 4326
req.query.outSR = 102067

or

req.query.sourceSR = 4326
req.query.outSR = 5514

with or without filter geometry and projection.

But features are displayed in the map only if geometry and projection are set to true . And if is geometry = true, then server side spatial query in client app (e.g. ArcGIS API for Javascript app with webmap from ArcGIS Online) doesn't work. Here is example:

http://localhost:3002/koop/rest/services/mhd/FeatureServer/0/query?f=json&geometry={"spatialReference":{"latestWkid":5514,"wkid":102067},"xmin":-791276.1168079382,"ymin":-990623.8865756259,"xmax":-790053.1243553757,"ymax":-989400.8941230633}&maxRecordCountFactor=3&outFields=CISLO,OBJECTID&outSR=102067&resultType=tile&returnExceededLimitFeatures=false&spatialRel=esriSpatialRelIntersects&where=1=1&geometryType=esriGeometryEnvelope&inSR=102067

[rgwozdz]

@wellbloud-cz - thanks very much for your patience. I would really like to find the way forward on this - it's actually a priority for our Koop roadmap. Would you be willing to share your Code Pen and some sample data with me?

Also, can you clarify a bit more regarding your last screen grab? You noted that the spatial query doesn't work, however I see features on the map and in query response. I see the popup box has no content though. I wonder if we are thinking of the same thing with regards to "spatial query". To me, a spatial query in Koop refers to a request with a geometry query param that limits the data returned according the extent of the geometry value.

It looks like the data is there, but feature attributes are not rendering in the popup on click.

[kamil-novak]

Hello @rgwozdz, thanks for your help!

I created my experimental data as public.

My experimental provider service with WGS coordinates is here: https://mapytest.mesto-most.cz/xmlserver/xml Experimental Koop feature service with S-JTSK transformation is here: https://mapytest.mesto-most.cz/koop/koop/rest/services/mhd/FeatureServer/0/ Webmap: http://arcg.is/HDiH8 Code Pen: https://codepen.io/wellbloud/pen/RwwEjMY

I suppose that SpatialQuery doesn't work correctly becasuse if I want to identify feature (for showing popup), server returns all features in feature service (popups are created for all features in the map). Here is request example for spatial identification feature (click event in the map):

https://mapytest.mesto-most.cz/koop/koop/rest/services/mhd/FeatureServer/0/query?f=json&geometry={"spatialReference":{"latestWkid":5514,"wkid":102067},"xmin":-791174.610076911,"ymin":-989667.0313657958,"xmax":-791142.8600134109,"ymax":-989635.2813022956}&outFields=CISLO,ID,OBJECTID&outSR=102067&returnM=true&returnZ=true&spatialRel=esriSpatialRelIntersects&where=1=1&geometryType=esriGeometryEnvelope&inSR=102067

You can see that every extent parameter in url returns all features.

But attribute query works good (it returns only one feature): https://mapytest.mesto-most.cz/koop/koop/rest/services/mhd/FeatureServer/0/query?where=ID=3

[kamil-novak]

Hello @rgwozdz, I would like ask you if there is some new knowledge in this issue or if my previous answer is not understandable (I can correct some information). Thank you.

[rgwozdz]

Hello @wellbloud-cz - thank you for reaching out again and sorry for the slow response. Just looking at this now and testing locally. I think my intially suggested workaround will not work as I hoped. But there may be another option in the short-term; Let me clarify and explain:

For WKID 5514 or 102067, you should be using this WKT (it is from http://epsg.io/5514):

PROJCS["S-JTSK / Krovak East North",
    GEOGCS["S-JTSK",
        DATUM["System_Jednotne_Trigonometricke_Site_Katastralni",
            SPHEROID["Bessel 1841",6377397.155,299.1528128,
                AUTHORITY["EPSG","7004"]],
            TOWGS84[589,76,480,0,0,0,0],
            AUTHORITY["EPSG","6156"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4156"]],
    PROJECTION["Krovak"],
    PARAMETER["latitude_of_center",49.5],
    PARAMETER["longitude_of_center",24.83333333333333],
    PARAMETER["azimuth",30.28813972222222],
    PARAMETER["pseudo_standard_parallel_1",78.5],
    PARAMETER["scale_factor",0.9999],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["X",EAST],
    AXIS["Y",NORTH],
    AUTHORITY["EPSG","5514"]]

Meaning, you should set your sourceSR that WKT:

req.query.sourceSR = `PROJCS["S-JTSK / Krovak East North",GEOGCS["S-JTSK",DATUM["System_Jednotne_Trigonometricke_Site_Katastralni",SPHEROID["Bessel 1841",6377397.155,299.1528128,AUTHORITY["EPSG","7004"]],TOWGS84[589,76,480,0,0,0,0],AUTHORITY["EPSG","6156"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4156"]],PROJECTION["Krovak"],PARAMETER["latitude_of_center",49.5],PARAMETER["longitude_of_center",24.83333333333333],PARAMETER["azimuth",30.28813972222222],PARAMETER["pseudo_standard_parallel_1",78.5],PARAMETER["scale_factor",0.9999],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["X",EAST],AXIS["Y",NORTH],AUTHORITY["EPSG","5514"]]`

If your req.query.outSR arrives as ether 5514 or 102067, your should also set your outSR:

req.query.outSR = `PROJCS["S-JTSK / Krovak East North",GEOGCS["S-JTSK",DATUM["System_Jednotne_Trigonometricke_Site_Katastralni",SPHEROID["Bessel 1841",6377397.155,299.1528128,AUTHORITY["EPSG","7004"]],TOWGS84[589,76,480,0,0,0,0],AUTHORITY["EPSG","6156"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4156"]],PROJECTION["Krovak"],PARAMETER["latitude_of_center",49.5],PARAMETER["longitude_of_center",24.83333333333333],PARAMETER["azimuth",30.28813972222222],PARAMETER["pseudo_standard_parallel_1",78.5],PARAMETER["scale_factor",0.9999],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["X",EAST],AXIS["Y",NORTH],AUTHORITY["EPSG","5514"]]`

This should make requests WITHOUT spatial queries work. Please give this a try.

However, spatial queries on a data source not in WGS84 present problems. Anytime there is spatial query using a geometry with your specialized projection you also need to set the following parameters to the noted WKT: req.query.geometry.spatialReference.latestWkid req.query.geometry.spatialReference.wkid req.query.inSR

By setting those, your spatial filter should work. But there is one final problem, and that is the conversion to Esri JSON includes a reprojection function that assumes the source data is in WGS84, and currently doesn't respect sourceSR. As a result geometry values come back null. I'm going to try to PR this ASAP, but it might take a week or more before it is complete.

The only workaround I can suggest is to reproject your data to WGS84 in getData function (and remove the assignment of sourceSR). You would still need to assign the WKT if spatial queries arrive with an using your special projection. Psuedo-code below:

// fetch data from source

// reproject data to WGS84 using proj4

if (req.query.outSR === 5514 || req.query.outSR === 102067) {
  req.query.outSR = `PROJCS["S-JTSK / Krovak East North",GEOGCS["S-JTSK",DATUM["System_Jednotne_Trigonometricke_Site_Katastralni",SPHEROID["Bessel 1841",6377397.155,299.1528128,AUTHORITY["EPSG","7004"]],TOWGS84[589,76,480,0,0,0,0],AUTHORITY["EPSG","6156"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4156"]],PROJECTION["Krovak"],PARAMETER["latitude_of_center",49.5],PARAMETER["longitude_of_center",24.83333333333333],PARAMETER["azimuth",30.28813972222222],PARAMETER["pseudo_standard_parallel_1",78.5],PARAMETER["scale_factor",0.9999],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["X",EAST],AXIS["Y",NORTH],AUTHORITY["EPSG","5514"]]`
}

// Check for spatial query in your specific project
const { geometry = {}, inSR } = req.query
const { spatialReference: { latestWkid, wkid } = {} } = geometry
const geomFilterSRs = [latestWkid, wkid, inSR]
if (geomFilterSRs.includes(102067) || geomFilterSRs.includes(5514)) {
  delete req.query.geometry.spatialReference
  req.query.inSR = `PROJCS["S-JTSK / Krovak East North",GEOGCS["S-JTSK",DATUM["System_Jednotne_Trigonometricke_Site_Katastralni",SPHEROID["Bessel 1841",6377397.155,299.1528128,AUTHORITY["EPSG","7004"]],TOWGS84[589,76,480,0,0,0,0],AUTHORITY["EPSG","6156"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4156"]],PROJECTION["Krovak"],PARAMETER["latitude_of_center",49.5],PARAMETER["longitude_of_center",24.83333333333333],PARAMETER["azimuth",30.28813972222222],PARAMETER["pseudo_standard_parallel_1",78.5],PARAMETER["scale_factor",0.9999],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["X",EAST],AXIS["Y",NORTH],AUTHORITY["EPSG","5514"]]`
}

[kamil-novak]

Hello @rgwozdz, I definitely confirm yout described workaround. It works. Thanks a lot. Other problem is that if I set cache geojson.ttl = 10, spatial query in webmap often doesn't return any features.

I have prepared experimental node.js server, which provides new xml data every three seconds. XML service is here https://mapytest.mesto-most.cz/xmlserver/xml (this server rotates three data sets - first with features IDs 1,2,3; second with IDs 1,2,3,4,5,6 and last with IDs 3,4,7,8,9).

Webmap http://arcg.is/HDiH8 has set refresh interval = 6 seconds. Koop JSON service is refreshed every 10 second and provides good data https://mapytest.mesto-most.cz/koop/koop/rest/services/mhd/FeatureServer/0/query.

Can it be related to problem with reprojection function?

Thanks

[rgwozdz]

I tried loading your web map, and received errors about it not being able to load the basemap.

That is odd about the cache setting. Just to confirm, you are saying if you define geojson.ttl requests with a spatial query do not return any features? It's hard to imagine how the cache would break spatial filitering.

[kamil-novak]

I am sorry for problem with webmap, but the basemap should work. Could you tray to load once more, please.

Yes, I confirm that I have define geojson.ttl and in many cases webmap requests do not return any features.

[rgwozdz]

Still not able.

[kamil-novak]

I don't understand it, but I changed the basemap and here is full URL to webmap: https://www.arcgis.com/home/webmap/viewer.html?webmap=1001ec54568c4a79964445760c797d32

I tried to open it using VPN and anonymous tab and it work.

[kamil-novak]

Hello @rgwozdz, I just would like close this topic. Workaround with your "Pseudo-code" is working. Cache issue was caused by our settings of communication between IIS and Node.js. Thanks a lot.

[rgwozdz]

Ok, by the way, we have recently added a new feature to make these kind of workarounds easier; check out Provider before and after transformation function options. These are functions you can pass into provider registration and allow you to execute code before or after the getData method. With these, you can transform coordinates or fetch WKTs without altering a providers getData method.

[kamil-novak]

Hello @rgwozdz, Can I ask you one more question about this issue. After using your workaround is all right. But now I have noticed that in ArcGIS Dashboards (web version) I can't display labels for Koop feature layer with this workaround. When I cancel this workaround, labels are displayed. This issue is only in Dashboard application. Other interfaces (Map Viewer 3.x or 4.x, application in JS API 4.x) are OK. Is there any solution for this? Thanks

[kamil-novak]