Add ability to add an external WMS

[GoogleCodeExporter]

Add ability to manually add an external WMS from a list of WMS servers provided 
by MassGIS and also ability to manually enter a WMS server URL.

Original issue reported on code.google.com by emily.hu...@state.ma.us on 29 Jul 2011 at 2:57

[GoogleCodeExporter]

This involved major changes, so I hope I didn't break anything.

Committed revision 132.

There is a getcaps field below the available data layers list.

A few things:

* I'm not inclined to allow queries against external WMS-es.  (You'd have to 
proxy them anyway.)

* I'm not inclined to allow printing of external WMS-es.  Printing can be a lot 
less forgiving than the browser.  E.g. the external WMS-es are tiled (since 
some request bomb w/ a singleTile whose w x h is too large), but once you pass 
the request to the 
print engine, it wants to make them singleTile again.

* The getcaps window that appears (when successful) will attempt to pull down 
the zoom-to bbox as well as available projections.  The available projs will 
function just like our only_proj restrictions (when available).

Original comment by cpl...@gmail.com on 22 Sep 2011 at 2:55

• Changed state: Fixed

[GoogleCodeExporter]

I'm confused, I thought this ability was NOT going to be provided in 
OLIVER/MORIS but in a separate third-party application by ASA?  Adding external 
WMS here opens up 100 problems in the viewer.  Because external WMS may not 
support Mass. state plane, or query, etc, etc, how do we communicate that 
complexity to the user? 

Maybe I wasn't paying enough attention at the last conf. call...

Original comment by Aleda.Fr...@state.ma.us on 22 Sep 2011 at 3:03

[GoogleCodeExporter]

Wow.  Well, I missed that one too, then.  I agree that it opens up a whole lot 
of worms, but since it was on the SOW, I thought I needed to do it.  Should I 
roll back the changes and omit this until we regroup?

Original comment by cpl...@gmail.com on 22 Sep 2011 at 3:07

[GoogleCodeExporter]

I would actually.... to keep things simple... I'm more concerned right now 
about getting the extract by polygon to work... 

Original comment by Aleda.Fr...@state.ma.us on 22 Sep 2011 at 3:17

[GoogleCodeExporter]

Changes have been rolled back w/ the latest release.

Committed revision 133.

Original comment by cpl...@gmail.com on 22 Sep 2011 at 5:45

• Changed state: Accepted

[GoogleCodeExporter]

Some suggested addresses:

NOAA / BOEM : multipurpose marine cadastre
http://csc.noaa.gov/arcgis/rest/services/MMC/MultipurposeMarineCadastre/MapServe
r

NOAA : Marine Protected Areas 
http://egisws02.nos.noaa.gov/ArcGIS/rest/services/MPA

EPA: various datasets, including Permit Compliance System data (e.g., NPDES 
dischargers), Toxic Release Inventory data, etc.
http://www.epa.gov/geospatial/data.html
GIS Server Name: geodata.epa.gov

NOAA Charts WMS
http://egisws02.nos.noaa.gov/ArcGIS/rest/services/RNC

USFWS: wetlands data and status
http://www.fws.gov/wetlands/Data/WebMapServices.html
GIS Server Name: FWS_Wetlands_WMS
URL: http://137.227.242.85/ArcGIS/services/FWS_Wetlands_WMS/mapserver/wmsserver?

Multipurpose Marine Cadastre REST Services  NOAA Coastal Services Center

Original comment by daniel.s...@state.ma.us on 21 Oct 2011 at 8:53

[GoogleCodeExporter]

Just to make sure I get this right . . . you are going to provide a set of 
GetCapabilities links that you've vetted for reliability and projection 
support?  And my end will simply list these links (maybe similar to how I list 
the zoom-to scales) and a dialog box will popup where the user can double-click 
on a layer to have it added to the map.

Just a note of caution . . . w/i one getcaps bundle you may be presenting the 
user w/ layers you'd rather they not see, and I don't plan on filtering 
anything out.

I am not planning on support dynamic styles, and I'm not planning on supporting 
query.

Original comment by cpl...@gmail.com on 23 Nov 2011 at 2:45

[GoogleCodeExporter]

Here are a few WMS services to get you started. They serve data in the Web 
Mercator (Google) projection, EPSG:900913:

NOAA Electronic Navigational Charts (ENC)
http://ocs-spatial.ncd.noaa.gov/wmsconnector/com.esri.wms.Esrimap/encdirect?requ
est=GetCapabilities&service=WMS

USGS National Hydrography Dataset (NHD)
http://services.nationalmap.gov/arcgis/services/nhd/MapServer/WMSServer?request=
GetCapabilities&service=WMS

We're interested in a least a handful of others, but they're currently not 
being served in 900913 and 26986. That said I expect at least a couple to be 
available in 900913 in the near future. Will CZM/MassGIS be able to add these 
ourselves in the future? Also, some WMS services are offered in other version 
of Web Mercator (102113, 102100, and 3857; the latter of which was recently 
recognized by EPSG to be their official Web Mercator projection code). I 
believe it is identical to 900913, therefore would we be able to display 
external services in 3857 with our non-Custom basemaps? 

Here's an excerpt from 
http://docs.openlayers.org/library/spherical_mercator.html.
"Spherical Mercator has an official designation of EPSG:3857. However, before 
this was established, a large amount of software used the identifier 
EPSG:900913. This is an unofficial code, but is still the commonly used code in 
OpenLayers."

As an aside, it's my understanding that we'll be able use Cascading WMS to 
project data from external services once we upgrade to GeoServer v2.11. There 
may be other ways to do this as well. It's important that we have the ability 
to hardcode new WMS services as they become available, and for users to add 
custom services. 

Original comment by marc.car...@state.ma.us on 23 Nov 2011 at 3:09

[GoogleCodeExporter]

"...presenting the user w/ layers you'd rather they not see, and I don't plan 
on filtering anything out."

Do you mean that...

1) users will see all layers available in a service, but have the option of 
adding individual layers to the map; or 
2) users will not be able to add individual layers to the map; they will have 
to add the entire contents of a service?

I presume it's the former, in which case, that's fine.

Original comment by marc.car...@state.ma.us on 23 Nov 2011 at 4:41

[GoogleCodeExporter]

Yes, door #1.

Original comment by cpl...@gmail.com on 23 Nov 2011 at 10:11

[GoogleCodeExporter]

Right, 3857 == 900913.

My current line of thought (yes, there is only one), is that I'll put the WMS 
getcaps setup in moris.php, so that you can customize it for each deployment.

Original comment by cpl...@gmail.com on 23 Nov 2011 at 10:13

[GoogleCodeExporter]

I have ability in GeoServer to publish the additional projections as different 
numbers.
3857 is part of the GeoServer projections package.  We offer 900913 
additionally. 

Original comment by Aleda.Fr...@state.ma.us on 23 Nov 2011 at 10:18

[GoogleCodeExporter]

Aleda, add something like this to your .php's:

      var externalGetCaps = [
        {name : 'NOAA Electronic Navigational Charts (ENC)',url : 'http://ocs-spatial.ncd.noaa.gov/wmsconnector/com.esri.wms.Esrimap/encdirect?request=GetCapabilities&service=WMS&version=1.1.1'}
      ];

Underneath the bannerHeight var.

I think this external WMS ability will be very dicey.  I tried adding the NHD 
and couldn't get anything valuable to return, neither here in MORIS nor in 
uDig.  The LAND layer of the ENC works OK, though.

A few things to consider:

  * Zooming to layer.  The GetCaps includes the bbox for each added layer.  There is a strong likelihood that a bbox will be outside of the bbox constraints you have defined for each base layer in MORIS.  I'm not sure what should happen here.

  * Printing.  I have enabled printing of remote WMS-es.  Again, LAND is a good working example.

  * Permalinks.  I thought about adding them to permalink, but I think it's a bad idea.  If you were to add several remote WMS-es to a permalink, I'd need to pass along the full URL to each, and we could very easily run into URL size limitations.  Also, you as administrators may choose to +/- external servers which would create a disconnect between a permalink and what you really intend to offer.

  * General stability.  We have such tight control over MassGIS layers and so little, if any, over remote layers.  I would vote that you play around w/ adding and removing your desired WMS getcaps services (making sure to include the VERSION param like I have above) to see if you like what you are seeing, and then reevaluate whether or not to keep them.  Don't I hear rumblings of an eventual geoserver upgrade which would enable cascading wms-es?  That would be a far better approach in my opinion.

  * Troubleshooting.  As much as I'd like to help, I don't think I should be asked to troubleshoot why a potential WMS might not be working.  The NHD one, for example, seems to not like the SRS param even though what I see go by in firebug is perfectly valid.  Maybe that explains why I can't see anything in uDig, either.

v. 0.91
Committed revision 242.

Original comment by cpl...@gmail.com on 28 Nov 2011 at 5:42

[GoogleCodeExporter]

Can you add a screenshot of the LAND layer working for you?  I want to make 
sure I'm looking in the correct place on the map... 

Original comment by Aleda.Fr...@state.ma.us on 28 Nov 2011 at 6:10

[GoogleCodeExporter]

Nevermind, it just drew for me. 

Original comment by Aleda.Fr...@state.ma.us on 28 Nov 2011 at 6:11

[GoogleCodeExporter]

I thought the button was going to be over on the right with Available 
datalayers? I don't mind either way, except that if it's going to be on the 
toolbar I think we need to have it be show/hide-able in toolConfig_default.js.  
Because most viewers will not want this tool.  However, if it's a small button 
over on the right with Available datalayers I wouldn't care about hiding it I 
think.  

Original comment by Aleda.Fr...@state.ma.us on 28 Nov 2011 at 6:38

[GoogleCodeExporter]

Where it goes doesn't matter to me.  I just stuck it somewhere for testing.  
Please be my guest to move it around and add it as a button!

Original comment by cpl...@gmail.com on 28 Nov 2011 at 7:05

[GoogleCodeExporter]

Oh, I'm sorry.  I didn't have my brain turned on when I read your comment.  
'With the available datalayers.'  I'm on the same page now.  Sure, I can move 
it.  I'll move it tomorrow.

Original comment by cpl...@gmail.com on 29 Nov 2011 at 1:02

[GoogleCodeExporter]

So moved.  Aleda, you mentioned having this appear in toolConfig.  I don't know 
the mechanics behind that.  Since this is an entire toolbar, it might be a bit 
touchy.

v. 0.92
Committed revision 247.

Original comment by cpl...@gmail.com on 29 Nov 2011 at 5:55

• Changed state: Fixed

[GoogleCodeExporter]

Charlton, I see it on the right and it works there but it's still on the bottom 
toolbar as well.. could you remove it from there?  Thanks! 

I figured out a way to put a message in there if there isn't any external data: 

       var externalGetCaps = [
           {name : 'None Available',url : 'none'}
       ];

That's probably good enough.  

Original comment by Aleda.Fr...@state.ma.us on 29 Nov 2011 at 6:49

[GoogleCodeExporter]

Removed now.

v. 0.93
Committed revision 248.

Original comment by cpl...@gmail.com on 29 Nov 2011 at 7:47

[GoogleCodeExporter]

Charlton,

We added an ArcGIS Server WMS that we have here at MassGIS and it does produce 
the datalayer list and draws the layer in Custom (we set it up with 26986).  
However, when we zoom in the ArcGIS Server layer zooms in but our GeoServer 
layers don't.  

See the "before" and "after" screenshots.
Unfortunately you can't access this server, but I can also attach her 
GetCapabilities if that helps. 

We are working on finding an ArcGIS Server WMS that works that you can also 
access.

Original comment by Aleda.Fr...@state.ma.us on 29 Nov 2011 at 8:44

Attachments:

• custom_with_arcgisserver_wms_before_zoom.PNG
• custom_with_arcgisserver_wms_after_zoom.PNG
• MassGIS_ArcGISServer_WMS_26986.xml

[GoogleCodeExporter]

Well, that's weird.  This only does it for your test case?  I really can't 
debug unless I can hit a svc.

Original comment by cpl...@gmail.com on 29 Nov 2011 at 9:58

[GoogleCodeExporter]

I'm not sure if this is related, but I fixed a bug that was causing external 
WMS's not to display correctly when changing from a non-custom epsg to a custom 
one.

v. 0.94
Committed revision 249.

Original comment by cpl...@gmail.com on 29 Nov 2011 at 10:18

[GoogleCodeExporter]

I have some more info that maybe will help - if I uncheck and recheck the 
MassGIS layer it's OK. That's because apparently when the zoom is done no new 
WMS request is made, but when I uncheck and recheck a new one is made.  Does 
that sound like what should be happening? Normally when I look at a GeoServer 
WMS MassGIS layer in Custom it resends the wms request when I zoom in and not 
when I uncheck and recheck (so it's the other way around).

Original comment by Aleda.Fr...@state.ma.us on 29 Nov 2011 at 10:19

[GoogleCodeExporter]

I grabbed v 0.94 but it didn't have an effect on comment 24's problem. 

Original comment by Aleda.Fr...@state.ma.us on 29 Nov 2011 at 10:30

[GoogleCodeExporter]

Your explanation of how the visibility should be working is correct.  Maybe the 
checkbox and the layer's visibility falls out of sync and wakes up when you 
toggle it back and forth.  I hope that's not the case.  I really need to get my 
hands on an example that I can access, though.  It will probably be impossible 
to debug, otherwise.

Original comment by cpl...@gmail.com on 29 Nov 2011 at 10:30

[GoogleCodeExporter]

Can we get an ArcGIS Server admin at ASA to set up a simple 1 shapefile 26986 
WMS mapservice that you can access?  We can supply a simple shapefile of town 
boundaries for example.  MassGIS doesn't have an external ArcGIS Server now 
(and won't for months) and no one else serves 26986 data. 

Original comment by Aleda.Fr...@state.ma.us on 29 Nov 2011 at 10:32

[GoogleCodeExporter]

I'll ask.

Original comment by cpl...@gmail.com on 29 Nov 2011 at 10:37

[GoogleCodeExporter]

If that doesn't work I could share my screen with you with joinme and with 
Firebug you could see what's happening. 

Original comment by Aleda.Fr...@state.ma.us on 1 Dec 2011 at 3:03

[GoogleCodeExporter]

Could the max zoom level restriction for the Google basemaps be removed? When I 
click zoom to layer for an external WMS layer, it often zooms out to the world. 
This works well for the Bing and OSM basemaps because they can zoom out to the 
world, but the Google basemaps usually zoom to Europe or Africa because they 
can't zoom out that far.

Original comment by emily.hu...@state.ma.us on 2 Dec 2011 at 4:05

[GoogleCodeExporter]

Sure, but we went through a lot of headache to figure those #'s out.

If you want to experiment w/ how things would look w/ those changes, you can 
remove any reference to minZoomLevel and maxZoomLevel in blocks like these:

  lyrBase['googleSatellite'] = new OpenLayers.Layer.Google(
     'googleSatellite'
    ,{
       'sphericalMercator' : true
      ,type                : google.maps.MapTypeId.SATELLITE
      ,minZoomLevel        : 7
      ,maxZoomLevel        : 20
    }
  );

So that would turn into:

  lyrBase['googleSatellite'] = new OpenLayers.Layer.Google(
     'googleSatellite'
    ,{
       'sphericalMercator' : true
      ,type                : google.maps.MapTypeId.SATELLITE
    }
  );

If you like those changes, commit them!

Original comment by cpl...@gmail.com on 2 Dec 2011 at 5:35

[GoogleCodeExporter]

OK Charlton I'll make these changes locally and have people decide whether they 
like it. 

Original comment by Aleda.Fr...@state.ma.us on 2 Dec 2011 at 5:45

[GoogleCodeExporter]

External wms-es should be ok to check/uncheck all now.  (The peoplegis & 
harvard examples.)

v. 0.97
Committed revision 254.

Original comment by cpl...@gmail.com on 2 Dec 2011 at 5:47

[GoogleCodeExporter]

The remove all should be better now.

v. 0.98
Committed revision 255.

Original comment by cpl...@gmail.com on 2 Dec 2011 at 5:56

[GoogleCodeExporter]

Is comment 22 still an issue?  I haven't had much response re. spinning up an 
ims service w/ your sample data.

Original comment by cpl...@gmail.com on 2 Dec 2011 at 5:58

[GoogleCodeExporter]

Yes, comment 22 is still an issue for me.  I open MORIS, switch to Custom, add 
a layer from the ArcGIS Server we have here, it draws fine.  But then I zoom in 
on the map and while the ArcGIS Server zooms in, the other GeoServer layers 
don't, so there's a mismatch.  At that point there is an error in Firebug 
(attached).  If I uncheck and recheck the GeoServer layer in the Active 
datalayers area it draws properly. As I said in comment 25: apparently when the 
zoom is done no new GeoServer WMS request is made, but when I uncheck and 
recheck the GeoServer layer a new one is made.  Does that sound like what 
should be happening? Normally when I look at a GeoServer WMS MassGIS layer in 
Custom it resends the wms request when I zoom in and not when I uncheck and 
recheck (so it seems to be backwards here).

Original comment by Aleda.Fr...@state.ma.us on 2 Dec 2011 at 7:03

Attachments:

• arcgis_server_zoomin_error.PNG

[GoogleCodeExporter]

And it zooms / pans fine when you don't use the custom proj?

If so, do other wms providers fail similarly when in the custom proj?

Original comment by cpl...@gmail.com on 2 Dec 2011 at 7:10

[GoogleCodeExporter]

That's the tricky part.  This MassGIS ArcGIS Server is only available in 
Custom.  The other ArcGIS Server we've found is only available in Google - and 
the data's for some reason not drawing from that one.  

Original comment by Aleda.Fr...@state.ma.us on 2 Dec 2011 at 7:16

[GoogleCodeExporter]

We've also noticed that while Remove all works, uncheck all doesn't work for 
the MassGIS ArcGIS Server layer.  

Original comment by Aleda.Fr...@state.ma.us on 2 Dec 2011 at 7:40

[GoogleCodeExporter]

Can you attach the full getcaps response from the arcgis server?  If this is a 
bug in my code I need to fix it, but if it's something peculiar about that svc, 
then I may need to bow out gracefully. 

Original comment by cpl...@gmail.com on 2 Dec 2011 at 9:12

[GoogleCodeExporter]

Attached - a service with 3 layers.

Original comment by Aleda.Fr...@state.ma.us on 2 Dec 2011 at 9:15

Attachments:

• MassGIS_ArcGISServer_WMS_26986.xml

[GoogleCodeExporter]

That's very helpful.  The problem is the 3rd layer, and it has nothing to do w/ 
the projection.  It's the layer name.  My bet is that you have a real MassGIS 
data called GISDATA.TOWNS_POLY that is confusing the mapper.  The mapper thinks 
it has metadata to go along w/ this identically named layer when it really 
doesn't know a thing about it at all.  So now we know.  More.

Original comment by cpl...@gmail.com on 2 Dec 2011 at 9:31

[GoogleCodeExporter]

Drum roll please . . .

The problem is not the fact that we may or may not know about a layer called 
GISDATA.TOWNS_POLY in our usual slice_getcaps world.  No, the problem is 
actually much simpler than that.  The problem is that the WMS layer NAME is 0.  
As in ZERO.  As in something that is probably evaluating to FALSE in GeoExt 
land.

I'm pretty sure that all my code is treating all layer names as strings, but 
the .js error that we're seeing comes from GeoExt, and I'm suspicious that it 
is seeing the layer name as FALSE instead of ZERO and causing something to 
fail.  This is getting triggered on a zoom which causes the legend to be 
checked (which is where it's really failing).  This breaks all the other layers 
causing them not to zoom until you toggle them on/off to force a refresh.

For kicks, I went into GeoExt.js and tweaked the offensive line so that it 
would not get executed no matter what, and now the 0 layer draws w/o a problem. 
 So it does sound like a GeoExt bug.  But I'm not sure if you want to live w/ 
this 1-off edit of mine in GeoExt.js (which I have not committed) or simply 
warn against layers named 0 and/or wait for GeoExt to fix it.

Original comment by cpl...@gmail.com on 3 Dec 2011 at 1:23

[GoogleCodeExporter]

Yes, I guarantee it's a GeoExt bug.  I modified a stock example of theirs and 
posted a bug report to the list.  When their digest becomes available, I'll 
post a link here.

Original comment by cpl...@gmail.com on 3 Dec 2011 at 1:33

[GoogleCodeExporter]

http://www.geoext.org/pipermail/users/2011-December/002782.html

Original comment by cpl...@gmail.com on 3 Dec 2011 at 1:34

[GoogleCodeExporter]

Nice work Charlton.  I'm happy to live with your code change until the GoeEXT 
folks make a fix.  If that happens before the contract ends, it would be nice 
to roll the corrected code into MORIS so that future programmers have an easier 
time getting up to speed.

Original comment by daniel.s...@state.ma.us on 5 Dec 2011 at 1:41

[GoogleCodeExporter]

Just committed the fix.  It looks like this little piece of code has something 
to do w/ the legend panel.  So if something looks broken that used to work, 
I'll blame my 1-off fix and roll it back.  Fixing external bugs really 
shouldn't be in my bag of tricks, so if it really does cause problems, we'll 
need to wait and let the GeoExt family make the official fix.

Original comment by cpl...@gmail.com on 5 Dec 2011 at 2:05

[GoogleCodeExporter]

Good catch Charlton.  Yes, in the land of ArcGIS Server mapservices the layers 
are numbered, starting with 0.  

Original comment by Aleda.Fr...@state.ma.us on 5 Dec 2011 at 2:05

[GoogleCodeExporter]

Red exclamation points appear at the wrong time for external WMS layers. A 
26986 WMS layer should get a red exclamation when using a 900913 basemap, and 
vice versa. This should persist when switching between Custom and non-Custom 
basemaps, as a new GetMap request is issued.

Original comment by marc.car...@state.ma.us on 5 Dec 2011 at 9:24