Species checklist for Leo for alpine areas

Original Issue  - 
https://code.google.com/p/alageospatialportal/issues/detail?id=632

Project Member Reported by leebel...@gmail.com, Aug 30, 2011 
When we are onto the new BioCache...

Leo Lymburner would appreciate a species list for the Alpine and Sedges classes 
in the Dynamic Landcover layer. 
 Oct 13, 2011 Project Member #1 leebel...@gmail.com 
Waiting on dynamic land cover layer on dev/prod. Upping to high as this layer 
is one of the most valuable we have.
 Owner: adam_col...@tpg.com.au Labels: -Priority-Medium Priority-High Oct 13, 2011 Project Member #2 leebel...@gmail.com 
adding self
 Cc: leebel...@gmail.com Oct 30, 2011 Project Member #3 adam_col...@tpg.com.au 
Added support for grid files as contextual layers.  dlcmv1 now operates 
correctly in layers-service on dev.
1. shape/ services are used to download an object area.  This needs a size 
limit and a message instead of the shape.  It is unreasonable to allow access 
to 500MB+ shapes.
2. Shapes produced from the grid file are accessed by a constructed id.  I do 
not think individual polygons generated from a grid require a pid.  

 Status: Started Oct 30, 2011 Project Member #4 leebel...@gmail.com 
Dev: DLC currently reports "Layer unavailable"
 Apr 11, 2012 Project Member #5 adam_col...@tpg.com.au 
Update.  Only absent from tabulation and Add to map | gazetteer polygon.
 Cc: -gavin.ja...@gmail.com Apr 11, 2012 Project Member #6 leebel...@gmail.com 
Attempting to select alpine area (Add to Map | Areas | Select from mapped 
contextual | DLC. Got

The server is temporarily out of service.
Would you like to try again?

(404: Not Found)
 Apr 11, 2012 Project Member #7 leebel...@gmail.com 
It did map after error reported. Assumed classes would be in gaz but appears 
not. Would be very useful if they were - otherwise can't answer Leo's question.

"Primarily Vegetated Natural & Semi-Natural Terrestrial Vegetation Herbaceous 
Graminoids Open Alpine"
 Cc: -ajay.ranipeta chris.fl...@gmail.com Apr 11, 2012 Project Member #8 leebel...@gmail.com 
Just noted comment 5. Sorry.
 Apr 20, 2012 Project Member #9 adam_col...@tpg.com.au 
Changing to InTest for the gaz polygon functionality.  e.g. search for 'open 
alpine'.

These areas are as functional as environmental envelopes.  They do require the 
sampling in biocache for operations on biocache occurrence records.  This will 
be in the next batch.
 Status: InTest Apr 25, 2012 Project Member #10 leebel...@gmail.com 
This is still busted. Use gaz search on "alpine open" and multi-polygon is 
created by Area report won't work due to layer not indexed in BioCache. Adding 
Natasha. I'd like this done asap as Leo has been a fantastic supporter for the 
Atlas - and it has been on the books since August 2011!

The Dynamic Land Cover layer is one of the most important in the SP. We need it 
fully operational asap. ToCritical.

Also have a problem if I attempt an area from polygonal layer option on one 
alpine area. Spins wheels then produces error 404. Adam's problem.
 Status: Started Cc: natasha....@csiro.au Labels: -Priority-High Priority-Critical Apr 25, 2012 Project Member #11 natasha....@csiro.au 
What is the id for this layer?  Has it been released into production/synced to 
ala-macropus?
 May 2, 2012 Project Member #12 adam_col...@tpg.com.au 
Natasha the layer is cl1918.  layers-store sampling (for class names instead of 
numbers) and ala-maps data (for offset) updated today.  The class number 
instead of name was identified from the handful of biocache records with cl1918 
values with &fl, but that does not appear to be working now.
 May 2, 2012 Project Member #13 natasha....@csiro.au 
Adam,

It looks as if the layer is in biocache:

http://biocache.ala.org.au/ws/occurrences/search?q=cl1918:[*%20TO%20*]&facets=cl
1918&facets=data_resource_uid&pageSize=0

Although there are only 858745 records that have values. I am not sure if this 
is correct for the layer.

Is there any action that I need to take?

 May 2, 2012 Project Member #14 adam_col...@tpg.com.au 
Could the 858745 records be the new/updated records since the last full 
refresh?  

If ala-macropus data sync is still daily then from tomorrow a full resampling 
for this layer will fix it.

Is there a way to test that the sampling is working for biocache, given the 
problems so far on our end?

 May 2, 2012 Project Member #15 natasha....@csiro.au 
Not all the records have bee reprocessed since the last time I performed a 
complete resample/reprocess (on the 2012-04-14).

You can see this by inspecting the values for the last_processed_date:
http://biocache.ala.org.au/ws/occurrences/search?q=cl1918:[*%20TO%20*]%20&facets
=last_processed_date&facets=cl1918&pageSize=0

    "label": "2012-04-14T12:11:11Z",
    "count": 4853

The total number of records does seem a little low. But I am not sure about the 
coverage for the layer.

When the layer is available I will sample it and re run the processing on these 
858745 records. 

It will be applied to all records the next time we perform a complete 
reprocess.  

Is this OK?

 May 2, 2012 Project Member #16 adam_col...@tpg.com.au 
Perfect.  Can test it then.

Coverage is terrestrial Australia.  Similar but not the same number of records 
as the  others e.g. el874 26485514.  
 May 21, 2012 Project Member #17 leebel...@gmail.com 
Re-assign to Lee and prod Natasha to inform me post BioCache re-index, then 
I'll get data to Leo
 Owner: leebel...@gmail.com Cc: -chris.fl...@gmail.com -leebel...@gmail.com May 15, 2013 Project Member #18 moyesyside 
can this issue be closed ?
 May 15, 2013 Project Member #19 leebel...@gmail.com 
No, but I'll work on the lists now.
 May 15, 2013 Project Member #20 leebel...@gmail.com 
We have a problem. If I do the following:

1. Add to Map | Layers | DLC - it maps the layer

2. Add to Map | Area | Gazetteer polygon | primarily ........open alpine

The alpine areas do NOT match. A worry. (2) is MUCH larger than it should be 
(not restricted to the Alpine class).
 Cc: -natasha....@csiro.au May 15, 2013 Project Member #21 leebel...@gmail.com 
(No comment was entered for this change.)
 Cc: natasha....@csiro.au moyesyside May 16, 2013 Project Member #22 moyesyside 
Added Chris.

I've just  tested this i think and it seems ok to me. See screenshot attached.

Lee - can you attach a screenshot or add further detail ?
 Screen Shot 2013-05-16 at 4.59.07 PM.png 324 KB   View   Download Cc: -natasha....@csiro.au chris.fl...@gmail.com May 16, 2013 Project Member #23 leebel...@gmail.com 
I am 100% that the gazetteer class for alpine (and probably all others) is not 
correct. Images are of the alpine class for all Australia and a zoom in on Lake 
Eyre which is reported as part of alpine class in gazetteer but is BRINE class 
on DLC.
 DLC_Alpine_Class.gif 301 KB   View   Download DLC_Alpine_Class_Lake Eyre.gif 44.0 KB   View   Download DLC_Brine_Class_Lake Eyre.gif 236 KB   View   Download Jun 5, 2013 Project Member #24 chris.fl...@gmail.com 
Hi Lee,

This problem is caused due to a problem with the special ingestion process that 
Adam developed to generate a contextual layer from a large gridded dataset. The 
only two layers that have been ingested using this process are Dynamic Land 
Cover and the new Land Cover Type layer that I got from AusCover.

For layers loaded via this process, a lot of data is read straight off files on 
disk where for other contextual layers it would be read from the layers 
database.

Because of this disk vs database difference, it is not possible to render 
gazetter areas for these layers in the ordinary way. So what Adam has done is 
display the entire layer that the layer belongs to, and then render it with a 
custom style to (in theory) only highlight the relevant area from the gazetter 
search. 

A problem with the generation of the custom style means that an incorrect area 
is highlighted.

I can fix the problem with the custom style, however there are a few other 
problems here:

1. The legend for the corresponding layer is displayed. e.g. the DLC legend is 
displayed when a gazetter search is done for "open alpine". This does not 
happen when a gazetteer area from other layers is displayed.

2. The hover tool displays values for the corresponding layer, even in 
non-highlighted areas. This does not happen when a gazeteer area from other 
layers is displayed.

3. The gazetteer area cannot be exported, used in an add to map/species etc. 
E.g. display "open alpine" and then try add to map/species.

I can fix the aforementioned style problem to get areas to display correctly, 
but as you can see this is only part of the problem.

Thoughts Lee and Dave?

 Jun 5, 2013 Project Member #25 leebel...@gmail.com 
Thanks Chris. I remember Adam telling me about this. We do need a mechanism for 
generating polygons based on a class of what I called a 'non-unique (polygon) 
contextual layer' (e.g., States and Territories, DLC etc. 'Unique polygon 
contextual layers' are for example LGAs (I presume) or drainage catchments.

The gazetteer in the SP is one of the most powerful tools, mainly due to the 
integration of a wide range of location information, not just the traditional 
Australian point gazetteer. 

The power of 'Add to Map | Areas | Gazetteer polygon' is extreme. I suspect the 
biggest use will be for 'Tools | Area analysis' (and the Tools | Tabulation of 
course). This was the context of the request from Leo (and this will be a VERY 
common request!). Also, as you probably know, I'm EXTREMELY keen to beef up the 
Area Report. There are a number of easily added measures that would be great to 
see. This feature would be a game-changer.

We can certainly live with your 1 and 2 but the export bit of 3 is a pity (that 
we may have to live with). I've had about 10 requests for the ability to export 
the polygons from one class of a contextual map - mainly because this info is 
not available by other means.

 Jun 14, 2013 Project Member #26 chris.fl...@gmail.com 
I have fixed the display of the gazetteer areas for DLC and also the Auscover 
land type layer. Note that these areas still can't be used for export, add to 
map species etc.
 Jun 18, 2013 Project Member #27 leebel...@gmail.com 
NOPE: Problems remain.

Scenario-

1. Go to http://spatial.ala.org.au/
2. Add to Map | Area | Gazetteer polygon
3. Enter "open alpine" (for example) and select that class from the DLC layer
4. Tools | Area report - and use the default selected area  (Primarily 
Vegetated Natural & Semi-Natural Terrestrial Vegetation )
5. Download species list
6. Map all occurrences

Problems:

IThe area is mapped fine but the species list doesn't make sense with the Area 
report saying

1. Area is 0 sq km!
2. 3 species which map out at lat/long 0/0 with 4 occurrences...

 Jun 18, 2013 Project Member #28 chris.fl...@gmail.com 
I should have expanded on my comment "Note that these areas still can't be used 
for export, add to map species etc." - it is now possible to correctly display 
classes from the DLC and Auscover land cover type layers via a gazetter search. 
That was the bug fix that I made.

However the resulting "area" cannot be used in further operations. Export, add 
species to map, area report all will not work or will not work correctly. It 
appears that the support for contextual layers derived from large grids was not 
completed. This is not a bug but an incomplete feature.
 Jul 9, 2013 Project Member #29 chris.fl...@gmail.com 
(No comment was entered for this change.)
 Labels: -Priority-Critical Priority-Medium
Original issue reported on code.google.com by moyesyside on 8 Aug 2013 at 12:40
bsed / ala

Species checklist for Leo for alpine areas #76
