search

3liz / lizmap-web-client

Transfer a QGIS project on a server, Lizmap is providing the web interface to browse it
https://www.lizmap.com
Mozilla Public License 2.0
254 stars 143 forks source link

[Bug/Enhancement]: Popup not working on external WMS #3497

Closed r9zzai closed 1 year ago

r9zzai commented 1 year ago

What is the Bug/Enhancement?

When having external WMS sources in the QGIS project. When clicking on a feature of the external WMS source then Lizmap Popup is says that there is no object on that place and there is a 400 Error in the network log.

Proposed solution: replace function gfiGmlToHtml in lizmap/modules/lizmap/lib/Request/WMSRequest.php

    protected function gfiGmlToHtml($gmldata, $configLayer)
    {
        $xml = $this->loadXmlString($gmldata, 'GetFeatureInfoHtml');

        if (!$xml || count($xml->children()) == 0) {
            return '';
        }

        // Create HTML response
        $layerTitle = $configLayer->title;

        $HTMLResponse = "";
        foreach( $xml->children() as $xmlfeature ) {
            $HTMLResponse .= "<h4>${layerTitle}</h4><div class='lizmapPopupDiv'><table class='lizmapPopupTable'>";
            foreach ($xmlfeature->attributes() as $key => $value) {
                $HTMLResponse .= "<tr><td>${key}&nbsp;:&nbsp;</td><td>${value}</td></tr>";
            }
            $HTMLResponse .= '</table></div>';
        } 
        if ($HTMLResponse == "") {
            return '';
        }

        return $HTMLResponse;
    }

Further Questions: in the original mentioned function the name of the layer plus "_layer" is checked in the xml response:

        $layerstring = $configLayer->name.'_layer';
        if (!property_exists($xml, $layerstring)) {
            return '';
        }

how can that be achieved that the external WMS response gets modified that the response contains name of the layer plus "_layer" so that the popup is working for external WMS layers? Do I something wrong here acivating Popup for external WMS Layers?

Demo Project: http://52.59.84.1/lizmap-web-client-3.6.0/lizmap/www/index.php/view/map?repository=4&project=test

Steps to reproduce the issue

  1. Add external WMS layer to the QGIS project (e.g. https://www.lfu.bayern.de/gdi/wms/energieatlas/photovoltaikanlagen)
  2. Check Popup and Check Receive Images directly from that WMS-Serer for that layer in the QGIS Lizmap Plugin
  3. Publish the project, activate the layer in the layer tree and click on a feature

Versions

Versions :

Check Lizmap plugin

QGIS server version, only if the section above doesn't mention the QGIS Server version

No response

Operating system

Debian GNU/Linux 11 (bullseye)

Browsers

Firefox

Browsers version

110.0 (64-Bit)

Relevant log output

no log
nworr commented 1 year ago

The request has an error 400 code because the info_format provided by lizmap when doing GetFeatureInfo request is not available for the server . But server seems to response with a default format , and the response is handled by lizmap

For exemple request for https://www.lfu.bayern.de/gdi/wms/energieatlas/photovoltaikanlagen return an code 400 but the body contains data.

For https://geoservices.bayern.de/wms/ , it's the same , but the response is empty (even in QGIS desktop) like raw requset in broswer

r9zzai commented 1 year ago

Thank you for the info. For example https://geoportal.freiburg.de/wms/gdm_address/gdm_address is working (GetMap and GetFeatureInfo) even without Check Receive Images directly from that WMS-Serer in the lizmap QGIS desktop Plugin.

Can my solution be merged or is there anything against it? I mean its nearly the same if its not a gml File then return '' and if there are no children in the xml File then return '' because in both cases we try to parse the children in the xml file.

nworr commented 1 year ago

I think i got it , in your first comment the problem with the lfu.bayern WMS is that the XML response returned has features info in the attributes of the FIELDS tag (see raw request )

XML response from external WMS can come in very various way, so lizmap just return the body

But IMHO, the gfiGmlToHtml function aims to format GML response, and not XML, so your solution is too specific.

but a workaround may can be made using as js script to format the response in the popup in the browser side

    lizMap.events.on({  
    'lizmappopupdisplayed': function(e) {   
        //get popupcontent
        let popupContent = $('#popupcontent .lizmapPopupContent').html()
                // format content
               }
   });

see lizmap doc

r9zzai commented 1 year ago

Okay, thank you for your opinion and tip. Js Script was my first approach. But here i was stuck on not recognizing that gml info_format. So my code tried to recreate every popup and i cant recreate the buttons for example to edit a feature. Another not so nice result of the js code is that the created element in the popup table gets stuck in css format so it cant be deleted. So if i click a few times on many activated layers there are hundreds of popups stuck in the css format. But that is not a performance issue at all. It just would be cleaner to do that in PHP.

Here ist my Js script solution for external WMS layers not having GML in info_format, maybe it does help others:

map_crs = "EPSG:3857"
wms_crs = "EPSG:31468"//transform to needed crs in wms query, most layer will work with EPSG 3857
// wms_crs = "EPSG:3857"
noresult="Kein Treffer"//condition for popups not to show if in html of getfeatureinfo
div_div1cls_arr=[]//popup features

lizMap.events.on({
    uicreated: function(e) {
    lizMap.map.events.register("click", lizMap.map , querywmsgetfeatureinfofun);
    html=`
    <script type="text/javascript" src="https://cdn.jsdelivr.net/npm/table-to-json@1.0.0/lib/jquery.tabletojson.min.js"></script>
    <div id="externalwmspopupcontent"></div>
    `
    $("#content").append(html)

    // to_observe = document.getElementById("dock")
    to_observe = document.getElementById("nav-tab-switcher")
    observer2.observe(to_observe, {attributes : true});

    }
});

lizMap.events.on({
    lizmappopupdisplayed: function(e) {
        $(".lizmapPopupContent").find('h4').remove()//remove nothing is found header  
    }
});

querywmsgetfeatureinfofun=async function querywmsgetfeatureinfo(e){
    to_observe = document.getElementById("popupcontent")
    observer.observe(to_observe, {attributes : true});
    //maybe observe dock and check if popupcontent and switcher is active
    //then click dock-close after that click button-switcher

    point=e.xy //x,y
    size=lizMap.map.getSize() //w,h
    extent=lizMap.map.getExtent() //left bottom right top
    lyrs=lizMap.map.layers
    randomstrs=[] //clear list of valid layer randomstr
    for (var i=0;i<lyrs.length;i++){
        randomstr=makeid(5)//randomstr for each request to hide stale css elements
        randomstrs.push(randomstr)
        data= await querywmslayersgetfeatureinfofun(point,size,extent,lyrs,i,randomstr)
    }
    $("#externalwmspopupcontent")[0].randomstrs=randomstrs
    checkpopups(randomstrs) //hide stale popup features
}

//make random string
function makeid(length) {
    var result           = '';
    var characters       = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
    var charactersLength = characters.length;
    for ( var i = 0; i < length; i++ ) {
        result += characters.charAt(Math.floor(Math.random() * charactersLength));
    }
    return result;
}

//build wms query
querywmslayersgetfeatureinfofun=async function querywmslayersgetfeatureinfo(point,size,extent,lyrs,i,randomstr) { 
  return new Promise(function(resolve, reject) {  
    bsrlizmap="index.php"
    bsr=window.location.href.split("?")[1] //possibility to include another lizmap project as "external" wms
      dsply=lyrs[i].div.attributes.style.value
      try {
      if (lyrs[i].isVisible == true || dsply.includes("display: block")){ 
      if (lyrs[i].url.includes(bsr) == true) {
          urli=window.location.href.replace("view/map","lizmap/service")
          urli+="&SERVICE=WMS&VERSION="
          use_wms_crs=map_crs
      }else if (lyrs[i].url.includes(bsrlizmap) == true) {
          urli=lyrs[i].url
          urli+="&SERVICE=WMS&VERSION="
          use_wms_crs=map_crs
      }else{
          urli=lyrs[i].url
          urli+="?SERVICE=WMS&VERSION="  
            use_wms_crs=wms_crs    
      }
          urli+=lyrs[i].params["VERSION"]
          urli+="&REQUEST=GetFeatureInfo&BBOX="

        cp1 = new OpenLayers.Geometry.Point(extent["left"], extent["bottom"]);
        cp2 = new OpenLayers.Geometry.Point(extent["right"], extent["top"]);
        cp1.transform(map_crs, use_wms_crs);
        cp2.transform(map_crs, use_wms_crs);
        urli+=String(cp1.x)+","+String(cp1.y)+","+String(cp2.x)+","+String(cp2.y)
        urli+="&CRS="+use_wms_crs+"&SRS="+use_wms_crs+"&LAYERS="
        urli+=lyrs[i].params["LAYERS"]
        urli+="&STYLES=&FORMAT=image/png&QUERY_LAYERS="
        urli+=lyrs[i].params["LAYERS"]
        urli+="&INFO_FORMAT=text/html&FEATURE_COUNT=10&"
        urli+="I="+point["x"]+"&J="+point["y"]+"&X="+point["x"]+"&Y="+point["y"]+"&WIDTH="+size["w"]+"&HEIGHT="+size["h"]
        urli=urli.replace("??","?")
        // console.log("getfeatureinfourl ", urli)
        $.ajax({
          type: 'get',
          url:urli,//+"&callback=?", 
          success:function(data){
              // console.log("urli is",urli)
              if (data.includes("lizmap")==false) {
              if (data.indexOf("<iframe")>-1) {
                  data=new DOMParser().parseFromString(data, "text/html");
                    iframe= $(data).find("iframe")
                    urli=iframe[0].src
                    $.ajax({
                      type: 'post',
                      url:"../../../gis/geturlcontent.php", 
                      data:{urli:urli},
                      success:function(data){
                          arr=parseHTMLTables(data,1)
                        // push arr into table
                        if (arr.length>0){
                            buildtableonpopup(lyrs[i].params["LAYERS"],arr,randomstr)
                          // buildtableonpopupfun(data,lyrs,i,randomstr)
                        }
                          resolve("")
                      }
                    })

              }else{
                  arr=parseHTMLTables(data,0)
                // push arr into table
                if (arr.length>0){ 
                buildtableonpopup(lyrs[i].params["LAYERS"],arr,randomstr)
                // buildtableonpopupfun(data,lyrs,i,randomstr) 
                        }
                resolve("")
              }
              }

          }
        })
        resolve("") //resolve nothing if there is no ajax
      }
      resolve("")
      }catch(e){console.log("err " ,e )}
   }) 
}

div_div1cls_arr=[]
//hide stale popup features
function checkpopups(randomstrs){ 
    for (g in div_div1cls_arr) {
        div_div1cls_arr[g].hidden=true
        for ( ff in randomstrs) {
            if (div_div1cls_arr[g].id.indexOf(randomstrs[ff])>-1) {
                div_div1cls_arr[g].hidden=false
                break
            }
        }
    }
    for (g in div_div1cls_arr) {
        try{ //popup active
            $(".lizmapPopupContent")[0].append(div_div1cls_arr[g])
            $("#popupcontent")[0].append(div_div1cls_arr[g])
        }catch(e){ }
    }
}

//add active to classliste of popupcontent if it has gone and readd popup features
var observer = new MutationObserver(function(mutations) {
  mutations.forEach(function(mutation) {
    try { 
        let cl=mutation.target.classList
        if (!(cl.contains("active"))) {
            $("#nav-tab-popupcontent")[0].classList.add("active")
            $(".lizmapPopupContent").find('h4').remove()
            cl.add("active")
            adock=$("#dock")
            adock.css( "display", "block")
            adock.css( "width", "auto%")
            randomstrs=$("#externalwmspopupcontent")[0].randomstrs //get current identifiers of popup features
            checkpopups(randomstrs) //readd popup features
            observer.disconnect()
        }
    } catch(e) {}
  });
});

//maybe observe dock and check if popupcontent and switcher is active
//then click dock-close after that click button-switcher
//happens if layer has no popup but is editable
var observer2 = new MutationObserver(function(mutations) {
  mutations.forEach(function(mutation) {
    try { 
        cl1=$("#nav-tab-switcher").hasClass("active")
        cl2=$("#nav-tab-popupcontent").hasClass("active")
        if (cl1 && cl2) {
            $("nav-tab-popupcontent").removeClass("active")
            cl2=$("#nav-tab-popupcontent").hasClass("active")
            $("#dock-close").click()
            $("#button-switcher").click()   
        }
    } catch(e) {console.log("err ",e)}
  });
});

arr=[]
//build table
function buildtableonpopup(title,arr,randomstr) {
    feats=[]
    //console.log("title is ", title , " arr is ", arr)
    if (arr[0][0]=="0"){
        narrr=[]
        for (var f=1;f<arr.length;f++) {
            narrr.push({'0':arr[f][0],'1':arr[f][1]})
        }
        arr=narrr
    }

    ffeat=arr[0][0]
    for (f in arr) {
        if (arr[f][0]==ffeat) {
            feats.push(Number(f))
        }

    }
    if (feats.length>1) {
        leof=feats[1]
    }else{
        leof=arr.length
    }

    for (f in feats) {
      narr=[]
        for (var g=0;g<leof;g++) {
            try {
                if([arr[feats[f]+g][0],arr[feats[f]+g][1]].toString()!=[ "Feld", "Wert" ].toString()) {
                    //console.log([arr[feats[f]+g][0],arr[feats[f]+g][1]].toString!=[ "Feld", "Wert" ].toString()
        narr.push([arr[feats[f]+g][0],arr[feats[f]+g][1]])
        }
            }catch(e) {}
      }
      popupbuilder(title,narr,randomstr)
    }    
}

function popupbuilder(title,arr,randomstr) {
    div1h41cls="lizmapPopupTitle"
    div_div1h41cls = document.createElement('h4');
    div_div1h41cls.classList.add(div1h41cls);
    div_div1h41cls.textContent=title

    div1div1cls="lizmapPopupDiv"
    div_div1div1cls = document.createElement('div');
    div_div1div1cls.classList.add(div1div1cls);

    div1divtblcls="table"// table-condensed table-striped table-bordered lizmapPopupTable"
    table = document.createElement("table")
    table.classList.add(div1divtblcls);
    //table table-condensed table-striped table-bordered lizmapPopupTable
    //table.classList.add("table-condense");
    table.classList.add("table-condensed");
    table.classList.add("table-striped");
    table.classList.add("table-bordered");
    table.classList.add("lizmapPopupTable");

      //header row
      header = table.createTHead()
      row = header.insertRow();
      cellA = row.insertCell();
      cellB = row.insertCell();
      cellA.outerHTML = "<th>Feld</th>";
      cellB.outerHTML = "<th>Wert</th>";
      tbody = table.createTBody()

    for (var i = 0 ; i<arr.length;i++) { 
          // normal rows
          row = tbody.insertRow();
          cellA = row.insertCell();
          cellB = row.insertCell();
          cellA.outerHTML = "<th>" + arr[i][0] + "</th>";
          cellB.innerHTML = arr[i][1];
    }
    div_div1div1cls.appendChild(table);

    div1cls="lizmapPopupSingleFeature"
    div_div1cls = document.createElement('div');
    div_div1cls.classList.add(div1cls);
    div_div1cls.id="wmsexternal"+randomstr
    div_div1cls.appendChild(div_div1h41cls);
    div_div1cls.appendChild(div_div1div1cls);

    $(".lizmapPopupContent")[0].appendChild(div_div1cls)
    $("#popupcontent")[0].appendChild(div_div1cls)
    div_div1cls_arr.push(div_div1cls)
}

htmltablewtf=""
//parse tables to html tables
function parseHTMLTables(data,special) {
  jobjs=[]
    data=new DOMParser().parseFromString(data, "text/html");
    //tbls= $("#table").tableToJSON()
    tbls= $(data).find("table")
    arr=[]
    htmltablewtf=data
    if (tbls.length>0) {
      for (var i=0;i<tbls.length;i++) {
        tbl=tbls[i]
        tbl.id="wmstbl"+i
        if ($("#" + "wmstbl"+i ).length == 1) {
          // Replace
          $("#" + "wmstbl"+i ).replaceWith(tbl)
        }else {
          // Append
          $("#map-content").append(tbl)
        }
        if (special) {
            myData = tbl.rows
            my_liste = []
            for (var i = 0; i < myData.length; i++) {
                    el = myData[i].children
                    my_el = []
                    for (var j = 0; j < el.length; j++) {
                            my_el.push(el[j].innerText);
                    }
                    my_liste.push(my_el)

            }
            jobj=[]
            for (var f=1;f<my_liste.length;f++) {
              jobj.push({'0':my_liste[f][0],'1':my_liste[f][1]}) 
            }

        }else{
            jobj=$("#" + "wmstbl"+i).tableToJSON()
        }
        // console.log("obj is " , jobj)
        tbl.style=[]
        tbl.style["display"]="none" //needs to be displayed for tableToJSON function
        jobjs.push(jobj)
        arr=appendJsonTableToPopup(jobj)
      }
    }

    return arr
}

//conditions to choose custom functions to parse different html tables in 2d array
function appendJsonTableToPopup(jobj) {
  arr=[]
  try {if (Object.keys(jobj[0]).length==1) { //n2 - table
    arr=appendJsonTableToPopup_type_2n_1(jobj)
  }}catch(e){}
  try {if (Object.keys(jobj[0]).length==2) { //2n(2) - table
    arr=appendJsonTableToPopup_type_2n_2(jobj)
  }}catch(e){}
  try {if (Object.keys(jobj[0]).length>2) { //2n - table
    arr=appendJsonTableToPopup_type_2n_3(jobj)
  }}catch(e){}
  return arr

}

//custom functions to parse different html tables in 2d array
function appendJsonTableToPopup_type_2n_1(jobj) {
    try { 
        arr=[]
        kys=Object.keys(jobj)
        for (i=0;i<kys.length;i++) {
            ky=Object.keys(jobj[kys[i]])[0]
            arr.push([ky, jobj[kys[i]][ky]])
        }
    }catch(e){
        arr=[]
        kys=Object.keys(jobj)
        for (i=0;i<kys.length;i++) {
          arr.push([kys[i], jobj[kys[i]]])
        }
    }
    return arr  
}
function appendJsonTableToPopup_type_2n_2(jobj) {
    arr=[]
    kys=Object.keys(jobj[0])
    arr.push([kys[0],kys[1]])
    for (i=0;i<jobj.length;i++) {
      arr.push([jobj[i][kys[0]], jobj[i][kys[1]]])
    }
    return arr  
}
function appendJsonTableToPopup_type_2n_3(jobj) {
    arr=[]
    kys=Object.keys(jobj[0])
    for (i=0;i<kys.length;i++) {
      arr.push([kys[i], jobj[0][kys[i]]])
    }
    return arr  
}

Demo: http://52.59.84.1/lizmap-web-client-3.6.0/lizmap/www/index.php/view/map?repository=4&project=test2

additional tiny PHP script (requested in above Js script) for re request of WMS GetFeatureInfo having <iframe> tag in response (e.g. https://geoservices.bayern.de/wms/v1/ogc_bauleitplan.cgi?):

<?php
$urli=$_POST["urli"];
$urli=urldecode($urli);
$urli=str_replace("xyz","?",$urli);

$ch = curl_init($urli);
curl_setopt($ch, CURLOPT_HEADER, false);    // we don't want headers
curl_setopt($ch, CURLOPT_NOBODY, false);    // we need body
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_TIMEOUT,10);
$response = curl_exec($ch);
$httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
echo $response;
?>