String conversion issue

[jeromelesaux]

Hi I try know to use search criteria with my upnp server. with curl I've got the expected results : curl -v -H 'SOAPACTION: "urn:schemas-upnp-org:service:ContentDirectory:1#Search"' -H 'content-type: text/xml ;charset="utf-8"' "http://10.188.2.125:8200/ctl/ContentDir" -d "@search.xml"

with search file content : `<?xml version="1.0" encoding="utf-8"?> <s:Envelope xmlns:ns0="urn:schemas-upnp-org:service:ContentDirectory:1" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">

* * (dc:title contains "star wars") and (upnp:class derivedfrom "object.item.videoItem") 0 0 ` When I use the goupnp api like this : ` result, returnedNumber, totalMatches, _, err := client.Search("*", "(dc:title contains \"star wars\") and (upnp:class derivedfrom \"object.item.videoItem\")", "*", 0, 0, "")` the quotes are interpreted as html in my upnp server, here the log of the query : `[2017/11/03 12:27:43] sql.c:88: error: prepare failed: near "&": syntax error SELECT (select count(distinct DETAIL_ID) from OBJECTS o left join DETAILS d on (o.DETAIL_ID = d.ID) where (OBJECT_ID glob '*$*') and ((d.TITLE like "star wars") and (o.CLASS like "object.item.videoItem") )) + (select count(*) from OBJECTS o left join DETAILS d on (o.DETAIL_ID = d.ID) where (OBJECT_ID = '*') and ((d.TITLE like "star wars") and (o.CLASS like "object.item.videoItem") )) [2017/11/03 12:27:43] upnpsoap.c:123: warn: Returning UPnPError 708: Unsupported or invalid search criteria` the quotes are replaced by the html code. I tried to find a workaround with runes, but I am still the same issue. I have you got any advises ? Best regards Jerome

[huin]

Hm, this is beyond my limited knowledge of UPnP. At a guess I'd say that it looks like the server isn't decoding the XML properly. Could you check what the payload looks like that is sent to the server?

To check this, at this point in your local copy of this library: https://github.com/huin/goupnp/blob/master/soap/soap.go#L40

Put this code:

fmt.Println(string(requestBytes))

I'm expecting that you will see a part of it looking like this:

...<SearchCriteria>(dc:title contains &#34;star wars&#34;) ...

This is valid, and a correct XML encoding of the query. But if the server doesn't decode it properly then the " are being read literally. If that's the case, then things might be tricky, as I'm not sure how to get the Go XML library to not escape the double quotes. You could try using single quotes instead, but I rather suspect that they would be similarly escaped as well.

[jeromelesaux]

Hi Huin you're right the soap envelop convert the quote to " html code. But when I see the specification, searchCriteria wait for quote or double quote and not html code. http://upnp.org/specs/av/UPnP-av-ContentDirectory-v1-Service.pdf Simple quote or double are converted. How can I bypass the conversion ?

Best regards

Jerome

[huin]

As goupnp stands right now, you won't be able to bypass it, the library will have to be changed to make this work. My interpretation of the spec would be that it should accept XML entities (otherwise how would the relOp part of the grammar allow for literal < without breaking the XML it's embedded within.

Looking at https://golang.org/pkg/encoding/xml/#Marshal I'm thinking that it might be possible to tag the outgoing data elements encoded as text in XML as innerxml, and arrange for the value passed to be only escaping <, >, and &, which are the absolute minimum to be escaped and not do dangerous things in XML text elements (but crucially not " or ', which should only need to be escaped within an XML tag). This should still be acceptable and safe XML and meet your immediate needs.

This would affect all the generated code, such as the code here:

https://github.com/huin/goupnp/blob/master/dcps/av1/av1.go#L2423

A quick hack to see if this approach would work for you would be to edit (but not commit) the file above to change the (current) line 2428 from

SearchCriteria string

to

SearchCriteria string `xml:",innerxml"`

(Note that those are backticks, not single quotes, and the comma is not a typo)

See if that solves this specific case you have and let me know. Note that this is not a complete solution, as:

a) We need to change the code generator to do this in future. b) This will break the XML as soon as a <, >, or & appears in the search criteria (although I wonder how your media server will cope with that case without this change -- I'd be interested in knowing).

[huin]

If possible, could you test branch server-xml-workaround? I'm hoping that this is a functional workaround for the problem.

[jeromelesaux]

Hi Huin, well, the issue comes from the line 129 of the file soap. The EncodeElement function of the xml encoder escape string. I see no solution for the moment. Thanks a lot for the branch, I tested with it, but same result.

Jerome

[jeromelesaux]

Hi Huin, I find a workaround, I create my own soap client like this : `var URN_ContentDirectory_1 = "urn:schemas-upnp-org:service:ContentDirectory:1"

type SoapEnvelope struct { XMLName xml.Name xml:"http://schemas.xmlsoap.org/soap/envelope/ Envelope" Body *SoapBody }

type SoapFault struct { Faultstring string Detail string }

type SoapBody struct { XMLName xml.Name xml:"http://schemas.xmlsoap.org/soap/envelope/ Body" Fault SoapFault Search UpnpContentDirectorySearchRequest xml:"urn:schemas-upnp-org:service:ContentDirectory:1 Search" SearchResponse *UpnpContentDirectorySearchResponse xml:"urn:schemas-upnp-org:service:ContentDirectory:1 SearchResponse" }

type UpnpContentDirectoryClient struct { Url *url.URL xml:"-" }

func NewUpnpContentDirectoryClient(url url.URL) (UpnpContentDirectoryClient) { return &UpnpContentDirectoryClient{Url: url} }

func (c *UpnpContentDirectoryClient) Search(ContainerID string, SearchCriteria string, Filter string, StartingIndex string, RequestedCount string, SortCriteria string) (Result string, NumberReturned uint32, TotalMatches uint32, UpdateID uint32, err error) { search := &UpnpContentDirectorySearchRequest{ ContainerID: ContainerID, SearchCriteria: ""+SearchCriteria+"", Filter: Filter, StartingIndex: StartingIndex, RequestedCount: RequestedCount, SortCriteria: SortCriteria, } env := &SoapEnvelope{Body:&SoapBody{Search:search,Fault:nil}} w := &bytes.Buffer{} err = xml.NewEncoder(w).Encode(env) if err != nil { fmt.Println(err) return "",0,0,0,err } fmt.Printf("Envelope SOAP:%s",string(w.String())) httpClient := &http.Client{} httpRequest,err := http.NewRequest("POST",c.Url.String(),bytes.NewBuffer(w.Bytes())) if err != nil { fmt.Printf("%v",err) return "",0,0,0,err } httpRequest.Header.Set("SOAPACTION"," + URN_ContentDirectory_1 + #Search") httpRequest.Header.Set("CONTENT-TYPE","text/xml; charset=\"utf-8\"") httpResponse,err := httpClient.Do(httpRequest) if err != nil { fmt.Printf("%v",err) return "",0,0,0,err } if httpResponse.StatusCode != 200 { fmt.Printf("%v",httpResponse) return "",0,0,0,errors.New("http code "+ httpResponse.Status) } defer httpResponse.Body.Close()

response := &SoapEnvelope{}
err = xml.NewDecoder(httpResponse.Body).Decode(response)
if err != nil {
    fmt.Println(err)
    return "",0,0,0,err
}
sr := response.Body.SearchResponse
return sr.Result,sr.NumberReturned,sr.TotalMatches,sr.UpdateID,nil

}

type UpnpContentDirectorySearchRequest struct { ContainerID string xml:"ContainerID" SearchCriteria string xml:",innerxml" Filter string xml:"Filter" StartingIndex string xml:"StartingIndex" RequestedCount string xml:"RequestedCount" SortCriteria string xml:"SortCriteria" }

type UpnpContentDirectorySearchResponse struct { Result string NumberReturned uint32 TotalMatches uint32 UpdateID uint32 }

type DIDLLite struct { XMLName xml.Name DC string xml:"xmlns:dc,attr" UPNP string xml:"xmlns:upnp,attr" XSI string xml:"xmlns:xsi,attr" XLOC string xml:"xsi:schemaLocation,attr" Objects []Object xml:"item" }

type Object struct { ID string xml:"id,attr" Parent string xml:"parentID,attr" Restricted string xml:"restricted,attr" Title string xml:"title" Creator string xml:"creator" Class string xml:"class" Date string xml:"date" Results []Res xml:"res" }

type Res struct { Resolution string xml:"resolution,attr" Size uint64 xml:"size,attr" ProtocolInfo string xml:"protocolInfo,attr" Duration string xml:"duration,attr" Bitrate string xml:"bitrate,attr" SampleFrequency uint64 xml:"sampleFrequency" NrAudioChannels uint64 xml:"nrAudioChannels" Value string xml:",chardata" }`

and I can get the results from my device media server. Thanks for your time and your advises.

Jerome

[huin]

Ah yes, I'd completely forgotten that I'd written that bit of code. I guess that actually makes things easier, but I'll have to try again later when I get a moment.

[jeromelesaux]

No problem Huin, I'll stay tuned ;).

You can close this issue. Jerome

[huin]

I've had another crack at this in branch server-xml-workaround-2. I feel a bit more confident about this as it actually had an existing test.

[jeromelesaux]

Hi Huin, I've tested and it works as expected. You fixed my issue. Thanks for your time.

Jerome

[huin]

Excellent, thanks for the report and confirmation. I'll get this merged into master.

[huin]

Fixed in 991e174e2e678f42c106d9f3587cebff65be9a70.