SOCAT: QC and WOCE comments character problems

[karlmsmith]

Reported by @karlmsmith on 3 Mar 2016 00:05 UTC Some characters in QC comments (and presumably WOCE comments as well) cause problems, despite being encoded, when submitted. (Presumably because the comment is part of the URL and the characters are interpreted as part of the URL structure instead of part of a comment parameter.) The current hack is to translate problem-causing characters into words: ' and " are removed

becomes ' greater than ' < becomes ' less than ' % becomes ' percent ' & becomes ' and ' which is an ugly hack.

Kevin Sullivan (AOML) just reported using +/- (the three character string) in a comment submits okay but the + is lost. So '+/-0.03' turns into '/-0.03' in the comment seen in LAS. (I suspect the + is being interpreted as part of the URL and getting lost in the process.)

Seems like the proper solution is to make the comment a POST parameter instead of a GET parameter, especially since the QC comment could potentially be rather long. But not sure how to do this in the LASRequest framework. Currently the code doing the submit looks like:

    var Req = new LASRequest('$las_request.toJavaScriptSafeString()');
    Req.setOperation("SOCAT_QC_submit");
    Req.setProperty("qc","cruise_ID","$las_request.getProperty('qc', 'cruise_ID')");
    Req.setProperty("qc","region_IDs",regionIDs);
    Req.setProperty("qc","flag",flag);
    Req.setProperty("qc","comment",comment);
    Req.setProperty("qc","reviewer",reviewer);

    var reqXmlText = Req.getXMLText();
    reqXmlText = escape(reqXmlText);
    document.location="ProductServer.do?xml=" + reqXmlText;

Migrated-From: http://dunkel.pmel.noaa.gov/trac/las/ticket/1727

[karlmsmith]

Modified by @karlmsmith on 3 Mar 2016 00:12 UTC

[karlmsmith]

Comment by @noaaroland on 10 Mar 2016 22:30 UTC The problem was that the value of the xml parameter was being decoded twice. The first time the plus sign gets decoded from %2B to a "+" character. The second time it becomes a blank.

Apparently, struts2 decodes parameters automatically. Who knew? Generally not a problem for us to also decode it except with plus signs.

I did two things. I change the JS function from encode (which is depreciated) to encodeURIComponent which is more modern.

I now do all decoding on the server in a static function that attempts to determine if the decoding was actually necessary and if not it restores the original string.

[karlmsmith]

Comment by @karlmsmith on 15 Mar 2016 00:17 UTC Remove the hacks that converts symbols < > ' " & % to words. This change causes problems when these symbols (or some of them, or maybe some others) appear in the string. Realized the easy solution is to pass the comment string encoded as if it was arbitrary binary data. So implemented this by encoding as a UTF-16 hex string (so lots of 00 prefix to normal characters, but easier to encode and decode). Might be able to switch to Base64 (typically what is used for binary data and is more compact) if the length becomes an issue. Has no problem with extended characters, although the database character set is latin-1 so limitations are now from the database table.