tomcat 8.5 causes encoding issues

[nathandunn]

In tomcat 8.5 / production a target like this:

/apollo/annotator/loadLink?loc=scf7180000302113:425429..426136&usda=[gov/apollo/Drosophila%20biarmipes/jbrowse/?loc:scf7180000302113:425429..426136]&addStores=%7B"url"%3A%7B"type"%3A"JBrowse%2FStore%2FSeqFeature%2FGFF3"%2C"urlTemplate"%3A"https%3A%2F%2Fabc123.com%2Ftraining%2Fwebapp%2Fmedia%2Fblast%2Ftask%2F3733ff81a0ae4ad4a4d795d06110f9d0%2FDbia02082011-genome.fa.gff"%7D%7D&addTracks=[{"label":"BLAST+%20Results","category":"0.%20Reference%20Assembly","type":"WebApollo/View/Track/DraggableBLASTFeatures","store":"url","style":{"renderClassName":"gray-center-30pct","subfeatureClasses":{"match_part":"blast-match_part"}}}]&tracks=DNA,Annotations,BLAST+%20Results&highlight=&organism=Drosophila%20biarmipes&clientToken=119857622881828907356&format=null

returns a 400 error because on the backend.

    13-Nov-2017 16:06:22.881 INFO [http-nio-8080-exec-3] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
 Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
 java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986
    at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:472)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:683)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

[nathandunn]

Parts are:

• [x] http://localhost:8080/apollo/annotator/loadLink?

• [x] loc=scf7180000302113:425429..426136

• [x] usda=[gov/apollo/Drosophila%20biarmipes/jbrowse/?loc:scf7180000302113:425429..426136]

• FAILS - addStores=%7B%22url%22%3A%7B%22type%22%3A%22JBrowse%2FStore%2FSeqFeature%2FGFF3%22%2C%22urlTemplate%22%3A%22https%3A%2F%2Fi5k.nal.usda.gov%2Ftraining%2Fwebapp%2Fmedia%2Fblast%2Ftask%2F3733ff81a0ae4ad4a4d795d06110f9d0%2FDbia02082011-genome.fa.gff%22%7D%7D

• FaILS addTracks=[{%22label%22:%22BLAST+%20Results%22,%22category%22:%220.%20Reference%20Assembly%22,%22type%22:%22WebApollo/View/Track/DraggableBLASTFeatures%22,%22store%22:%22url%22,%22style%22:{%22renderClassName%22:%22gray-center-30pct%22,%22subfeatureClasses%22:{%22match_part%22:%22blast-match_part%22}}}]

• [x] tracks=DNA,Annotations,BLAST+%20Results

• [x] highlight=

• [x] organism=Drosophila%20biarmipes

• [x] clientToken=119857622881828907356&format=null

[nathandunn]

• [ ] Dev-mode
  • [x] build ':tomcat:7.0.55.2'
  • [x] build ':tomcat:8.0.33'
  • [x] build ':tomcat:9.0.0.M4.1'
• [ ] war-mode
  • [ ] build ':tomcat:7.0.55.2'
  • [x] build ':tomcat:8.0.33'
  • [x] build ':tomcat:9.0.0.M4.1'
• [ ] deployed mode
  • [x] ':tomcat:7.0.55.2'
  • [x] ':tomcat:8.0,x'
  • ~~ ':tomcat:8.5,x' ~~ FAIL
  • [ ] ':tomcat:9.0.0.M4.1'

[nathandunn]

FYI @childers / @mpoelchau

So good news is that I have VERY specific code to handle bugs in addStores and addTracks.

I am hoping I can detect these versions directly and fix it that way.

[childers]

Great, thanks for tackling this so quickly!

[nathandunn]

The best “solution” I found was to add this line in conf/catalina.properties

tomcat.util.http.parser.HttpParser.requestTargetAllow=|{}

From reading this: https://stackoverflow.com/questions/41053653/tomcat-8-is-not-able-to-handle-get-request-with-in-query-parameters

The workaround is to have these be posts or hash the URL params when sending via some common hash.

[nathandunn]

Even when encoded, the decoded version still seems to come back out.