search

slackapi / java-slack-sdk

Slack Developer Kit (including Bolt for Java) for any JVM language
https://slack.dev/java-slack-sdk/
MIT License
570 stars 209 forks source link

HTTP4K Duplicate Header exception #1264

Closed slushpupie closed 7 months ago

slushpupie commented 7 months ago

When using Bolt with http4k in an environment with multiple proxy passes, we countered an error where Bolt cannot handle multiple HTTP headers with the same name.

The app specifically in question is the Open Source Emoji Manager

Running a local http request:

POST http://localhost:3000//slack_events/v1/emojimanager-v1_events
Accept: application/x-www-form-urlencoded
X-Forwarded-Host: proxy1.example.com
X-Forwarded-Host: proxy2.example.com

(body does not matter, it fails before it can be parsed)

This throws an exception:

java.lang.IllegalStateException: Duplicate key X-forwarded-host (attempted merging values [proxy1.example.com] and [proxy2.example.com])
    at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135)
    at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:182)
    at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
    at com.slack.api.bolt.http4k.Http4kSlackApp.toSlackRequest(Http4kSlackApp.java:116)
    at com.slack.api.bolt.http4k.Http4kSlackApp.handleEventRequest(Http4kSlackApp.java:76)
    at com.slack.api.bolt.http4k.Http4kSlackApp.invoke(Http4kSlackApp.java:52)
    at com.target.slack.Http4KEmojiManagerApp.access$invoke$s2082440944(Http4KEmojiManagerApp.kt:15)
    at com.target.slack.Http4KEmojiManagerApp$routes$5.invoke(Http4KEmojiManagerApp.kt:24)
    at com.target.slack.Http4KEmojiManagerApp$routes$5.invoke(Http4KEmojiManagerApp.kt:24)
    at org.http4k.routing.TemplateRouter$match$1.invoke(Router.kt:188)
    at org.http4k.routing.TemplateRouter$match$1.invoke(Router.kt:188)
    at org.http4k.routing.RouterMatch$MatchingHandler.invoke(Router.kt:65)
    at org.http4k.routing.RouterMatch$MatchingHandler.invoke(Router.kt:60)
    at org.http4k.routing.RouterBasedHttpHandler.invoke(RouterBasedHttpHandler.kt:19)
    at org.http4k.routing.RouterBasedHttpHandler.invoke(RouterBasedHttpHandler.kt:13)
    at com.target.slack.Http4KEmojiManagerApp.invoke(Http4KEmojiManagerApp.kt:28)
    at com.target.slack.Http4KEmojiManagerApp.invoke(Http4KEmojiManagerApp.kt:15)
    at org.http4k.server.HttpExchangeHandler.handle(HttpExchangeHandler.kt:39)
    at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:95)
    at jdk.httpserver/sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:82)
    at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:98)
    at jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:728)
    at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:95)
    at jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:695)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
    at java.base/java.util.concurrent.ForkJoinTask.doExec$$$capture(ForkJoinTask.java:373)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)

Looking at RFC723 Section 3.2.2

A sender MUST NOT generate multiple header fields with the same field name in a message unless either the entire field value for that header field is defined as a comma-separated list [i.e., #(values)] or the header field is a well-known exception (as noted below).

This should be allowed

Reproducible in:

java -version
sw_vers && uname -v # or `ver`

The Slack SDK version

+--- com.slack.api:bolt:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1
|    +--- com.slack.api:slack-api-client:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    |    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt-http4k:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    +--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-servlet:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-socket-mode:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt:1.36.1 (n)
+--- com.slack.api:bolt-http4k:1.36.1 (n)
+--- com.slack.api:bolt-servlet:1.36.1 (n)
+--- com.slack.api:bolt-socket-mode:1.36.1 (n)
+--- com.slack.api:slack-api-model-kotlin-extension:1.36.1 (n)
+--- com.slack.api:slack-api-client-kotlin-extension:1.36.1 (n)
+--- com.slack.api:bolt:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1
|    +--- com.slack.api:slack-api-client:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    |    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt-http4k:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    +--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-servlet:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-socket-mode:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1
|    +--- com.slack.api:slack-api-client:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    |    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt-http4k:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    +--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-servlet:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-socket-mode:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1
|    +--- com.slack.api:slack-api-client:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    |    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt-http4k:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    +--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-servlet:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-socket-mode:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1
|    +--- com.slack.api:slack-api-client:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    |    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt-http4k:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    +--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-servlet:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-socket-mode:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1
|    +--- com.slack.api:slack-api-client:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1
|    |    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    |    +--- com.slack.api:slack-api-client:1.36.1 (*)
+--- com.slack.api:bolt-http4k:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    +--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-servlet:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:bolt-socket-mode:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)
|    +--- com.slack.api:slack-app-backend:1.36.1 (*)
|    \--- com.slack.api:bolt:1.36.1 (*)
+--- com.slack.api:slack-api-model-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model:1.36.1 (*)
+--- com.slack.api:slack-api-client-kotlin-extension:1.36.1
|    +--- com.slack.api:slack-api-model-kotlin-extension:1.36.1 (*)
|    +--- com.slack.api:slack-api-client:1.36.1 (*)

Java Runtime version

openjdk version "17" 2021-09-14
OpenJDK Runtime Environment Temurin-17+35 (build 17+35)
OpenJDK 64-Bit Server VM Temurin-17+35 (build 17+35, mixed mode)

OS info

Multiple..

Requirements

Please make sure if this topic is specific to this SDK. For general questions/issues about Slack API platform or its server-side, could you submit questions at https://my.slack.com/help/requests/new instead. :bow:

Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you agree to those rules.

filmaj commented 7 months ago

Note: I am not super well versed in Java so probably I am messing something up in my app setup 😅

I'm having trouble reproducing this in a standard gradle app using bolt-jetty. If I provide no body, as you suggest, in the HTTP request to the local server, I get an 'invalid request' response. Here is the curl output: 

➜ curl -X POST http://localhost:3000/slack/events -v -H 'Accept: application/x-www-form-urlencoded' -H 'X-Forwarded-Host: proxy1.example.com' -H 'X-Forwarded-Host: proxy2.example.com'
*   Trying [::1]:3000...
* Connected to localhost (::1) port 3000
> POST /slack/events HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/8.4.0
> Accept: application/x-www-form-urlencoded
> X-Forwarded-Host: proxy1.example.com
> X-Forwarded-Host: proxy2.example.com
>
< HTTP/1.1 400 Bad Request
< Date: Fri, 05 Jan 2024 17:51:17 GMT
< Content-Type: text/plain;charset=iso-8859-1
< Content-Length: 15
<
* Connection #0 to host localhost left intact
Invalid Request%

And here is the relevant log of my Bolt for Java app, using Gradle, running the stock 'getting started' app as described on slack.dev:

[qtp405215542-27] WARN com.slack.api.bolt.util.SlackRequestParser - No request pattern detected for

Can you share code on how to set up this app w/ HTTP4K? Sorry for the request, again I am not a regular java user so I am not sure what HTTP4k is 😅

slushpupie commented 7 months ago

@filmaj I dont think this error will happen in bolt-jetty, it looks like the problem is in bolt-http4k , specifically around here: https://github.com/slackapi/java-slack-sdk/blob/main/bolt-http4k/src/main/java/com/slack/api/bolt/http4k/Http4kSlackApp.java#L114-L116

I think the collections api can't handle a map with multiple keys of the same name?

seratch commented 7 months ago

Hi @slushpupie, thank you so much for reporting this issue. Indeed, this is a bug, which should be resolved. I will send a PR to fix it within a few business days (not today as I am still OOO).