Open wzorgdrager opened 6 years ago
Update: what I tried now is to 'shade' akka in the twitter4s library into an alternative package with the following lines in my build:
assemblyShadeRules in assembly := Seq(
ShadeRule
.rename("akka.**" -> "twitterAkka.@1")
.inAll
)
The shading works fine, but I then run into the following runtime error:
com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'twitterAkka'
at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:156)
at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:149)
at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:176)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:188)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:193)
at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:250)
at twitterAkka.actor.ActorSystem$Settings.<init>(ActorSystem.scala:315)
at twitterAkka.actor.ActorSystemImpl.<init>(ActorSystem.scala:683)
at twitterAkka.actor.ActorSystem$.apply(ActorSystem.scala:245)
at twitterAkka.actor.ActorSystem$.apply(ActorSystem.scala:288)
at twitterAkka.actor.ActorSystem$.apply(ActorSystem.scala:233)
at com.danielasfregola.twitter4s.TwitterRestClient$.$lessinit$greater$default$3(TwitterRestClient.scala:31)
at com.danielasfregola.twitter4s.TwitterRestClient$.apply(TwitterRestClient.scala:71)
at org.codefeedr.plugin.twitter.stages.TwitterTrendingStatusSource.getRestClient(TwitterTrendingStatusInput.scala:80)
at org.codefeedr.plugin.twitter.stages.TwitterTrendingStatusSource.requestTrending(TwitterTrendingStatusInput.scala:126)
at org.codefeedr.plugin.twitter.stages.TwitterTrendingStatusSource.run(TwitterTrendingStatusInput.scala:91)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:86)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:55)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:94)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:264)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
at java.lang.Thread.run(Thread.java:748)
Hope you can help.
My apologies for the late reply.
It seems that it is having problems creating the ActorSystem...? At least, this is my best guess. What happens if you use the function withActorSystem
and you pass the actor system as a parameter?
Regards, D.
So I am finding a similar issues for unrelated reasons. The reason as to why shading isn't working in this case is because of how Akka names its packages (ie akka.blah.blah). The relocation process ends up changing more than is intended however due to this.
Akka has reference.conf
files that it uses to keep defaults. The structure of which, when referenced in text looks similar "akka.blah.blah". I suspect it changes this in the code thinking that it is a stringified class name, resulting in your case looking for a key twitterAkka.blah.blah
However the associated reference.conf
files are not changed as well and have the original akka.blah.blah
path.
Yes, you're right on that. We resolved this back then by downgrading some versions of dependencies we were using. However, in the long run that isn't really a good solution.
Hi,
I'm currently running a Twitter4s application in a Flink cluster. And I'm running into the following error:
Basically what my application does:
Here some snippets of my code:
And the
startStream
method:I think this error is caused by version conflicts related to Akka (Flink also uses Akka), but I have no idea how to solve it. The Twitter4s version I'm using: 5.5