search

saros-project / saros

Open Source IDE plugin for distributed collaborative software development
https://www.saros-project.org
GNU General Public License v2.0
158 stars 52 forks source link

[BUILD][S] removes log4j metadata from build artifact #1109

Closed jrtberlin closed 3 years ago

jrtberlin commented 3 years ago

Checklist

Description

This commit removes the Log4j2Plugins.dat from the server jar to enable classpath scanning for log4jas it is not OSGi compliant.

Currently, server builds produce StatusLogger Unrecognized format specifier and StatusLogger Unrecognized conversion specifier errors because the log4j metadata files are not merged together.

~~@Drakulix mentioned (German) that there is a student group working on the move away from OSGi builds. The shadow Gradle plugin and the git history suggests that this was already a work in progress with issues for the picocontainer.~~

This commit results in a working logging component without errors (at least I have not found them) in exchange for a small performance hit on startup. (maybe around a second)

Ideally, the changes in this commit get removed in the future with a different build process (e.g. using shadowJar tasks with the log4j transformer)

This commit does not affect the build artefacts of other components than the server.

Edit: The assumption that the server uses OSGi was wrong.

jrtberlin commented 3 years ago

@Drakulix You are right, that's a good idea. Let me know if this is how you intended the comment.

srossbach commented 3 years ago

The server does not run in an OSGi enviroment so what is the problem here ?

jrtberlin commented 3 years ago

The server does not run in an OSGi enviroment so what is the problem here ?

Oh, I did assume wrong then. There is an issue with the log4j2plugin.dat file. Removing the file results in log4j using classpath scanning and the errors are gone.

I get the following error using the server:

ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [level]
ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [logger]
ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [msg]
ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [n]
ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [level]
ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [logger]
ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [msg]
ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [n]
ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
srossbach commented 3 years ago 
INFO  20:44:52,508 [main] (SarosServer.java:35) Starting server...
INFO  20:44:52,598 [main] (ContainerContext.java:163) initializing context...
DEBUG 20:44:52,823 [main] (XMPPAccountStore.java:146) loading accounts from file: C:\Users\***\.saros\config.dat
DEBUG 20:44:53,483 [main] (XMPPAccountStore.java:184) loaded 2 account(s)
DEBUG 20:44:53,745 [main] (DataTransferManager.java:335) used stream service order for the current XMPP connection: [XMPP-Socks5-Stream-Service, XMPP-IBB-Stream-Service]
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.account.XMPPAccountStore
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.upnp.internal.UPnPAccessImpl
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.upnp.internal.UPnPServiceImpl
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.stun.internal.StunServiceImpl
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.xmpp.XMPPConnectionService
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.stream.Socks5StreamService
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.stream.IBBStreamService
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.internal.DataTransferManager
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.server.preferences.PersistencePreferenceStore
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.server.preferences.ServerPreferences
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.communication.connection.ConnectionHandler
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.DispatchThreadContext
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.internal.XMPPReceiver
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.internal.XMPPTransmitter
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.xmpp.subscription.SubscriptionHandler
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.net.xmpp.contact.XMPPContactsService
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.communication.InfoManager
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: java.lang.String
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.versioning.VersionManager
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.communication.chat.muc.MultiUserChatService
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.communication.chat.single.SingleUserChatService
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.context.ContainerContext
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.negotiation.hooks.SessionNegotiationHookManager
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.negotiation.SessionNegotiationFactory
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.session.SarosSessionManager
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.editor.colorstorage.ColorIDSetStorage
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.negotiation.AdditionalResourceDataFactory
DEBUG 20:44:53,893 [main] (ContainerContext.java:184) created context component: saros.session.ColorNegotiationHook
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.session.ResourceNegotiationTypeHook
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.net.xmpp.filetransfer.XMPPFileTransferManager
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.server.dummies.NullRemoteProgressIndicatorFactory
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.monitoring.remote.RemoteProgressManager
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.observables.FileReplacementInProgressObservable
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.concurrent.watchdog.IsInconsistentObservable
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.server.filesystem.ServerPathFactoryImpl
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.server.filesystem.ServerWorkspaceImpl
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.server.session.ServerSessionContextFactory
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.server.synchronize.ServerUISynchronizerImpl
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.server.net.SubscriptionAuthorizer
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.server.session.NegotiationHandler
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.server.session.JoinSessionRequestHandler
DEBUG 20:44:53,898 [main] (ContainerContext.java:184) created context component: saros.server.net.ServerFeatureAdvertiser
INFO  20:44:53,898 [main] (ContainerContext.java:191) successfully initialized context
DEBUG 20:44:53,908 [main] (DataTransferManager.java:335) used stream service order for the current XMPP connection: [XMPP-Socks5-Stream-Service, XMPP-IBB-Stream-Service]
Nov 27, 2020 8:44:53 PM org.jivesoftware.smack.provider.UrlProviderFileInitializer initialize
INFO: Loading providers for file [classpath:META-INF/core.providers]
Nov 27, 2020 8:44:53 PM org.jivesoftware.smack.provider.UrlProviderFileInitializer initialize
INFO: Loading providers for file [classpath:META-INF/extension.providers]
DEBUG 20:44:54,003 [main] (XMPPConnectionService.java:424) started Socks5 proxy on port: 53475 [listening on all interfaces]
DEBUG 20:44:54,258 [main] (XMPPConnectionService.java:441) using autodetected addresses: [192.168.178.20]
DEBUG 20:44:54,268 [main] (XMPPConnectionService.java:384)  new connection state == CONNECTING
ERROR 20:44:54,298 [main] (XMPPConnectionService.java:386)  new connection state == ERROR
org.jivesoftware.smack.XMPPException: ssaros-con.imp.fu-berlin.de:5222 Exception: Could not connect to ssaros-con.imp.fu-berlin.de:5222.; 
    at org.jivesoftware.smack.XMPPConnection.connectUsingConfiguration(XMPPConnection.java:592) ~[smack-3.4.1.jar:?]
    at org.jivesoftware.smack.XMPPConnection.connect(XMPPConnection.java:1010) ~[smack-3.4.1.jar:?]
    at saros.net.xmpp.XMPPConnectionService.connect(XMPPConnectionService.java:258) [main/:?]
    at saros.communication.connection.ConnectionHandler.connectXMPPInternal(ConnectionHandler.java:276) [main/:?]
    at saros.communication.connection.ConnectionHandler.connect(ConnectionHandler.java:156) [main/:?]
    at saros.server.ServerLifecycle.connectToXMPPServer(ServerLifecycle.java:72) [main/:?]
    at saros.server.ServerLifecycle.initializeContext(ServerLifecycle.java:31) [main/:?]
    at saros.context.AbstractContextLifecycle.start(AbstractContextLifecycle.java:96) [main/:?]

This is the output when running the Server from Eclipse (all I had to do was to copy the log4j2.xml from the eclipse project into the current classpath, even without the file I do not get such errors)

Classpath when running from Eclipse is:

/Z:/git_repository/saros/server/bin/main/
/Z:/git_repository/saros/server/bin/test/
/Z:/git_repository/saros/server/bin/default/
/Z:/git_repository/saros/core/bin/main/
/Z:/git_repository/saros/core/bin/test/
/Z:/git_repository/saros/core/bin/default/
/Z:/gradle_user_home/caches/modules-2/files-2.1/commons-codec/commons-codec/1.3/fd32786786e2adb664d5ecc965da47629dca14ba/commons-codec-1.3.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/commons-io/commons-io/2.0.1/7ffdb02f95af1c1a208544e076cea5b8e66e731a/commons-io-2.0.1.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.8.1/6505a72a097d9270f7a9e7bf42c4238283247755/commons-lang3-3.8.1.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/javax.jmdns/jmdns/3.4.1/7ba2dcf7effb5ca9b00736b2a066fff3dd01d432/jmdns-3.4.1.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/xpp3/xpp3/1.1.4c/9b988ea84b9e4e9f1874e390ce099b8ac12cfff5/xpp3-1.1.4c.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/com.thoughtworks.xstream/xstream/1.4.10/dfecae23647abc9d9fd0416629a4213a3882b101/xstream-1.4.10.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.gnu.inet/libidn/1.15/b5bede3c1c031a827b604da31768ddaf833495c6/libidn-1.15.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.13.3/4e857439fc4fe974d212adaaaa3b118b8b50e3ec/log4j-core-2.13.3.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-1.2-api/2.13.3/6060aef755239b82bbc84bd92eb80ff9f4e48dd7/log4j-1.2-api-2.13.3.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.13.3/ec1508160b93d274b1add34419b897bae84c6ca9/log4j-api-2.13.3.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/xmlpull/xmlpull/1.1.3.1/2b8e230d2ab644e4ecaa94db7cdedbc40c805dfa/xmlpull-1.1.3.1.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/xpp3/xpp3_min/1.1.4c/19d4e90b43059058f6e056f794f0ea4030d60b86/xpp3_min-1.1.4c.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.powermock/powermock-module-junit4/2.0.5/c922fc29c82664e06466a7ce1face1661d688255/powermock-module-junit4-2.0.5.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.powermock/powermock-module-junit4-common/2.0.5/d02a42a4cc6d9229a11b1bc5c37a3f5f2c342d0a/powermock-module-junit4-common-2.0.5.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-4.12.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.powermock/powermock-api-easymock/2.0.5/a4bca999c461a2787026ce161846affba451fee9/powermock-api-easymock-2.0.5.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.easymock/easymock/4.0.1/26f1c39bd323b6ac7fe0c4c8e33bfdffcdde03a2/easymock-4.0.1.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.powermock/powermock-api-support/2.0.5/f7e9d65624f55c9c15ebd89a3a8770d1bb21e49c/powermock-api-support-2.0.5.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.powermock/powermock-core/2.0.5/d5d5ca75413883e00595185d79714e0911c7358e/powermock-core-2.0.5.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-1.3.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.powermock/powermock-reflect/2.0.5/6bca328201936519e08bb1d8fdf37c0a3d7075d0/powermock-reflect-2.0.5.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.javassist/javassist/3.25.0-GA/442dc1f9fd520130bd18da938622f4f9b2e5fba3/javassist-3.25.0-GA.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/cglib/cglib-nodep/3.2.9/27ca91ebc2b82f844e62a7ba8c2c1fdf9b84fa80/cglib-nodep-3.2.9.jar
/Z:/git_repository/saros/core/libs/weupnp.jar
/Z:/git_repository/saros/core/libs/smack-3.4.1.jar
/Z:/git_repository/saros/core/libs/smackx-3.4.1.jar
/Z:/git_repository/saros/libs/picocontainer-2.11.2-patched_relocated.jar
/Z:/gradle_user_home/caches/modules-2/files-2.1/org.apache.commons/commons-collections4/4.2/54ebea0a5b653d3c680131e73fe807bb8f78c4ed/commons-collections4-4.2.jar
srossbach commented 3 years ago

The server does not run in an OSGi enviroment so what is the problem here ?

Oh, I did assume wrong then. There is an issue with the log4j2plugin.dat file. Removing the file results in log4j using classpath scanning and the errors are gone.

Have not build the server with Marven yet. Is this only one big Jar file ? I did not find any starting scripts.

jrtberlin commented 3 years ago

@srossbach I'm building via ./gradlew cleanAll sarosServer and the result is one big jar file. That's why I did assume a bundled jar or an uber jar with possibly two .dat files that are not merged together from core and from the server part. Hope this clears it a bit up.

srossbach commented 3 years ago

Yeah I just read some Stackoverflow threads and it seems the Maven Shadow plugin generally ***** something up. We do not have 2 .dat files (at least I do not know where the second file is coming from).

Please update the commit message as this not an OSGi issue.

jrtberlin commented 3 years ago

@srossbach thank you for looking into it! I also updated the PR text with strikethroughs to keep the reasoning behind this PR clear without confusing future readers.