openzipkin / brave-cassandra

Tracing instrumentation for Cassandra and the DataStax Java Driver
Apache License 2.0
12 stars 14 forks source link

Fix ITTracingSession on Windows #22

Open codefromthecrypt opened 5 years ago

codefromthecrypt commented 5 years ago

Thanks @reta for reporting

We might want to setup jenkins windows to know we are failing here (cc @abesto)

If we used docker, probably we could avoid windows related glitches. This would imply shipping the tracing driver to the docker container though. cc @bsideup

 [INFO] Running brave.cassandra.driver.ITTracingSession
  20:54:13,355 WARN  [main] config.DatabaseDescriptor (DatabaseDescriptor.java:556) - Only 24.977GiB free across all data volumes. Consider adding more capacity to your cluster or removing obsolete snapshots
  20:54:13,363 WARN  [main] config.DatabaseDescriptor (DatabaseDescriptor.java:673) - index_interval has been deprecated and should be removed from cassandra.yaml
  20:54:13,364 WARN  [main] config.DatabaseDescriptor (DatabaseDescriptor.java:677) - Please rename encryption_options as server_encryption_options in the yaml
  20:54:14,170 ERROR [COMMIT-LOG-ALLOCATOR] utils.JVMStabilityInspector (JVMStabilityInspector.java:102) - Exiting due to error while processing commit log during initialization.
org.apache.cassandra.io.FSWriteError: java.nio.file.AccessDeniedException: target\embeddedCassandra\commitlog\CommitLog-6-1553129653826.log
        at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:142) ~[cassandra-all-3.11.4.jar:3.11.4]
        at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:159) ~[cassandra-all-3.11.4.jar:3.11.4]
        at org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:409) ~[cassandra-all-3.11.4.jar:3.11.4]
        at org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager.discardAvailableSegment(AbstractCommitLogSegmentManager.java:473) ~[cassandra-all-3.11.4.jar:3.11.4]
        at org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager.access$200(AbstractCommitLogSegmentManager.java:46) ~[cassandra-all-3.11.4.jar:3.11.4]
        at org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager$1.runMayThrow(AbstractCommitLogSegmentManager.java:115) ~[cassandra-all-3.11.4.jar:3.11.4]
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[cassandra-all-3.11.4.jar:3.11.4]
        at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81) ~[cassandra-all-3.11.4.jar:3.11.4]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.0.56.Final.jar:4.0.56.Final]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.nio.file.AccessDeniedException: target\embeddedCassandra\commitlog\CommitLog-6-1553129653826.log
        at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89) ~[?:?]
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103) ~[?:?]
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108) ~[?:?]
        at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:270) ~[?:?]
        at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:105) ~[?:?]
        at java.nio.file.Files.delete(Files.java:1141) ~[?:?]
        at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:136) ~[cassandra-all-3.11.4.jar:3.11.4]
        ... 9 more
abesto commented 5 years ago

Good thinking. FWIW, we might want to be sparing - there are only 6 Windows nodes under ASF Jenkins (https://cwiki.apache.org/confluence/display/INFRA/Jenkins+node+labels), so we probably shouldn't run everything on Windows too.

reta commented 5 years ago

@adriancole Interesting fact, I accidentally run the ITTracingSession tests with JDK1.8 (should be at least JDK11) and they passed green, no issues. Switching to 11 brought the failures back.

codefromthecrypt commented 5 years ago

linking some related issues..

https://github.com/jsevellec/cassandra-unit/issues/283 https://discuss.lightbend.com/t/cassandra-error-while-starting-application/2614/3

probably least time would be spent switching to docker, just an issue of adding the tracing stuff to docker container classpath.. this might end up taking longer. @bsideup do you have an example of adding local classes to testcontainers java container?

bsideup commented 5 years ago

@adriancole GenericContainer#withCopyFileToContainer can be used to copy files from tests' host into a container. Then it is only a matter of having a reference to the file with classes using your build system. Here is an example of how I did it for Gradle: https://github.com/bsideup/javaagent-boilerplate/blob/57e30bd7504e683d8df9ab935bc3e3a169e72828/build.gradle#L45 https://github.com/bsideup/javaagent-boilerplate/blob/57e30bd7504e683d8df9ab935bc3e3a169e72828/src/test/java/support/GroovyTestApp.java#L13

There are also libraries which help to extract .class files from classloaders.

I hope this helps!

codefromthecrypt commented 5 years ago

@prashantbhardwaj this is not the CassandraCQLUnit or cassandra project. We are users of cassandra.. and also we don't do troubleshooting via github issues even if we did. I would recommend finding the owners of CassandraCQLUnit or any other cassandra support channel to troubleshoot your issue.

GuruprasadBennur commented 5 years ago

no use of this git hub discussions.