search

broadinstitute / gatk

Official code repository for GATK versions 4 and up
https://software.broadinstitute.org/gatk
Other
1.68k stars 587 forks source link

HDF5Library uses incorrect logger. #3763

Open samuelklee opened 6 years ago

samuelklee commented 6 years ago

Results in a warning message.

lbergelson commented 6 years ago

@samuelklee Is it something in our code or in the hdf5 java or native code that we wrap? What's the error?

samuelklee commented 6 years ago

I think it's simply that HDF5Library uses org.apache.log4j.LogManager rather than org.apache.logging.log4j.LogManager.

samuelklee commented 6 years ago

Here's the warning that pops up:

log4j:WARN No appenders could be found for logger (org.broadinstitute.hdf5.HDF5Library).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
lbergelson commented 6 years ago

Ah, thanks, that sounds like an easy fix then.

samuelklee commented 6 years ago

@lbergelson I forgot about this low hanging fruit. Any chance that you can take care of it?

samuelklee commented 5 years ago

@lbergelson can you take care of this? The PR in the other repo has been open for a while.

Emmalynchen commented 5 years ago

Running into a similar issue when launching cnv_somatic_pair_workflow 1.3 with gatk docker 4.1 on Terra. Any suggestions?

log4j:WARN No appenders could be found for logger (org.broadinstitute.hdf5.HDF5Library).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
22:50:56.367 INFO  DenoiseReadCounts - Reading read-counts file (/cromwell_root/fc-88ae2d4c-8e67-478a-893c-65db97d40c80/9a75a2c6-0c0b-4903-80e6-0082435645c1/CNVSomaticPairWorkflow/ca25a6aa-92b6-4a41-b41a-740cf40b2143/call-CollectCountsTumor/S027-1A-deduped.counts.hdf5)...

Also failing to delocalize files at the DenoiseReadCounts step:

Task CNVSomaticPairWorkflow.DenoiseReadCountsTumor:NA:1 failed. Job exit code 1. 
Check gs://fc-88ae2d4c-8e67-478a-893c-65db97d40c80/65a36270-4934-4791-a57f-2c70c2f42c0e/CNVSomaticPairWorkflow/9c8d57c3-abf7-4805-b612-04d61e8c7727/call-DenoiseReadCountsTumor/stderr for more information. PAPI error code 5. 
10: Failed to delocalize files: failed to copy the following files: "/mnt/local-disk/S027-1A-deduped.denoisedCR.tsv -> gs://fc-88ae2d4c-8e67-478a-893c-65db97d40c80/65a36270-4934-4791-a57f-2c70c2f42c0e/CNVSomaticPairWorkflow/9c8d57c3-abf7-4805-b612-04d61e8c7727/call-DenoiseReadCountsTumor/S027-1A-deduped.denoisedCR.tsv (cp failed: gsutil -q -m cp -L /var/log/google-genomics/out.log /mnt/local-disk/S027-1A-deduped.denoisedCR.tsv gs://fc-88ae2d4c-8e67-478a-893c-65db97d40c80/65a36270-4934-4791-a57f-2c70c2f42c0e/CNVSomaticPairWorkflow/9c8d57c3-abf7-4805-b612-04d61e8c7727/call-DenoiseReadCountsTumor/S027-1A-deduped.denoisedCR.tsv, command failed: CommandException: No URLs matched: /mnt/local-disk/S027-1A-deduped.denoisedCR.tsv\nCommandException: 1 file/object could not be transferred.\n);
samuelklee commented 5 years ago

@Emmalynchen I wouldn't worry about the log4j:WARN messages discussed in this thread---they're just harmless annoyances that pop up because we haven't gotten around to making sure the HDF5Library dependency uses the same logger as the rest of the GATK.

Looking at your initial post (before you edited it), it looks like DenoiseReadCounts is failing because the panel of normals contains different intervals than those in the read-count collection you are trying to denoise:

22:50:58.635 INFO  SVDDenoisingUtils - Validating sample intervals against original intervals used to build panel of normals...
22:50:59.487 INFO  DenoiseReadCounts - Shutting down engine
[May 7, 2019 10:50:59 PM UTC] org.broadinstitute.hellbender.tools.copynumber.DenoiseReadCounts done. Elapsed time: 0.06 minutes.
Runtime.totalMemory()=894959616
java.lang.IllegalArgumentException: Sample intervals must be identical to the original intervals used to build the panel of normals.

You might try asking for more pointers over in the GATK Forums (https://gatkforums.broadinstitute.org/gatk), if you need them. Good luck!

Emmalynchen commented 5 years ago

@samuelklee Thank you for the pointers and catching that even after the edit! I saw the error but thought I was using the right intervals. Will do some digging.

samuelklee commented 5 years ago

@lbergelson did we ever close the loop on this?

samuelklee commented 5 years ago

@droazen can we take care of this? See https://gatkforums.broadinstitute.org/gatk/discussion/24348/log4j-warn-please-initialize-the-log4j-system-properly#latest.

samuelklee commented 3 years ago

@droazen @lbergelson any chance we can close this out?

droazen commented 3 years ago

@samuelklee Sure, would you care to review https://github.com/broadinstitute/hdf5-java-bindings/pull/14 to see whether it fixes this issue?