search

genepi / imputationserver

Michigan Imputation Server: A new web-based service for imputation that facilitates access to new reference panels and greatly improves user experience and productivity
https://imputationserver.sph.umich.edu/
GNU Affero General Public License v3.0
77 stars 41 forks source link

Incorrect job log path #129

Open mamanambiya opened 9 months ago

mamanambiya commented 9 months ago

Getting java.io.IOException: File '...workspace/job-20231206-180508-110/logs/job.txt' not found in workspace. error while on the disk the file exists in /home/ubuntu/imputationserver/workspace/job-20231206-180508-110/job.txt. There is a missed or additional logs subfolder.

Experiencing error with both versions v1.8.0-beta3 and v1.8.0-beta4, Cloudgene v3.0.0-beta4.

lukfor commented 9 months ago

Hey. Thanks for reporting this. imputation server works only with cloudgene version 2.x

mamanambiya commented 9 months ago

Oh I see. Thanks

mamanambiya commented 7 months ago

Also experiencing that with Cloudgene 2.8.0 with imputationserver 1.6.7 and 1.8.0, with Hadoop 3.3.6.

java.io.FileNotFoundException: File hdfs://0.0.0.0:9000/user/ubuntu/mylogs/job_1706875618134_0006 does not exist.
        at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:1117)
        at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:149)
        at org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1188)
        at org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1185)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:1195)
        at genepi.hadoop.HdfsUtil.merge(HdfsUtil.java:412)
        at genepi.hadoop.HdfsUtil.merge(HdfsUtil.java:405)
        at genepi.hadoop.log.LogCollector.save(LogCollector.java:29)
        at genepi.imputationserver.steps.imputation.ImputationJob.cleanupJob(ImputationJob.java:211)
        at genepi.hadoop.HadoopJob.execute(HadoopJob.java:292)
        at genepi.imputationserver.util.ParallelHadoopJobStep$BackgroundHadoopJob.run(ParallelHadoopJobStep.java:106)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)