search

quiltdata / nf-quilt

GNU General Public License v3.0
9 stars 1 forks source link

pkg input not working #231

Closed drernie closed 2 months ago

drernie commented 2 months ago

In particular, it does not seem to install and iterate over files. Currently can only test via 'pkg-test'; need a way to unit test the relevant issue.

drernie commented 2 months ago

The input package is apparently being downloaded okay, but not copied over to the working directory:

Aug-29 17:36:28.824 [FileTransfer-1] DEBUG nextflow.file.FilePorter - Copying foreign file quilt+s3://udp-spec#package=nf-quilt%2fsource@c4e44f6932f13f626b8640d54fa235c1bfea675f1ad3b5b022a99b3aeb18c637 to work dir: /Users/ernest/GitHub/nf-quilt/work/stage-b2bdcd4c-7ade-4451-8b8f-fcabee69c5a9/85/7187f84449798b0c2e1df2c3199dde/udp-spec#package=nf-quilt%2fsource@c4e44f6932f13f626b8640d54fa235c1bfea675f1ad3b5b022a99b3aeb18c637
Aug-29 17:36:28.825 [FileTransfer-1] DEBUG n.quilt.nio.QuiltFileSystemProvider - QuiltFileSystemProvider.canDownload: udp-spec#package=nf-quilt%2fsource@c4e44f6932f13f626b8640d54fa235c1bfea675f1ad3b5b022a99b3aeb18c637 -> /Users/ernest/GitHub/nf-quilt/work/stage-b2bdcd4c-7ade-4451-8b8f-fcabee69c5a9/85/7187f84449798b0c2e1df2c3199dde/udp-spec#package=nf-quilt%2fsource@c4e44f6932f13f626b8640d54fa235c1bfea675f1ad3b5b022a99b3aeb18c637
Aug-29 17:36:28.825 [FileTransfer-1] DEBUG n.quilt.nio.QuiltFileSystemProvider - QuiltFileSystemProvider.download: udp-spec#package=nf-quilt%2fsource@c4e44f6932f13f626b8640d54fa235c1bfea675f1ad3b5b022a99b3aeb18c637 -> /Users/ernest/GitHub/nf-quilt/work/stage-b2bdcd4c-7ade-4451-8b8f-fcabee69c5a9/85/7187f84449798b0c2e1df2c3199dde/udp-spec#package=nf-quilt%2fsource@c4e44f6932f13f626b8640d54fa235c1bfea675f1ad3b5b022a99b3aeb18c637
Aug-29 17:36:28.825 [FileTransfer-1] DEBUG nextflow.quilt.jep.QuiltPackage - QuiltPackage.forParsed[udp-spec#package=nf-quilt%2fsource@c4e44f6932f13f626b8640d54fa235c1bfea675f1ad3b5b022a99b3aeb18c637]
Aug-29 17:36:28.825 [FileTransfer-1] DEBUG nextflow.quilt.jep.QuiltPackage - QuiltPackage.forParsed[udp-spec#package=nf-quilt%2fsource@c4e44f6932f13f626b8640d54fa235c1bfea675f1ad3b5b022a99b3aeb18c637]
Aug-29 17:36:28.883 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Aug-29 17:36:28.885 [Task submitter] INFO  nextflow.Session - [42/ff2858] Submitted process > transfer (1)
Aug-29 17:36:28.911 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: transfer (1); status: COMPLETED; exit: 0; error: -; workDir: /Users/ernest/GitHub/nf-quilt/work/42/ff2858d66d7f657fc65fec6ecab83a]
Aug-29 17:36:28.911 [Task monitor] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FinalizeTask' minSize=10; maxSize=24; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Aug-29 17:36:28.921 [FinalizeTask-1] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=transfer (1); work-dir=/Users/ernest/GitHub/nf-quilt/work/42/ff2858d66d7f657fc65fec6ecab83a
  error [nextflow.exception.MissingFileException]: Missing output file(s) `inputs/**` expected by process `transfer (1)`
Aug-29 17:36:28.938 [FinalizeTask-1] ERROR nextflow.processor.TaskProcessor - Error executing process > 'transfer (1)'
drernie commented 2 months ago

Fixed. Need a test that verifies use of copyDirectory on download. Does that require manually calling 'download' and reading the fake work directory?