nextflow-io / nextflow

A DSL for data-driven computational pipelines
http://nextflow.io
Apache License 2.0
2.68k stars 622 forks source link

Using machineType does not set task.cpus #3660

Open BeyondTheProof opened 1 year ago

BeyondTheProof commented 1 year ago

Bug report

Expected behavior and actual behavior

When running on google-batch or google-lifesciences, providing a machineType in the configuration file correctly finds a machine expected resources, but fails to set task.cpus

Steps to reproduce the problem

main.nf:

nextflow.enable.dsl = 2
process PRINT_CPUS {
    container "ubuntu:latest"
    machineType "n1-standard-2"
    output:
    stdout
    script:
    """
    #!/usr/bin/env bash

    echo "CPUs: $task.cpus"    
"""
}

workflow {
    PRINT_CPUS().view()
}

nextflow.config:

google {
    project = "PROJECT_NAME"
    region = "us-central1"
}
process.executor = "google-batch"

Program output

Expected: 2 (number of cpus for n1-standard-2) Actual: 1

Environment

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

bentsherman commented 1 year ago

It turns out the machine type and cpus are independent, and supposedly it is possible to use both settings to pack multiple tasks on the same VM. However, I haven't been able to make it work in my testing... there may be some other Google Batch settings required that I'm missing.