nextflow pull not working if nextflow.config is a symbolic link

[zichner]

Bug report

If the actual nextflow.config file is not located in the top-level directory of a pipeline repository, but only linked there using a symbolic link, nextflow pull (v0.32.0) gives the following error:

ERROR ~ Project config file is malformed -- Cause: No signature of method: groovy.util.ConfigObject.div() is applicable for argument types: (ConfigObject) values: [[:]]
Possible solutions: is(java.lang.Object), min(groovy.lang.Closure), wait(), size(), dump(), find()

Using a symbolic link for nextflow.config was working without problems until NF version 0.31.1

This is likely related to changes due to #863

Is this something that can be fixed or is the use of symbolic links considered an absolute no-go?

Thanks!

[pditommaso]

It should work. Could you provide the steps to replicate the issue?

[zichner]

It seems to be a problem with gitlab.

Here an example (https://gitlab.com/zichner/nextflow-pull-test-gitlab):

$ ./nextflow-0.32.0-all pull zichner/nextflow-pull-test-gitlab -hub gitlab
Checking zichner/nextflow-pull-test-gitlab ...
ERROR ~ Project config file is malformed -- Cause: No signature of method: groovy.util.ConfigObject.div() is applicable for argument types: (ConfigObject) values: [[:]]
Possible solutions: is(java.lang.Object), min(groovy.lang.Closure), wait(), size(), any(), find()

In contrast, without the symbolic link (https://gitlab.com/zichner/nextflow-pull-test-gitlab2):

$ ./nextflow-0.32.0-all pull zichner/nextflow-pull-test-gitlab2 -hub gitlab
Checking zichner/nextflow-pull-test-gitlab2 ...
 downloaded from https://gitlab.com/zichner/nextflow-pull-test-gitlab2.git - revision: f707777999 [master]

[pditommaso]

The problem is that a symlink is stored as a plain file, eg:

{
    "file_name": "nextflow.config",
    "file_path": "nextflow.config",
    "size": 20,
    "encoding": "base64",
    "content_sha256": "c50b0d932cf4304d0910e894892b224b825428d23c26f10e824a683b5a7ac90f",
    "ref": "master",
    "blob_id": "a231dc2ce991cecd261cad3cd2b61362167e8929",
    "commit_id": "1c0812064ae361b02c00cc264fbd03473c640766",
    "last_commit_id": "60101601ac686221ba9a62da9c3b16161df5af5f",
    "content": "Y29uZi9uZXh0Zmxvdy5jb25maWc="
}

If you decode the content:

$ echo "Y29uZi9uZXh0Zmxvdy5jb25maWc=" | base64 --decode
conf/nextflow.config

Basically what is happening is that NF tries to interprets the string conf/nextflow.config as the config content and therefore reports that error.

AFAIK the API does not provide a way to know if a file is a symlink. Not sure how this can be solved. Maybe with some heuristic trying to resolve as a path when it seems relative paths. But it looks fragile.

Why it was working? Because before #863 the exception was ignored and the content of the manifest was ignored.

[pditommaso]

Interestingly GitHub api returns if the file is a symlink, see here.

Can't find for GitLab and I didn't check for BitBucket.

[pditommaso]

I'm closing this for now because symlinks are not supposed to work remotely. If a some point GitLab and Bitbucket will provide an API to detected symbolic links, it could be implemented.