Improve error message for missing Git provider configuration

[davidmasp]

Bug report

Hi

As suggested in the gitter I open a issue for this.

Expected behavior and actual behavior

After configuring a instance of my private server in the $HOME/.nextflow/scm I can pull private repositories from the server with nextflow pull but can't run them afterwards. I can also run hem the first time with nextflow run but can't when I try to re-run them.

In order to test it I have cloned this and pushed to a repo in my private git server named hello.

Program output

The first time I run it I get

$ nextflow run dmas/hello -hub hub_name
N E X T F L O W  ~  version 0.30.0
Pulling dmas/hello ...
 downloaded from <GIT_REPO_URL>
Launching `dmas/hello` [mad_woese] - revision: c9b0ec7286 [master]
[warm up] executor > local
[f9/32e605] Submitted process > sayHello (4)
[47/78e060] Submitted process > sayHello (1)
[19/628431] Submitted process > sayHello (3)
[62/4b15fa] Submitted process > sayHello (2)
Bonjour world!
Hola world!
Hello world!
Ciao world!

The second time though

$ nextflow run dmas/hello -hub hub_name
N E X T F L O W  ~  version 0.30.0
Corrupted git repository at path: /home/dmas/.nextflow/assets/dmas/hello

And this is the .nextflow.log when I run nextflow -trace nextflow run dmas/hello.

$ cat .nextflow.log
Aug-28 14:25:17.171 [main] DEBUG nextflow.cli.Launcher - $> /home/dmas/bin/nextflow -trace nextflow run dmas/hello
Aug-28 14:25:17.280 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 0.30.0
Aug-28 14:25:17.733 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/dmas/.nextflow/assets/dmas/hello/.git/config; branch: master; remote: origin; url: <GIT_REPO_URL>
Aug-28 14:25:17.757 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/dmas/.nextflow/assets/dmas/hello/.git/config; branch: master; remote: origin; url: <GIT_REPO_URL>
Aug-28 14:25:17.758 [main] DEBUG nextflow.scm.AssetManager - Too bad! Can't find any provider from git config. Check file `/home/dmas/.nextflow/assets/dmas/hello/.git/config`
Aug-28 14:25:17.762 [main] DEBUG nextflow.cli.Launcher - Operation aborted
nextflow.exception.AbortOperationException: Corrupted git repository at path: /home/dmas/.nextflow/assets/dmas/hello
        at nextflow.scm.AssetManager.validateProjectDir(AssetManager.groovy:191)
        at nextflow.scm.AssetManager.build(AssetManager.groovy:133)
        at nextflow.scm.AssetManager.<init>(AssetManager.groovy:112)
        at nextflow.cli.CmdRun.getScriptFile(CmdRun.groovy:284)
        at nextflow.cli.CmdRun.run(CmdRun.groovy:207)
        at nextflow.cli.Launcher.run(Launcher.groovy:428)
        at nextflow.cli.Launcher.main(Launcher.groovy:582)

Steps to reproduce the problem

Can't provide a full reproducible example because this is a local machine but basically I think these steps should reproduce the error.

1. Configure private gitlab server at $HOME/.nextflow/scm
2. Clone this in a folder (not nextflow related)
3. Add a remote pointing to a hello reporitory in your private gitlab server
4. push there
5. nextflow pull username/hello -hub yourserver or nextflow run username/hello -hub yourserver , this works fine
6. nextflow run username/hello this gives the error.

Environment

• Nextflow version: 0.30.0
• Java version: OpenJDK Runtime Environment (build 1.8.0_171-b10)
• Operating system: Linux

[pditommaso]

Umm, this is suspect:

Aug-28 14:25:17.758 [main] DEBUG nextflow.scm.AssetManager - Too bad! Can't find any provider from git config. Check file `/home/dmas/.nextflow/assets/dmas/hello/.git/config`

[pditommaso]

Does that file exist ?

[davidmasp]

Yep,

$ more /home/dmas/.nextflow/assets/dmas/hello/.git/config
[core]
        repositoryformatversion = 0
        filemode = true
        logallrefupdates = true
[remote "origin"]
        url = <GIT_REPO_URL>
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master

[pditommaso]

You removed the most important info ie the url :) Use a fake host name but provide the complete url structure.

[davidmasp]

okay sorry

$ more /home/dmas/.nextflow/assets/dmas/hello/.git/config
[core]
        repositoryformatversion = 0
        filemode = true
        logallrefupdates = true
[remote "origin"]
        url =  http://server.name.es/gitlab/dmas/hello.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master

[pditommaso]

I think the problem is that you haven't provided a valid scm configuration file.

[davidmasp]

I see now it could be the endpoint that is wrong, let me check it and let you know

[pditommaso]

Any update on this ?

[davidmasp]

Hi, tried to change the API endpoint with http://server.name.es/gitlab/api/v4 but then everything breaks and I cannot even pull the pipeline.

[davidmasp]

I am unsure what should go in the endpoint field, is the only one I didn't set up in the scm file

[davidmasp]

If this isn't the problem I don't know what I am ding wrong with the scm file.

[pditommaso]

Have you specified the server attribute ?

https://www.nextflow.io/docs/latest/sharing.html#private-server-configuration

[davidmasp]

yes, server is http://server.name.es/gitlab

[pditommaso]

I guess it should be http://server.name.es

[davidmasp]

mmm I think it doesn't work because gitlab is running under a relative url. See this.

with server = 'http://server.name.es'

$ nextflow pull dmas/hello -hub agendas
Checking dmas/hello ...
Cannot find `dmas/hello` -- Make sure exists a GitLab repository at this address http://server.name.es/dmas/hello

[pditommaso]

and how is the endpoint value ?

[pditommaso]

What I mean you need to specify both server and endpoint attributes.

[davidmasp]

Ah okay, I misunderstood the endpoint field. It works flawlessly now. For whatever reason I didn't though to be a configuration problem because it was downloading the repo fine. Thanks a lot for the help

[davidmasp]

for my future me, proper way to set up a scm file with gitlab private server configured with relative path.

server = 'http://server.name.es'
endpoint = 'http://server.name.es/gitlab'

[pditommaso]

I've also realised that the reported error is misleading. I'll modify to provide a more detailed message.

[davidmasp]

great