I have issue pulling from a private gitlab repository, and the error is unclear for me. Maybe clarifying it would be helpful for other users.
Expected behavior and actual behavior
As described on slack (initial problem description) , I am trying to pull an internal nextflow pipeline from within my company network. Our code is stored on gitlab.
The error obtained is not clear, It would be nice to be able to see the html page that seems to be returned . I am assuming it is a 404.
Following nextflow documentation, I updated my ~/.nextflow/scm file:
I tried with a personal token or a project token, both returned the same issue
then using either nextflow pull or nextflow run I always retrieve the same error:
nextflow run subproject/project_name -hub company_gitlab
Checking https://my_company_gitlab_server/some_user/nfcoreranseqfork ...
WARN: Cannot read project manifest -- Cause: Unable to determine the current character, it is not a string, number, array, or object
The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<!DOCTYPE html>
^
Unable to determine the current character, it is not a string, number, array, or object
The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<!DOCTYPE html>
^
It looks like a 404 but the log is not informative enough, nor do I see the html page returned to help me debug this.
I would like to be able to see something more informative.
Program output
May-07 11:47:30.242 [main] DEBUG nextflow.cli.Launcher - $> nextflow run subproject/project_name -hub company_gitlab
May-07 11:47:30.304 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 23.10.1
May-07 11:47:30.329 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/myusername/.nextflow/plugins; core-plugins: nf-amazon@2.1.4,nf-azure@1.3.3,nf-cloudcache@0.3.0,nf-codecommit@0.1.5,nf-console@1.0.6,nf-ga4gh@1.1.0,nf-google@1.8.3,nf-tower@1.6.3,nf-wave@1.0.1
May-07 11:47:30.338 [main] INFO o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
May-07 11:47:30.339 [main] INFO o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
May-07 11:47:30.342 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
May-07 11:47:30.350 [main] INFO org.pf4j.AbstractPluginManager - No plugins
May-07 11:47:30.362 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /home/myusername/.nextflow/scm
May-07 11:47:30.799 [main] DEBUG nextflow.scm.ProviderConfig - Detected SCM config: [providers:[company_gitlab:[server:https://gitlab.company.com/team/directory, user:myusername, platform:gitlab, token:******]]]
May-07 11:47:30.812 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
May-07 11:47:30.826 [main] INFO nextflow.cli.CmdRun - Pulling subproject/project_name ...
May-07 11:47:30.828 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials ****:-] -> https://gitlab.company.com/team/directory/api/v4/projects/subproject%2Fproject_name
May-07 11:47:31.329 [main] WARN nextflow.scm.AssetManager - Cannot read project manifest -- Cause: Unable to determine the current character, it is not a string, number, array, or object
The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<!DOCTYPE html>
^
groovy.json.JsonException: Unable to determine the current character, it is not a string, number, array, or object
The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<!DOCTYPE html>
^
at org.apache.groovy.json.internal.JsonParserCharArray.decodeValueInternal(JsonParserCharArray.java:202)
at org.apache.groovy.json.internal.JsonParserCharArray.decodeValue(JsonParserCharArray.java:153)
at org.apache.groovy.json.internal.JsonParserCharArray.decodeFromChars(JsonParserCharArray.java:43)
at org.apache.groovy.json.internal.JsonParserCharArray.parse(JsonParserCharArray.java:380)
[... log too long]
May-07 11:47:31.337 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials ****:-] -> https://gitlab.company.com/team/project/api/v4/projects/subproject%2Fproject_name
May-07 11:47:31.512 [main] ERROR nextflow.cli.Launcher - @unknown
groovy.json.JsonException: Unable to determine the current character, it is not a string, number, array, or object
The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<!DOCTYPE html>
^
at org.apache.groovy.json.internal.JsonParserCharArray.decodeValueInternal(JsonParserCharArray.java:202)
at org.apache.groovy.json.internal.JsonParserCharArray.decodeValue(JsonParserCharArray.java:153)
at org.apache.groovy.json.internal.JsonParserCharArray.decodeFromChars(JsonParserCharArray.java:43)
at org.apache.groovy.json.internal.JsonParserCharArray.parse(JsonParserCharArray.java:380)
at org.apache.groovy.json.internal.BaseJsonParser.parse(BaseJsonParser.java:113)
at groovy.json.JsonSlurper.parseText(JsonSlurper.java:204)
at nextflow.scm.RepositoryProvider.memoizedMethodPriv$invokeAndParseResponseString(RepositoryProvider.groovy:283)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1254)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:1036)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1019)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:97)
at nextflow.scm.RepositoryProvider$_closure3.doCall(RepositoryProvider.groovy)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
at groovy.lang.Closure.call(Closure.java:427)
at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137)
at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137)
at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113)
at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136)
at groovy.lang.Closure.call(Closure.java:416)
at nextflow.scm.RepositoryProvider.invokeAndParseResponse(RepositoryProvider.groovy)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:48)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:189)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:57)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
at nextflow.scm.GitlabRepositoryProvider.getDefaultBranch(GitlabRepositoryProvider.groovy:60)
at nextflow.scm.GitlabRepositoryProvider.getContentUrl(GitlabRepositoryProvider.groovy:88)
at nextflow.scm.GitlabRepositoryProvider.readBytes(GitlabRepositoryProvider.groovy:115)
at nextflow.scm.RepositoryProvider.validateFor(RepositoryProvider.groovy:310)
at nextflow.scm.AssetManager.checkValidRemoteRepo(AssetManager.groovy:376)
at nextflow.scm.AssetManager.download(AssetManager.groovy:584)
at nextflow.cli.CmdRun.getScriptFile0(CmdRun.groovy:529)
at nextflow.cli.CmdRun.getScriptFile(CmdRun.groovy:462)
at nextflow.cli.CmdRun.run(CmdRun.groovy:317)
at nextflow.cli.Launcher.run(Launcher.groovy:500)
at nextflow.cli.Launcher.main(Launcher.groovy:672)
Environment
Nextflow version: 23.10.1
Java version: openjdk 11.0.22 2024-01-16
Operating system: Ubuntu 22.04
Bash version: bash -> 5.0.17(1)
Workaround
As a workaround we just pull from the repo directly and execture main.nf ourselves
I have issue pulling from a private gitlab repository, and the error is unclear for me. Maybe clarifying it would be helpful for other users.
Expected behavior and actual behavior
As described on slack (initial problem description) , I am trying to pull an internal nextflow pipeline from within my company network. Our code is stored on gitlab.
The error obtained is not clear, It would be nice to be able to see the html page that seems to be returned . I am assuming it is a 404.
Following nextflow documentation, I updated my ~/.nextflow/scm file:
I tried with a personal token or a project token, both returned the same issue
then using either nextflow pull or nextflow run I always retrieve the same error:
It looks like a 404 but the log is not informative enough, nor do I see the html page returned to help me debug this.
I would like to be able to see something more informative.
Program output
Environment
Workaround
As a workaround we just pull from the repo directly and execture main.nf ourselves