jenkinsci / gitlab-branch-source-plugin

A Jenkins Plugin for GitLab Multibranch Pipeline jobs and Folder Organization
https://plugins.jenkins.io/gitlab-branch-source
MIT License
123 stars 93 forks source link

404 Project/Group not found after updating to 621+ #198

Closed AB-xdev closed 2 years ago

AB-xdev commented 2 years ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.332.2 OS: Linux - 5.13.0-1021-aws --- ace-editor:1.1 antisamy-markup-formatter:2.7 apache-httpcomponents-client-4-api:4.5.13-1.0 authentication-tokens:1.4 authorize-project:1.4.0 blueocean:1.25.3 blueocean-autofavorite:1.2.5 blueocean-bitbucket-pipeline:1.25.3 blueocean-commons:1.25.3 blueocean-config:1.25.3 blueocean-core-js:1.25.3 blueocean-dashboard:1.25.3 blueocean-display-url:2.4.1 blueocean-events:1.25.3 blueocean-git-pipeline:1.25.3 blueocean-github-pipeline:1.25.3 blueocean-i18n:1.25.3 blueocean-jwt:1.25.3 blueocean-personalization:1.25.3 blueocean-pipeline-api-impl:1.25.3 blueocean-pipeline-editor:1.25.3 blueocean-pipeline-scm-api:1.25.3 blueocean-rest:1.25.3 blueocean-rest-impl:1.25.3 blueocean-web:1.25.3 bootstrap4-api:4.6.0-4 bootstrap5-api:5.1.3-6 bouncycastle-api:2.26 branch-api:2.1046.v0ca_37783ecc5 build-timeout:1.20 caffeine-api:2.9.3-65.v6a_47d0f4d1fe checks-api:1.7.3 cloudbees-bitbucket-branch-source:765.v5a_2d6a_23c01d cloudbees-folder:6.714.v79e858ef76a_2 command-launcher:1.6 copyartifact:1.46.4 credentials:1087.1089.v2f1b_9a_b_040e4 credentials-binding:523.vd859a_4b_122e6 dark-theme:156.v6cf16af6f9ef display-url-api:2.3.6 docker-commons:1.19 docker-workflow:1.28 durable-task:496.va67c6f9eefa7 echarts-api:5.3.2-1 email-ext:2.88 favorite:2.4.1 font-awesome-api:6.0.0-1 git:4.11.1 git-client:3.11.0 git-server:1.10 github:1.34.3 github-api:1.303-400.v35c2d8258028 github-branch-source:1598.v91207e9f9b_4a_ gitlab-api:5.0.1-69.vcdc19e779072 gitlab-branch-source:625.v85cf3a_400cfe gradle:1.38 handlebars:3.0.8 handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953 htmlpublisher:1.30 jackson2-api:2.13.2.20220328-273.v11d70a_b_a_1a_52 javax-activation-api:1.2.0-3 javax-mail-api:1.6.2-6 jaxb:2.3.0.1 jdk-tool:1.5 jenkins-design-language:1.25.3 jersey2-api:2.35-7 jjwt-api:0.11.2-71.v2722b_b_06a_2a_f jnr-posix-api:3.1.7-3 jquery3-api:3.6.0-3 jsch:0.1.55.2 junit:1.60 locale:144.v1a_998824ddb_3 lockable-resources:2.14 mailer:414.vcc4c33714601 matrix-auth:3.1.1 matrix-project:758.v7a_ea_491852f3 momentjs:1.1.1 okhttp-api:4.9.3-105.vb96869f8ac3a pam-auth:1.7 pipeline-build-step:2.18 pipeline-github-lib:36.v4c01db_ca_ed16 pipeline-graph-analysis:195.v5812d95a_a_2f9 pipeline-input-step:448.v37cea_9a_10a_70 pipeline-milestone-step:101.vd572fef9d926 pipeline-model-api:2.2077.vc78ec45162f1 pipeline-model-definition:2.2077.vc78ec45162f1 pipeline-model-extensions:2.2077.vc78ec45162f1 pipeline-rest-api:2.24 pipeline-stage-step:293.v200037eefcd5 pipeline-stage-tags-metadata:2.2077.vc78ec45162f1 pipeline-stage-view:2.24 pipeline-utility-steps:2.12.0 plain-credentials:1.8 plugin-util-api:2.16.0 popper-api:1.16.1-3 popper2-api:2.11.5-1 pubsub-light:1.16 resource-disposer:0.19 scm-api:608.vfa_f971c5a_a_e9 scm-filter-branch-pr:0.5.1 script-security:1158.v7c1b_73a_69a_08 snakeyaml-api:1.30.1 sse-gateway:1.25 ssh-credentials:277.v95c2fec1c047 ssh-slaves:1.814.vc82988f54b_10 sshd:3.228.v4c9f9e652c86 structs:318.va_f3ccb_729b_71 theme-manager:0.6 timestamper:1.17 token-macro:293.v283932a_0a_b_49 trilead-api:1.57.v6e90e07157e1 variant:1.4 workflow-aggregator:2.7 workflow-api:1144.v61c3180fa_03f workflow-basic-steps:948.v2c72a_091b_b_68 workflow-cps:2689.v434009a_31b_f1 workflow-cps-global-lib:570.v21311f4951f8 workflow-durable-task-step:1130.v8fd69d0b_8857 workflow-job:1180.v04c4e75dce43 workflow-multibranch:712.vc169a_1387405 workflow-scm-step:399.v9b_8f4da_65061 workflow-step-api:625.vd896b_f445a_f8 workflow-support:819.v37d707a_71d9b_ ws-cleanup:0.42 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Docker. Controller: jenkins/jenkins:2.332.2-lts-alpine

Reproduction steps

We use a self hosted GitLab Server (version 14.10.0). After installing the latest version of the gitlab-branch-source-Plugin (currently 625.v85cf3a_400cfe), all webhooks/builds fail - access to Gitlab in general fails - with:

2022-04-28 08:07:08.948+0000 [id=116]   WARNING j.s.a.SCMHeadEvent$DispatcherImpl#log: SCMEventListener.onSCMHeadEvent(SCMHeadEvent) jenkins.branch.MultiBranchProject$SCMEventListenerImpl@77b454ac propagated an exception
org.gitlab4j.api.GitLabApiException: 404 Project Not Found
        at org.gitlab4j.api.AbstractApi.validate(AbstractApi.java:642)
        at org.gitlab4j.api.AbstractApi.get(AbstractApi.java:214)
        at org.gitlab4j.api.ProjectApi.getProject(ProjectApi.java:746)
        at org.gitlab4j.api.ProjectApi.getProject(ProjectApi.java:678)
        at io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource.getGitlabProject(GitLabSCMSource.java:204)
Caused: java.lang.IllegalStateException: Failed to retrieve project group1/subgroup1/project1
        at io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource.getGitlabProject(GitLabSCMSource.java:209)
        at io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource.retrieve(GitLabSCMSource.java:308)
        at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
        at jenkins.scm.api.SCMSource.fetch(SCMSource.java:327)
        at jenkins.branch.MultiBranchProject$SCMEventListenerImpl.processHeadUpdate(MultiBranchProject.java:1587)
        at jenkins.branch.MultiBranchProject$SCMEventListenerImpl.onSCMHeadEvent(MultiBranchProject.java:1191)
        at jenkins.scm.api.SCMHeadEvent$DispatcherImpl.fire(SCMHeadEvent.java:246)
        at jenkins.scm.api.SCMHeadEvent$DispatcherImpl.fire(SCMHeadEvent.java:229)
        at jenkins.scm.api.SCMEvent$Dispatcher.run(SCMEvent.java:546)
        at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:67)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
2022-04-28 08:07:13.178+0000 [id=117]   WARNING j.s.a.SCMHeadEvent$DispatcherImpl#log: SCMEventListener.onSCMHeadEvent(SCMHeadEvent) jenkins.branch.MultiBranchProject$SCMEventListenerImpl@77b454ac propagated an exception
org.gitlab4j.api.GitLabApiException: 404 Project Not Found
        at org.gitlab4j.api.AbstractApi.validate(AbstractApi.java:642)
        at org.gitlab4j.api.AbstractApi.get(AbstractApi.java:214)
        at org.gitlab4j.api.ProjectApi.getProject(ProjectApi.java:746)
        at org.gitlab4j.api.ProjectApi.getProject(ProjectApi.java:678)
        at io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource.getGitlabProject(GitLabSCMSource.java:204)
Caused: java.lang.IllegalStateException: Failed to retrieve project group1/subgroup1/project1
        at io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource.getGitlabProject(GitLabSCMSource.java:209)
        at io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource.retrieve(GitLabSCMSource.java:308)
        at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
        at jenkins.scm.api.SCMSource.fetch(SCMSource.java:327)
        at jenkins.branch.MultiBranchProject$SCMEventListenerImpl.processHeadUpdate(MultiBranchProject.java:1678)
        at jenkins.branch.MultiBranchProject$SCMEventListenerImpl.onSCMHeadEvent(MultiBranchProject.java:1191)
        at jenkins.scm.api.SCMHeadEvent$DispatcherImpl.fire(SCMHeadEvent.java:246)
        at jenkins.scm.api.SCMHeadEvent$DispatcherImpl.fire(SCMHeadEvent.java:229)
        at jenkins.scm.api.SCMEvent$Dispatcher.run(SCMEvent.java:546)
        at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:67)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)

same problem occurs when trying to change the project/owner of a pipeline grafik

2022-04-28 08:51:06.872+0000 [id=14]    WARNING i.j.p.g.GitLabSCMSource$DescriptorImpl#doFillProjectPathItems: Exception caught:org.gitlab4j.api.GitLabApiException: 404 Group Not Found
org.gitlab4j.api.GitLabApiException: 404 Group Not Found
        at org.gitlab4j.api.AbstractApi.validate(AbstractApi.java:642)
        at org.gitlab4j.api.AbstractApi.get(AbstractApi.java:214)
        at org.gitlab4j.api.Pager.<init>(Pager.java:92)
        at org.gitlab4j.api.GroupApi.getProjects(GroupApi.java:436)
        at org.gitlab4j.api.GroupApi.getProjects(GroupApi.java:406)
        at io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource$DescriptorImpl.doFillProjectPathItems(GitLabSCMSource.java:855)
        at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(Unknown Source)
        at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:398)
        at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:410)
        at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:208)
        at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:141)
        at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:558)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
        at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:289)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
        at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:289)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
        at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:289)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
        at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:289)
        at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
        at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
        at org.kohsuke.stapler.Stapler.invoke(Stapler.java:694)
        at org.kohsuke.stapler.Stapler.service(Stapler.java:240)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
        at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:157)
        at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:248)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
        at hudson.plugins.locale.LocaleFilter.doFilter(LocaleFilter.java:42)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
        at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:129)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
        at jenkins.security.ResourceDomainFilter.doFilter(ResourceDomainFilter.java:81)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
        at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
        at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:60)
        at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
        at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:160)
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
        at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:154)
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:94)
        at jenkins.security.AcegiSecurityExceptionFilter.doFilter(AcegiSecurityExceptionFilter.java:52)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
        at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:54)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
        at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
        at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:102)
        at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:93)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
        at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:219)
        at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:213)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
        at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:97)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
        at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:63)
        at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
        at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:111)
        at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:172)
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
        at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:53)
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
        at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:86)
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
        at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
        at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:38)
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:516)
        at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
        at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
        at java.base/java.lang.Thread.run(Unknown Source)

The project and groups exist, and the jenkins-account on our GitLab server can access them.

The problem is not present when using the previous version 1.5.9.

Additional information:

Expected Results

No problems; projects/groups can be found.

Actual Results

See above.

Anything else?

No response

AB-xdev commented 2 years ago

Update:

In the meantime while writing this issue we managed to fix the problem by setting the availability scope of the API-access token (used inside Manage>Configuration) to GLOBAL (from SYSTEM).

Now everything works fine.