search

apache / netbeans

Apache NetBeans
https://netbeans.apache.org/
Apache License 2.0
2.66k stars 852 forks source link

Build fails with UnknownHostException: www.w3.org #4920

Closed jgneff closed 2 years ago

jgneff commented 2 years ago

Apache NetBeans version

Apache NetBeans 16 release candidate

What happened

The new file jakartaee_9.xsd in NetBeans 16 causes a direct DNS lookup during the build. No DNS lookups occurred in the builds for NetBeans 13, 14, or 15. The DNS request causes the build to fail when running behind a strict firewall that allows outbound traffic only through an HTTP or HTTPS proxy server.

The failure occurs while building the nb.cluster.enterprise modules. The build fails with the following messages:

     [echo] Building nb.cluster.enterprise modules
     [echo] api.web.webmodule,cloud.amazon,cloud.common,cloud.oracle,el.lexer,glassfish.common,glassfish.eecommon,glassfish.javaee,glassfish.tooling,gradle.javaee,j2ee.ant,j2ee.api.ejbmodule,j2ee.clientproject,j2ee.common,j2ee.core,j2ee.dd,j2ee.dd.webservice,j2ee.ddloaders,j2ee.earproject,j2ee.ejbcore,j2ee.ejbjarproject,j2ee.ejbrefactoring,j2ee.ejbverification,j2ee.genericserver,j2ee.kit,j2ee.platform,j2ee.sun.appsrv,j2ee.sun.dd,j2ee.sun.ddui,j2eeapis,j2eeserver,jakartaee8.api,jakartaee8.platform,jakartaee9.api,jakartaee9.platform,javaee.api,javaee.beanvalidation,javaee.project,javaee.resources,javaee.specs.support,javaee.wildfly,javaee7.api,javaee8.api,jellytools.enterprise,jsp.lexer,libs.amazon,libs.commons_fileupload,libs.elimpl,libs.glassfish_logging,libs.jackson,libs.jstl,maven.j2ee,maven.jaxws,micronaut,payara.common,payara.eecommon,payara.jakartaee,payara.micro,payara.tooling,profiler.j2ee,projectimport.eclipse.web,servletjspapi,spring.webmvc,tomcat5,web.beans,web.bootsfaces,web.client.rest,web.core,web.core.syntax,web.debug,web.el,web.freeform,web.jsf,web.jsf.editor,web.jsf.icefaces,web.jsf.kit,web.jsf.navigation,web.jsf.richfaces,web.jsf12,web.jsf12ri,web.jsf20,web.jsfapi,web.jspparser,web.kit,web.monitor,web.primefaces,web.project,web.refactoring,web.struts,weblogic.common,websocket,websvc.clientapi,websvc.core,websvc.customization,websvc.design,websvc.editor.hints,websvc.jaxws.lightapi,websvc.jaxwsapi,websvc.jaxwsmodel,websvc.kit,websvc.manager,websvc.metro.lib,websvc.owsm,websvc.projectapi,websvc.rest,websvc.restapi,websvc.restkit,websvc.restlib,websvc.saas.codegen.j2ee,websvc.utilities,websvc.websvcapi,websvc.wsstackapi
     [echo] Dependencies: nb.cluster.java,nb.cluster.ide,nb.cluster.extide,nb.cluster.profiler,nb.cluster.webcommon,nb.cluster.websvccommon,nb.cluster.platform,nb.cluster.harness
     [echo] ant version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019
     [echo] Java version: 11.0.16
     [echo] JAVA_HOME: /usr/lib/jvm/java-11-openjdk-amd64
     [echo] Bootstrapping NetBeans-specific Ant extensions...
     [echo] Processing build with branch release160 and hash d99b1939ee54e29a68806b0a444a1283e52e61fc
     [echo] Building branch: release160
     [echo] Build number : 16-rc1-d99b1939ee54e29a68806b0a444a1283e52e61fc
     [echo] Building enterprise/j2ee.core...
 [nb-javac] Ignoring source, target and bootclasspath as release has been set
     [echo] Building enterprise/j2ee.dd...
     [echo] generating code generation input file information
  [nbmerge] Failed to build target: all-j2ee.dd

BUILD FAILED
/build/strictly-netbeans/parts/netbeans/build/build-release-temp/nbbuild/build.xml:631: The following error occurred while executing this line:
/build/strictly-netbeans/parts/netbeans/build/build-release-temp/nbbuild/build.xml:626: The following error occurred while executing this line:
/build/strictly-netbeans/parts/netbeans/build/build-release-temp/nbbuild/build.xml:661: The following error occurred while executing this line:
/build/strictly-netbeans/parts/netbeans/build/build-release-temp/nbbuild/build.xml:644: The following error occurred while executing this line:
/build/strictly-netbeans/parts/netbeans/build/build-release-temp/nbbuild/build.xml:626: The following error occurred while executing this line:
/build/strictly-netbeans/parts/netbeans/build/build-release-temp/nbbuild/build.xml:673: The following error occurred while executing this line:
/build/strictly-netbeans/parts/netbeans/build/build-release-temp/enterprise/j2ee.dd/build.xml:167: Failed to parse document at 'http://www.w3.org/2001/xml.xsd'.
java.net.UnknownHostException: www.w3.org

The full stack trace is shown below:

java.net.UnknownHostException: www.w3.org ``` java.net.UnknownHostException: www.w3.org at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:229) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/java.net.Socket.connect(Socket.java:558) at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:182) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:508) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:603) at java.base/sun.net.www.http.HttpClient.(HttpClient.java:276) at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:375) at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:396) at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1253) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081) at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1015) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1592) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) at java.xml/com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:676) at java.xml/com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:150) at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:860) at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:824) at java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) at java.xml/com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:246) at java.xml/com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339) at java.xml/javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:178) at org.netbeans.modules.schema2beansdev.SchemaRep.readSchemaFromLocation(SchemaRep.java:4728) at org.netbeans.modules.schema2beansdev.SchemaRep$Import.readSchema(SchemaRep.java:2374) at org.netbeans.modules.schema2beansdev.SchemaRep.read(SchemaRep.java:4853) at org.netbeans.modules.schema2beansdev.SchemaRep.readSchemaElement(SchemaRep.java:4826) at org.netbeans.modules.schema2beansdev.SchemaRep.readSchemaFromLocation(SchemaRep.java:4767) at org.netbeans.modules.schema2beansdev.SchemaRep$Include.readSchema(SchemaRep.java:2265) at org.netbeans.modules.schema2beansdev.SchemaRep.read(SchemaRep.java:4853) at org.netbeans.modules.schema2beansdev.SchemaRep.readSchemaElement(SchemaRep.java:4826) at org.netbeans.modules.schema2beansdev.SchemaRep.readSchemaFromLocation(SchemaRep.java:4767) at org.netbeans.modules.schema2beansdev.SchemaRep$Include.readSchema(SchemaRep.java:2265) at org.netbeans.modules.schema2beansdev.SchemaRep.read(SchemaRep.java:4853) at org.netbeans.modules.schema2beansdev.SchemaRep.readSchemaElement(SchemaRep.java:4826) at org.netbeans.modules.schema2beansdev.SchemaRep.readDocument(SchemaRep.java:4784) at org.netbeans.modules.schema2beansdev.XMLSchemaParser.process(XMLSchemaParser.java:77) at org.netbeans.modules.schema2beansdev.GenBeans.doIt(GenBeans.java:821) at org.netbeans.modules.s2banttask.Schema2BeansAntTask.execute(Schema2BeansAntTask.java:62) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292) at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at org.apache.tools.ant.Target.execute(Target.java:449) at org.apache.tools.ant.Target.performTasks(Target.java:470) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388) at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:437) at jdk.internal.reflect.GeneratedMethodAccessor138.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at org.apache.tools.ant.Target.execute(Target.java:449) at org.netbeans.nbbuild.NbMerge.fixedModulesBuild(NbMerge.java:193) at org.netbeans.nbbuild.NbMerge.execute(NbMerge.java:274) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292) at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at org.apache.tools.ant.Target.execute(Target.java:449) at org.apache.tools.ant.Target.performTasks(Target.java:470) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388) at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:437) at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105) at org.netbeans.nbbuild.Repeat.execute(Repeat.java:106) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292) at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at java.base/java.util.Vector.forEach(Vector.java:1394) at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:67) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292) at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at org.apache.tools.ant.taskdefs.MacroInstance.execute(MacroInstance.java:391) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292) at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at org.apache.tools.ant.Target.execute(Target.java:449) at org.apache.tools.ant.Target.performTasks(Target.java:470) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388) at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:437) at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292) at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at org.apache.tools.ant.Target.execute(Target.java:449) at org.apache.tools.ant.Target.performTasks(Target.java:470) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388) at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:437) at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105) at org.netbeans.nbbuild.Repeat.execute(Repeat.java:106) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292) at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at java.base/java.util.Vector.forEach(Vector.java:1394) at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:67) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292) at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at org.apache.tools.ant.taskdefs.MacroInstance.execute(MacroInstance.java:391) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292) at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at org.apache.tools.ant.Target.execute(Target.java:449) at org.apache.tools.ant.Target.performTasks(Target.java:470) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388) at org.apache.tools.ant.Project.executeTarget(Project.java:1361) at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.Main.runBuild(Main.java:834) at org.apache.tools.ant.Main.startAnt(Main.java:223) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:284) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:101) ```

How to reproduce

Build NetBeans 16 or later on a machine that blocks DNS traffic on port 53.

Did this work correctly in an earlier version?

Apache NetBeans 15

Operating System

Kernel version: Linux lcy02-amd64-048 5.4.0-128-generic #144-Ubuntu SMP Tue Sep 20 11:00:04 UTC 2022 x86_64

JDK

Java version: 11.0.16

Apache NetBeans packaging

Own source build

Anything else

There's a one-line fix:

diff --git a/enterprise/j2ee.dd/src/org/netbeans/modules/j2ee/dd/impl/resources/jakartaee_9.xsd b/enterprise/j2ee.dd/src/org/netbeans/modules/j2ee/dd/impl/resources/jakartaee_9.xsd
index 16136f9ff3..dd48fd6ead 100644
--- a/enterprise/j2ee.dd/src/org/netbeans/modules/j2ee/dd/impl/resources/jakartaee_9.xsd
+++ b/enterprise/j2ee.dd/src/org/netbeans/modules/j2ee/dd/impl/resources/jakartaee_9.xsd
@@ -49,7 +49,7 @@
   </xsd:annotation>

   <xsd:import namespace="http://www.w3.org/XML/1998/namespace"
-              schemaLocation="http://www.w3.org/2001/xml.xsd"/>
+              schemaLocation="xml.xsd"/>

   <xsd:include schemaLocation="jakartaee_web_services_client_2_0.xsd"/>

That change causes the file jakartaee_9.xsd to import its schema locally like the previous versions:

$ for f in j2ee_1_4.xsd javaee_{5..8}.xsd; \
    do grep -B1 xml.xsd enterprise/j2ee.dd/src/org/netbeans/modules/j2ee/dd/impl/resources/$f; done
  <xsd:import namespace="http://www.w3.org/XML/1998/namespace"
          schemaLocation="xml.xsd"/>
  <xsd:import namespace="http://www.w3.org/XML/1998/namespace"
          schemaLocation="xml.xsd"/>
  <xsd:import namespace="http://www.w3.org/XML/1998/namespace"
              schemaLocation="xml.xsd"/>
  <xsd:import namespace="http://www.w3.org/XML/1998/namespace"
              schemaLocation="xml.xsd"/>
  <xsd:import namespace="http://www.w3.org/XML/1998/namespace"
              schemaLocation="xml.xsd"/>

in the following files:

enterprise/j2ee.dd/src/org/netbeans/modules/j2ee/dd/impl/resources/j2ee_1_4.xsd
enterprise/j2ee.dd/src/org/netbeans/modules/j2ee/dd/impl/resources/javaee_5.xsd
enterprise/j2ee.dd/src/org/netbeans/modules/j2ee/dd/impl/resources/javaee_6.xsd
enterprise/j2ee.dd/src/org/netbeans/modules/j2ee/dd/impl/resources/javaee_7.xsd
enterprise/j2ee.dd/src/org/netbeans/modules/j2ee/dd/impl/resources/javaee_8.xsd

Are you willing to submit a pull request?

Yes

Code of Conduct

Yes

jgneff commented 2 years ago

See the Launchpad question "Build can't resolve www.w3.org (NXDOMAIN)" for more background information.

neilcsmith-net commented 2 years ago

Curious what implications there might be for this at runtime in some environments too?

I wondered why you were hitting this problem - first time I've looked at strictly-netbeans - interesting!

Feel free to open a PR with this change against delivery. No guarantees it'll get in at this stage, though, unless it or something else gets prioritised to force another rc.

jgneff commented 2 years ago

Curious what implications there might be for this at runtime in some environments too?

I would be grateful to have a comment from Josh Juneau (@juneau001) or José Contreras (@pepness) on this issue or my follow-up pull request. It's just a one-line change, but there is more than one version of the xml.xsd file available from the W3C. The current local copy in the build appears to work for all versions of J2EE 1.4, JavaEE 5-8, and Jakarta 9, but perhaps we should replace it with the latest version.

See the section "Versioning policy for this schema document" at the very bottom of the page at Schema document for namespace http://www.w3.org/XML/1998/namespace.

I wondered why you were hitting this problem - first time I've looked at strictly-netbeans - interesting!

Yes, I was very surprised to get it working at all! I think even the Gradle issue can be fixed. But I hit this issue just from building on the Launchpad build farm, which anyone might want to do even for other packaging formats.

No guarantees it'll get in at this stage, though, unless it or something else gets prioritised to force another rc.

I have a workaround for now (-Dcluster.config=basic). When I build that way, I get "Available Plugins (38)" in the Plugins dialog. It seems those are the same module plugins that were built and included directly when I built with -Dcluster.config=release. It's just that now I'm getting them from the plugins portal. Is that right?

juneau001 commented 2 years ago

Thanks for making this PR. I was also wondering why you may have hit this issue, but being in a restricted environment, I've had issues with Maven POMs in the past as well...not particularly caused by this issue however.

The only issue I foresee is keeping up with the latest xml.xsd, but since it does not change very frequently we should be okay to move forward with this PR and document the change.

Thanks @jgneff we appreciate your time and assistance!