search

trustin / os-maven-plugin

A Maven plugin that sets various useful properties detected from ${os.name} and ${os.arch} properties.
Apache License 2.0
296 stars 66 forks source link

os.dected.arch=unknown from mips64el when buid grpc-java #24

Closed vans88 closed 6 years ago

vans88 commented 7 years ago

Firstly shou you my platform info:

kennel:4.4.21
Arch: mips64el
OS:Fedora21 
gradle:3.0.2

I have build os-maven-plugin from successfully, source code from github, when I run "mvn install -X " with os-maven-plugin, it dispaly as follows:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:15 min
[INFO] Finished at: 2016-10-13T10:56:16+08:00
[INFO] Final Memory: 90M/3441M
[INFO] ------------------------------------------------------------------------

Though os-maven-plugin has built done, when I build grpc-java( source from github) I am getting the following errors, why the "os.detected.arch" is "unknown"
------------------------------------------------------------------------
Detecting the operating system and CPU architecture
------------------------------------------------------------------------
os.detected.name=linux
os.detected.arch=unknown

FAILURE: Build failed with an exception.

* Where:
Build file '/usr/local/TenFlow/grpc-java/build.gradle' line: 60

* What went wrong:
A problem occurred evaluating root project 'grpc'.
> unknown os.arch: mips64el

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.

BUILD FAILED

Total time: 44.949 secs
Stopped 0 compiler daemon(s).
trustin commented 7 years ago

Could you paste the complete system properties? e.g. System.out.println(System.getProperties())

vans88 commented 7 years ago

Many thanks to trustin, I paste the complete system properties as follows, may be help to you analyze my situation

Java version: 1.8.0_25
Java vendor: Oracle Corporation
Java vendor URL: http://java.oracle.com/
Java home: /usr/lib/jvm/j2sdk-image/jre
Java vm.specification.version: 1.8
Java vm.specification.vendor: Oracle Corporation
Java vm.specification.name: Java Virtual Machine Specification
Java vm.version: 25.25-b02
Java vm.vendor: Oracle Corporation
Java vm.name: OpenJDK 64-Bit Server VM
Java specification.version: 1.8
Java specification.vendor: Oracle Corporation
Java specification.name: Java Platform API Specification
class.version: 52.0
class.path: .
library.path: /usr/java/packages/lib/mipsel:/usr/lib64:/lib64:/lib:/usr/lib
io.tmpdir: /tmp
java compiler: null
java ext dirs: /usr/lib/jvm/j2sdk-image/jre/lib/ext:/usr/java/packages/lib/ext
os.name: Linux
os.arch: mips64el
os.version: 4.4.21+
file.separator: /
path.separator: :
line.separator:

user name: root
user dir: /root/BigData
user home: /root
vans88 commented 7 years ago

Here I post the result of building grpc-java project which use os-maven-plugin to initialize the "os.detected" properties. The following content from pom.xml

......
<extension>
        <groupId>kr.motd.maven</groupId>
        <artifactId>os-maven-plugin</artifactId>
        <version>1.4.1.Final</version>
      </extension>
......

when I run ./gradlew build from the top dir of grpc-java source. I am get the following error:

------------------------------------------------------------------------
Detecting the operating system and CPU architecture
------------------------------------------------------------------------
os.detected.name=linux
os.detected.arch=unknown

FAILURE: Build failed with an exception.

* Where:
Build file '/usr/local/TenFlow/grpc-java/build.gradle' line: 60

* What went wrong:
A problem occurred evaluating root project 'grpc'.
> unknown os.arch: mips64el

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.

BUILD FAILED

Total time: 49.478 secs
Stopped 0 compiler daemon(s).
vans88 commented 7 years ago

I found something useful for my platform--mips4el in the following file: ./src/main/java/kr/motd/maven/os/Detector.java so,reference "arch x86_64" I　add the following lines in this file and "mvn install -X and mvn test" all done successifully,

 **System.out.printf("*******************************\n");
        System.out.printf("normallzed value: %s\n",value);
        if (value.matches("^(x8664|amd64|ia32e|em64t|x64)$")) {
            return "x86_64";
        }
        if (value.matches("^(mips|mips64|mips64el)$")) {
             return "mips64el";
        }**

But when I build grpc-java again the error "unknown os.arch: mips64el" is still exists. Can you give me some tips about my platfom to work around my error, Many Thanks.

vans88 commented 7 years ago

In examples dir from grpc-java source tree, I got the following info, But I didn't know it is right. normallzed value: mips64el [INFO] os.detected.name: linux [INFO] os.detected.arch: mips64el [INFO] os.detected.version: 4.4 [INFO] os.detected.version.major: 4 [INFO] os.detected.version.minor: 4 [INFO] os.detected.release: fedora [INFO] os.detected.release.version: 21 [INFO] os.detected.release.like.fedora: true [INFO] os.detected.classifier: linux-mips64el

vans88 commented 7 years ago

I modified the Detector.java file and add the following lines to support my platform--mips64el.

if (value.matches("^(mips|mips64|mips64el)$")) {
                  return "mips64el";
          }

then "mvn clean install -X" successfully, But when I run "./gradlew test --info" command, I still get the "unknown" error:

</failure>
  </testcase>
  <testcase name="setClassifierWithLikes" classname="com.google.gradle.osdetector.OsDetectorPluginTest" time="0.578">
    <failure message="kr.motd.maven.os.DetectionException: unknown os.arch: mips64el" type="kr.motd.maven.os.DetectionException">kr.motd.maven.os.DetectionException: unknown os.arch: mips64el
        at kr.motd.maven.os.Detector.detect(Detector.java:73)
        at com.google.gradle.osdetector.OsDetector$Impl.&lt;init&gt;(OsDetector.java:117)
        at com.google.gradle.osdetector.OsDetector.getImpl(OsDetector.java:64)
        at com.google.gradle.osdetector.OsDetector.getOs(OsDetector.java:34)
        at com.google.gradle.osdetector.OsDetector_Decorated.getOs(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
        at groovy.lang.MetaBeanProperty.getProperty(MetaBeanProperty.java:60)
        at org.gradle.api.internal.BeanDynamicObject$MetaClassAdapter.getProperty(BeanDynamicObject.java:153)
        at org.gradle.api.internal.BeanDynamicObject.getProperty(BeanDynamicObject.java:107)
        at org.gradle.api.internal.CompositeDynamicObject.getProperty(CompositeDynamicObject.java:78)
        at com.google.gradle.osdetector.OsDetector_Decorated.getProperty(Unknown Source)
vans88 commented 7 years ago

I modified the file Detector.java and added the following line to support my platform. then I have build and test(mvn test) successfully .but,when I run "./gradlew test --info" in top dir of osdetector-gradle-plugin,it doesn't work. Can you tell me the reason? Add the following lines to Detector.java:

 if (value.matches("^(mips|mips64|mips64el)$")) {
             return "mips64el";
        }
if ("mips64el".equals(value)) {
            return "mips64el";
        }

part of osdetector gradle tests results:

com.google.gradle.osdetector.OsDetectorPluginTest > pluginAddsExtensionToProject STANDARD_OUT
    08:32:21.896 [Test worker] DEBUG o.g.i.n.f.s.FileSystemServices - Native-platform file system integration is not available. Continuing with fallback.
    08:32:21.927 [Test worker] DEBUG o.g.i.n.f.s.FileSystemServices - Using UnsupportedSymlink implementation.
    08:32:21.938 [Test worker] DEBUG o.g.i.n.f.s.FileSystemServices - Unable to load org.gradle.internal.nativeintegration.filesystem.jdk7.PosixJdk7FilePermissionHandler. Continuing with fallback org.gradle.internal.nativeintegration.filesystem.services.UnsupportedFilePermissions.
    08:32:24.187 [Test worker] DEBUG o.g.i.n.services.NativeServices - Native-platform process integration is not available. Continuing with fallback.
    08:32:27.029 [Test worker] INFO  c.g.gradle.osdetector.OsDetector - ------------------------------------------------------------------------
    08:32:27.035 [Test worker] INFO  c.g.gradle.osdetector.OsDetector - Detecting the operating system and CPU architecture
    08:32:27.036 [Test worker] INFO  c.g.gradle.osdetector.OsDetector - ------------------------------------------------------------------------
    08:32:27.042 [Test worker] INFO  c.g.gradle.osdetector.OsDetector - os.detected.name=linux
    08:32:27.043 [Test worker] INFO  c.g.gradle.osdetector.OsDetector - os.detected.arch=unknown

com.google.gradle.osdetector.OsDetectorPluginTest > pluginAddsExtensionToProject FAILED
    kr.motd.maven.os.DetectionException: unknown os.arch: mips64el
        at kr.motd.maven.os.Detector.detect(Detector.java:73)
        at com.google.gradle.osdetector.OsDetector$Impl.<init>(OsDetector.java:117)
        at com.google.gradle.osdetector.OsDetector.getImpl(OsDetector.java:64)
        at com.google.gradle.osdetector.OsDetector.getOs(OsDetector.java:34)
        at com.google.gradle.osdetector.OsDetector_Decorated.getOs(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
        at groovy.lang.MetaBeanProperty.getProperty(MetaBeanProperty.java:60)
        at org.gradle.api.internal.BeanDynamicObject$MetaClassAdapter.getProperty(BeanDynamicObject.java:153)
        at org.gradle.api.internal.BeanDynamicObject.getProperty(BeanDynamicObject.java:107)
        at org.gradle.api.internal.CompositeDynamicObject.getProperty(CompositeDynamicObject.java:78)
        at com.google.gradle.osdetector.OsDetector_Decorated.getProperty(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:47)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:227)
        at com.google.gradle.osdetector.OsDetectorPluginTest.pluginAddsExtensionToProject(OsDetectorPluginTest.groovy:31)

at the top of software stack, I found the file Detecotr.java:73 which come from os-maven-plugin, and in the "build.gradle" from osdetector-gradle-plugin, I foud the following dependencies.
**dependencies in build.gradle between os-maven-plugin and osdetector-gradle-plugin:**
dependencies {
  compile gradleApi(),
          localGroovy(),
          'kr.motd.maven:os-maven-plugin:1.4.1.Final'
}
trustin commented 7 years ago

Sorry. Busy this week. Will take a look once I get some time.

vans88 commented 7 years ago

Thank you for your reply.

vans88 commented 7 years ago

I modified Detector.java file and commented the folllowing lines.

final String failOnUnknownOS = allProps.getProperty("failOnUnknownOS");
        log("------------------------------------------------------------------------");
        System.out.println("Debug-detectedArch is: " + detectedArch);
        if (!UNKNOWN.equals(detectedArch)) {
                log("***********************************");
        }
        /*if (!"false".equalsIgnoreCase(failOnUnknownOS)) {
         *   if (UNKNOWN.equals(detectedName)) {
         *       throw new DetectionException("unknown os.name: " + osName);
         *   }
         *   if (UNKNOWN.equals(detectedArch)) {
         *       throw new DetectionException("unknown os.arch: " + osArch);
         *   }
        }*/

But wen I run “./gradlew test” in the top dir of osdetector-gradle-plugin. The original error is still exists.

vans88 commented 7 years ago

@trustin I have tried many methods to fix the front error. Even through I use a fixed value "mips64el" shown at the later section. the situation doesn't get better.

A fixed value "mips64el"

log("------------------------------------------------------------------------");
        log("Detecting the operating system and CPU architecture");
        log("------------------------------------------------------------------------");

        final Properties allProps = new Properties(System.getProperties());
        allProps.putAll(props);

        final String osName = allProps.getProperty("os.name");
        final String osArch = allProps.getProperty("os.arch");
        final String osVersion = allProps.getProperty("os.version");

        final String detectedName = normalizeOs(osName);
        final String detectedArch = "mips64el";

        setProperty(props, DETECTED_NAME, detectedName);
        setProperty(props, DETECTED_ARCH, detectedArch);
vans88 commented 7 years ago

@trustin I'm so sorry to disturb you, Could you tell me how to use os-maven-plugin plugin to get os/arch information solely？ if combined with osdetector-gradle-plugin will complex the test. As for the source code of os-maven-plugin, the modifications added by me is to have mips64el supported in file Detector.java.

A segment of src/main/java/kr/motd/maven/os/Detector.java

 private static String normalizeArch(String value) {
        value = normalize(value);
        if ("mips64el".equals(value)) {
            return "mips64el";
        }
        if (value.matches("^(x8664|amd64|ia32e|em64t|x64)$")) {
            return "x86_64";
        }
        //if (value.matches("^(mips|mips64|mips64el)$")) {
         //    return "mips64el";
        //}
 ... .... ... ... ... .. ...
vans88 commented 7 years ago

My platform cpu info

[root@dscn2 calicoctl]# lscpu
Architecture:          mips64
Byte Order:            Little Endian
CPU(s):                15
On-line CPU(s) list:   0-14
Thread(s) per core:    1
Core(s) per socket:    7
Socket(s):             2
NUMA node(s):          4
Model name:            ICT Loongson-3B R2 (Loongson-3B1500) @ 1100MHz
NUMA node0 CPU(s):     0-2
NUMA node1 CPU(s):     3-6
NUMA node2 CPU(s):     7-10
NUMA node3 CPU(s):     11-14
trustin commented 6 years ago

I'm really sorry for a very late response in spite of your comments. I've just pushed a commit that adds MIPS and MIPSLE detection: 59fd029af72cdeffac6491df41de9b5d15bf3892

On the other hand, the plugin you are using is a Gradle version of this plugin built by Google, which means you need to talk to Google folkd to update their dependencies once I release os-maven-plugin with this fix included.