search

tronprotocol / java-tron

Java implementation of the Tron whitepaper
GNU Lesser General Public License v3.0
3.7k stars 1.39k forks source link

Expand ARM Architecture Compatibility #5954

Open halibobo1205 opened 3 weeks ago

halibobo1205 commented 3 weeks ago

Background

Java-Tron currently only supports the x86 architecture. Nevertheless, ARM architecture has gained significant traction recently, especially in cloud computing and mobile devices. ARM processors are known for their energy efficiency and cost-effectiveness, making them increasingly popular in data centers, cloud computing, and edge computing scenarios. It will be great to have an option to run Java-Tron using ARM architecture.

Key developments in ARM architecture:

ARM advantages:

Related Issues and PRs

endiaoekoe commented 3 weeks ago

I am totally in favor of extending ARM architecture compatibility. This will allow Java-Tron to run on more platforms and take advantage of the benefits of the ARM architecture, such as higher energy efficiency and lower cost. In my opinion, we can start with the following:

  1. Prioritize ARM support for key dependencies: For example, RocksDB/LevelDB is an important database component in Java-Tron and it is critical to ensure its compatibility on ARM.
  2. Establish an ARM test environment: We need to establish a dedicated ARM test environment to ensure the stability and performance of Java-Tron on ARM.
  3. Collaborate with the community: We can work with the community to solve ARM compatibility issues and share experiences and best practices.
tomatoishealthy commented 3 weeks ago

It sounds great, but I am a novice in ARM architecture. I am curious about the challenges of supporting ARM architecture.

Can you list something like a task list in the future? It is convenient to clearly understand the current status and future challenges.

halibobo1205 commented 3 weeks ago

Here are some common considerations:

[!IMPORTANT]

  1. JDK version compatibility Ensure the JDK version supports ARM Architecture. Consider using ARM-optimized JDK distributions.
    • Linux got support in JDK 9(non-LTS) by JEP 237
    • Windows got support in JDK 16(non-LTS) by JEP 388
    • Macs got support in JDK 17(LTS) by JEP 391

[!IMPORTANT]

  1. Native code JNI (Java Native Interface) or other native code. These native code components need to be recompiled or upgraded for ARM architecture.
    • LevelDBJni
    • RocksDBJni
    • zksnark-java-sdk

[!TIP]

  1. Endianness x86 is little-endian, while some ARM processors may be big-endian. Check if any operations in the code(such as TVM) depend on a specific endianness, especially when handling binary data.

[!Tip]

  1. Memory alignment: ARM architecture may have different memory alignment requirements than x86. Check for code(such as TVM) that assumes specific memory alignments.

[!Tip]

  1. Atomic operations and concurrency Some atomic operations(TVM) may be implemented differently on different architectures. Review concurrent code to ensure it works correctly on ARM as well.

[!CAUTION]

  1. Floating-point arithmetic ARM and x86 may have subtle differences in floating-point precision and behavior. For applications that rely on precise floating-point calculations, comprehensive testing is necessary.

[!TIP]

  1. Performance optimization x86-specific performance optimizations(TVM) may no longer be applicable on ARM. Consider using ARM-specific optimization techniques.

[!IMPORTANT]

  1. Third-party dependencies Ensure all third-party libraries and dependencies support ARM architecture. Some incompatible dependencies may need to be updated or replaced.
    • protoc-gen-grpc-java

[!IMPORTANT]

  1. Build and deployment process:
    • Update build scripts to support ARM architecture.
    • Ensure CI/CD pipelines can be built and tested in ARM environments.
    • Docker support

[!TIP]

  1. Hardware feature dependencies: Check if the code(TVM) relies on x86-specific hardware features. Alternatives may need to be found for ARM.

[!TIP]

  1. System calls and OS interactions If the code makes direct system calls, adjustments may be needed for ARM.

[!IMPORTANT]

  1. Cross-platform testing
    • Establish comprehensive test suites to ensure the functionality works correctly on ARM.
    • Conduct performance benchmarking to compare x86 and ARM performance differences.
317787106 commented 3 weeks ago

@halibobo1205 Do you want to support ARM Architecture and latest JVM version at the same time ? Or just support ARM Architecture using JDK8 ?

halibobo1205 commented 3 weeks ago

@317787106 JVM officially supports ARM:

According to Oracle Java SE Support Roadmap, JDK9 and JDK16 are non-LTS, and JDK 17 is LTS. Based on the above information, I propose that ARM support JDK17 as a minimum.

[!Warning] This is the last planned update of JDK 17 under the NFTC. Updates after September 2024 will be licensed under the Java SE OTN License (OTN) and production use beyond the limited free grants of the OTN license will require a fee.

endiaoekoe commented 3 weeks ago

Here are some common considerations:

Regarding JDK version compatibility. You recommend using an ARM-optimized JDK distribution. What specific ARM-optimized JDK distributions do you recommend? What are their performance and stability advantages?

abn2357 commented 3 weeks ago

When is the expected completion time for this work? It sounds like a big project.

halibobo1205 commented 3 weeks ago

@endiaoekoe I propose that ARM support JDK17 as a minimum.

halibobo1205 commented 3 weeks ago

@abn2357 Tron currently only supports JDK 8, based on the above information, JDK17 supports ARM fully, perhaps Tron needs to upgrade JDK17 first, which is another big project.

zeusoo001 commented 3 weeks ago

@halibobo1205 It sounds great, and I look forward to your implementation. I see that there may be subtle differences in floating point precision and behavior between ARM and x86. When supporting it, be sure to ensure data consistency. Also investigate whether there are other places that may cause data inconsistency.

Murphytron commented 3 weeks ago

This issue has been added to the core devs community call #22, welcome to share the latest progress @halibobo1205, and discuss together with @endiaoekoe @tomatoishealthy @317787106 @zeusoo001 @abn2357.

halibobo1205 commented 3 weeks ago
1. JDK version compatibility After some brief research, I found ARM 64-bit versions of JDK 8 available. cc @endiaoekoe @317787106 @abn2357 Provider Linux Mac Windows Notes
Oracle ✓ ✓ ✗ • Official support
• Requires payment for commercial use
Eclipse Temurin ✓ ✗ ✗ • Free OpenJDK
• Regularly updated and supported by the Adoptium community
Azul Zulu ✓ ✓ ✗ • Free OpenJDK
• Full enterprise version requires payment
BellSoft Liberica ✓ ✓ ✗ • Free OpenJDK for all users
• Relatively less well-known
Amazon Corretto ✓ ✓ ✗ • Free OpenJDK
• long-term support by Amazon
• Amazon runs Corretto internally on thousands of production services
halibobo1205 commented 3 weeks ago

6. Floating-point arithmetic known issues:

Unfortunately, Tron does use Math.pow() for floating-point calculations for the Bancor trading pair in ExchangeProcessor:

private long exchangeToSupply(long balance, long quant) {
    logger.debug("balance: " + balance);
    long newBalance = balance + quant;
    logger.debug("balance + quant: " + newBalance);

    double issuedSupply = -supply * (1.0 - Math.pow(1.0 + (double) quant / newBalance, 0.0005));
    logger.debug("issuedSupply: " + issuedSupply);
    long out = (long) issuedSupply;
    supply += out;

    return out;
  }

  private long exchangeFromSupply(long balance, long supplyQuant) {
    supply -= supplyQuant;

    double exchangeBalance =
        balance * (Math.pow(1.0 + (double) supplyQuant / supply, 2000.0) - 1.0);
    logger.debug("exchangeBalance: " + exchangeBalance);

    return (long) exchangeBalance;
  }

Test case

 @Test
  public void testPow() {
    double x = 29218;
    double q = 4761432;
    double ret = Math.pow(1.0 + x / q, 0.0005);
    double ret2 = StrictMath.pow(1.0 + x / q, 0.0005);

    System.out.printf("%s%n", doubleToHex(ret)); //  3ff000033518c576
    System.out.printf("%s%n", doubleToHex(ret2)); // 3ff000033518c575
    Assert.assertEquals(0, Double.compare(ret, ret2)); // fail in jdk8_X86, success in jdk8_ARM64
  }

  public static String doubleToHex(double input) {
    // Convert the starting value to the equivalent value in a long
    long doubleAsLong = Double.doubleToRawLongBits(input);
    // and then convert the long to a hex string
    return Long.toHexString(doubleAsLong);
  }

Tron Should Use StrictMath to Avoid Cross-Platform Consistency Issues. To help ensure the portability on ARM for Java-Tron, I suggest a new proposal to convert Math to StrictMath. cc @zeusoo001

317787106 commented 3 weeks ago

@halibobo1205 First support JDK8 on mac ARM and then extend to support JDK17 on linux and mac ARM may be smooth.

tomatoishealthy commented 3 weeks ago

JDK version compatibility After some brief research, I found ARM 64-bit versions of JDK 8 available. cc @endiaoekoe @317787106 @abn2357

Does this mean that there is no longer a dependency between ARM architecture upgrade and JDK upgrade?

In addition, TRON only focuses on Oracle JDK, right?

halibobo1205 commented 3 weeks ago

@tomatoishealthy, Oops! I'm sorry. I missed that java-tron currently only supports the Oracle JDK.

I found some issues that seem to indicate that OpenJDK is missing the JavaFX module and that Tron uses javafx.util.Pair, Odyssey-v3.7 released Mar 17, 2020, fix this by #2510. Until then, it seems to work with OpenJDK + openjfx. After that, Tron seems to be able to support OpenJDK. But Tron still claims to support only Oracle JDK8, is there any other reason? cc @317787106 @zeusoo001

halibobo1205 commented 3 weeks ago

2. Native code

JNI (Java Native Interface) or other native code. These native code components(JNI) must be recompiled or upgraded for ARM64 architecture and may include, but not limited to, the following.

halibobo1205 commented 3 weeks ago

8. Third-party dependencies

Ensure all third-party libraries and dependencies support ARM architecture. Some incompatible dependencies may need to be updated or replaced, including, but not limited to, the following.

halibobo1205 commented 2 weeks ago

[!Warning] This is the last planned update of JDK 17 under the NFTC. Updates after September 2024 will be licensed under the Java SE OTN License (OTN) and production use beyond the limited free grants of the OTN license will require a fee.

To avoid subsequent charges for commercial use, I recommend switching to OpenJDK.

halibobo1205 commented 1 week ago

[!Caution] Strong data consistency and finality Final data consistency is required for blockchain, and it's usually guaranteed by the world state. Unfortunately, Java-Tron doesn't have a world state. We need to think about how to ensure final data consistency.

halibobo1205 commented 6 days ago

1. JDK version compatibility Maybe try to support OpenJDK on ARM?