Open halibobo1205 opened 3 weeks ago
I am totally in favor of extending ARM architecture compatibility. This will allow Java-Tron to run on more platforms and take advantage of the benefits of the ARM architecture, such as higher energy efficiency and lower cost. In my opinion, we can start with the following:
It sounds great, but I am a novice in ARM architecture. I am curious about the challenges of supporting ARM architecture.
Can you list something like a task list in the future? It is convenient to clearly understand the current status and future challenges.
Here are some common considerations:
[!IMPORTANT]
- JDK version compatibility Ensure the JDK version supports ARM Architecture. Consider using ARM-optimized JDK distributions.
[!IMPORTANT]
- Native code JNI (Java Native Interface) or other native code. These native code components need to be recompiled or upgraded for ARM architecture.
- LevelDBJni
- RocksDBJni
- zksnark-java-sdk
[!TIP]
- Endianness x86 is little-endian, while some ARM processors may be big-endian. Check if any operations in the code(such as TVM) depend on a specific endianness, especially when handling binary data.
[!Tip]
- Memory alignment: ARM architecture may have different memory alignment requirements than x86. Check for code(such as TVM) that assumes specific memory alignments.
[!Tip]
- Atomic operations and concurrency Some atomic operations(TVM) may be implemented differently on different architectures. Review concurrent code to ensure it works correctly on ARM as well.
[!CAUTION]
- Floating-point arithmetic ARM and x86 may have subtle differences in floating-point precision and behavior. For applications that rely on precise floating-point calculations, comprehensive testing is necessary.
[!TIP]
- Performance optimization x86-specific performance optimizations(TVM) may no longer be applicable on ARM. Consider using ARM-specific optimization techniques.
[!IMPORTANT]
- Third-party dependencies Ensure all third-party libraries and dependencies support ARM architecture. Some incompatible dependencies may need to be updated or replaced.
- protoc-gen-grpc-java
[!IMPORTANT]
- Build and deployment process:
- Update build scripts to support ARM architecture.
- Ensure CI/CD pipelines can be built and tested in ARM environments.
- Docker support
[!TIP]
- Hardware feature dependencies: Check if the code(TVM) relies on x86-specific hardware features. Alternatives may need to be found for ARM.
[!TIP]
- System calls and OS interactions If the code makes direct system calls, adjustments may be needed for ARM.
[!IMPORTANT]
- Cross-platform testing
- Establish comprehensive test suites to ensure the functionality works correctly on ARM.
- Conduct performance benchmarking to compare x86 and ARM performance differences.
@halibobo1205 Do you want to support ARM Architecture and latest JVM version at the same time ? Or just support ARM Architecture using JDK8 ?
@317787106 JVM officially supports ARM:
According to Oracle Java SE Support Roadmap, JDK9 and JDK16 are non-LTS, and JDK 17 is LTS. Based on the above information, I propose that ARM support JDK17 as a minimum.
[!Warning] This is the last planned update of JDK 17 under the NFTC. Updates after September 2024 will be licensed under the Java SE OTN License (OTN) and production use beyond the limited free grants of the OTN license will require a fee.
Here are some common considerations:
Regarding JDK version compatibility. You recommend using an ARM-optimized JDK distribution. What specific ARM-optimized JDK distributions do you recommend? What are their performance and stability advantages?
When is the expected completion time for this work? It sounds like a big project.
@endiaoekoe I propose that ARM support JDK17 as a minimum.
@abn2357 Tron currently only supports JDK 8, based on the above information, JDK17 supports ARM fully, perhaps Tron needs to upgrade JDK17 first, which is another big project.
@halibobo1205 It sounds great, and I look forward to your implementation. I see that there may be subtle differences in floating point precision and behavior between ARM and x86. When supporting it, be sure to ensure data consistency. Also investigate whether there are other places that may cause data inconsistency.
This issue has been added to the core devs community call #22, welcome to share the latest progress @halibobo1205, and discuss together with @endiaoekoe @tomatoishealthy @317787106 @zeusoo001 @abn2357.
1. JDK version compatibility After some brief research, I found ARM 64-bit versions of JDK 8 available. cc @endiaoekoe @317787106 @abn2357 | Provider | Linux | Mac | Windows | Notes |
---|---|---|---|---|---|
Oracle | ✓ | ✓ | ✗ | • Official support • Requires payment for commercial use |
|
Eclipse Temurin | ✓ | ✗ | ✗ | • Free OpenJDK • Regularly updated and supported by the Adoptium community |
|
Azul Zulu | ✓ | ✓ | ✗ | • Free OpenJDK • Full enterprise version requires payment |
|
BellSoft Liberica | ✓ | ✓ | ✗ | • Free OpenJDK for all users • Relatively less well-known |
|
Amazon Corretto | ✓ | ✓ | ✗ | • Free OpenJDK • long-term support by Amazon • Amazon runs Corretto internally on thousands of production services |
6. Floating-point arithmetic known issues:
Unfortunately, Tron does use Math.pow()
for floating-point calculations for the Bancor trading pair in ExchangeProcessor:
private long exchangeToSupply(long balance, long quant) {
logger.debug("balance: " + balance);
long newBalance = balance + quant;
logger.debug("balance + quant: " + newBalance);
double issuedSupply = -supply * (1.0 - Math.pow(1.0 + (double) quant / newBalance, 0.0005));
logger.debug("issuedSupply: " + issuedSupply);
long out = (long) issuedSupply;
supply += out;
return out;
}
private long exchangeFromSupply(long balance, long supplyQuant) {
supply -= supplyQuant;
double exchangeBalance =
balance * (Math.pow(1.0 + (double) supplyQuant / supply, 2000.0) - 1.0);
logger.debug("exchangeBalance: " + exchangeBalance);
return (long) exchangeBalance;
}
Test case
@Test
public void testPow() {
double x = 29218;
double q = 4761432;
double ret = Math.pow(1.0 + x / q, 0.0005);
double ret2 = StrictMath.pow(1.0 + x / q, 0.0005);
System.out.printf("%s%n", doubleToHex(ret)); // 3ff000033518c576
System.out.printf("%s%n", doubleToHex(ret2)); // 3ff000033518c575
Assert.assertEquals(0, Double.compare(ret, ret2)); // fail in jdk8_X86, success in jdk8_ARM64
}
public static String doubleToHex(double input) {
// Convert the starting value to the equivalent value in a long
long doubleAsLong = Double.doubleToRawLongBits(input);
// and then convert the long to a hex string
return Long.toHexString(doubleAsLong);
}
Tron Should Use StrictMath to Avoid Cross-Platform Consistency Issues. To help ensure the portability on ARM for Java-Tron, I suggest a new proposal to convert Math to StrictMath. cc @zeusoo001
@halibobo1205 First support JDK8 on mac ARM and then extend to support JDK17 on linux and mac ARM may be smooth.
JDK version compatibility After some brief research, I found ARM 64-bit versions of JDK 8 available. cc @endiaoekoe @317787106 @abn2357
Does this mean that there is no longer a dependency between ARM architecture upgrade and JDK upgrade?
In addition, TRON only focuses on Oracle JDK, right?
@tomatoishealthy, Oops! I'm sorry. I missed that java-tron currently only supports the Oracle JDK.
I found some issues that seem to indicate that OpenJDK is missing the JavaFX module and that Tron uses javafx.util.Pair
, Odyssey-v3.7 released Mar 17, 2020, fix this by #2510. Until then, it seems to work with OpenJDK + openjfx. After that, Tron seems to be able to support OpenJDK. But Tron still claims to support only Oracle JDK8, is there any other reason? cc @317787106 @zeusoo001
2. Native code
JNI (Java Native Interface) or other native code. These native code components(JNI) must be recompiled or upgraded for ARM64 architecture and may include, but not limited to, the following.
[ ] LevelDBJni fusesource has stopped maintaining this project, no updates since Oct 17, 2013, forked by @halibobor, thanks to @folowing
[ ] RocksDBJni 6.29.4.1+ RocksJava support for ARM64 architecture
6.29.4.1+ is incompatible with leveldb: org.rocksdb.RocksDBException: bad block contents in file output-directory/database/asset-issue-v2/MANIFEST-000428, 5.15.10 is compatible with leveldb.
Toolkit.jar db convert
without safe mode -> ✅ rocksDB-5.15.10 -> ❌ rocksDB-6.29.4.1+ won't work
DBConvert.jar
or Toolkit.jar db convert --safe
with safe mode -> ✅ rocksDB-5.15.10 -> ✅ rocksDB-6.29.4.1+ is okToolkit.jar db convert
should force safe mode.Toolkit.jar db convert
without safe mode scenario.[x] zksnark-java-sdk is upgraded for ARM64 architecture since GreatVoyage-v4.7.0.1
8. Third-party dependencies
Ensure all third-party libraries and dependencies support ARM architecture. Some incompatible dependencies may need to be updated or replaced, including, but not limited to, the following.
protoc-gen-grpc-java
: see #4248
io.grpc:protoc-gen-grpc-java
from 1.9.0 to 1.52.1 for ARM64 architecture by #5254[!Warning] This is the last planned update of JDK 17 under the NFTC. Updates after September 2024 will be licensed under the Java SE OTN License (OTN) and production use beyond the limited free grants of the OTN license will require a fee.
To avoid subsequent charges for commercial use, I recommend switching to OpenJDK.
[!Caution] Strong data consistency and finality Final data consistency is required for blockchain, and it's usually guaranteed by the world state. Unfortunately, Java-Tron doesn't have a world state. We need to think about how to ensure final data consistency.
1. JDK version compatibility Maybe try to support OpenJDK on ARM?
Background
Java-Tron currently only supports the x86 architecture. Nevertheless, ARM architecture has gained significant traction recently, especially in cloud computing and mobile devices. ARM processors are known for their energy efficiency and cost-effectiveness, making them increasingly popular in data centers, cloud computing, and edge computing scenarios. It will be great to have an option to run Java-Tron using ARM architecture.
Key developments in ARM architecture:
ARM advantages:
Related Issues and PRs
5845
5815
4692
4592
4548
4534
4525
4248
4132
3834
363
Scope of Impact