Support big-endian hardware

[bestbeforetoday]

Big-endian hardware is currently explicitly unsupported:

https://github.com/paulmillr/noble-hashes/blob/9f9c435db889fbc905ffbaa8f57c085a4ccb83cf/src/utils.ts#L28-L31

As an indirect consumer of this package (via noble-curves), it would be great not have to worry whether consumers of my code want to run on big-endian hardware.

[paulmillr]

1. What specific big-endian hardware you want your consumers to be able to run this on?
2. The issue is not limited to noble. Any JS hashing library doesn't support BE hardware. The difference is that they don't even test for it.
3. It would probably be complicated to support. For example, I don't have BE hardware that can run node.js.

[bestbeforetoday]

The specific hardware I've had user feedback on is s390x.

The issue I have as a consuming package is that I am not making any direct use of @noble/hashes. I am only using @noble/curves, and only using that to create signatures of pre-computed digests with P-256 (and P-384) ECDSA private keys. Apparently just loading (Node.js require) the @noble/curves/p256 (and p384) modules causes the error about big-endian support to be thrown from @noble/hashes.

If there is any way to restructure modules (or where the error is generated) to avoid the error on load, and just allow signatures to be created with @noble/curves, that would be fine for my use-case.

[paulmillr]

Of course, because noble/curves/p256 uses noble-hashes for its hashing functionality. So, we early-error there.

I'm not even sure noble-curves would work correctly on BE arch -- with, or without noble-hashes.

Can this person comment-out the "error" throwing test and run all noble-curves and noble-hashes tests? I'm sure something will fail.

[paulmillr]

Just a few ideas:

• All constants would likely need to be changed
• Casts from u8 to u32 would need to be changed and can become much slower

If someone would donate hardware, fund the work, or do a pull request (that won't massively slow-down LE case) themselves, it would make sense to support it.

[bestbeforetoday]

Unfortunately I don't think they are familiar enough with Java/TypeScript to be able to offer any help - just running sample applications that make use of my packages on s390x - and I don't have access to any s390x hardware.

They do report that back-levelling my packages to a version before I switched to @noble/curves, things do run successfully. Previously I was using indutny/elliptic, which in turn looks to use indutny/hash.js.

Since I'm targeting Node.js, I wonder if another approach might be to create a short Weierstrass curve supplying a hash implementation based on Node's crypto implementation? This seems kinda fiddly to do with the abstract API. If this is a good approach, would it make sense to provide a convenience API method to allow users to create a curve using your curve definitions but with a custom hash function?

[paulmillr]

There is no guarantee packages (hashes, curves, indutny-hash) produce correct outputs at all. So, it's better to early-throw instead of providing false sense of compatibility. I also don't see how using curves with nodejs builtins would be better.

This seems kinda fiddly to do with the abstract API

Nothing fiddly, it's easy and takes a few lines of code. README has all the info.

[bestbeforetoday]

There is no guarantee packages (hashes, curves, indutny-hash) produce correct outputs at all. So, it's better to early-throw instead of providing false sense of compatibility. I also don't see how using curves with nodejs builtins would be better.

Can you explain what you mean by this? The outputs of the indutny ECDSA signing and Node.js hash implementations on big-endian systems both seems to interoperate correctly with signature verification implementations in Golang on big-endian and little-endian systems. Their behaviour seems to match that of equivalent signature generation and hash implementations in Golang and Java, both of which also interoperate (on both big-endian and little-endian systems) with the same Golang signature verification implementation. It seems unlikely to me that they all suffer from identical and compatible broken implementations.

Nothing fiddly, it's easy and takes a few lines of code. README has all the info.

The README says the Hash should take this form:

type CHash = {
  (message: Uint8Array): Uint8Array;
  blockLen: number;
  outputLen: number;
  create(): any;
};

What is the create() and what should it return?

[paulmillr]

It seems unlikely to me that they all suffer from identical and compatible broken implementations.

Run wycheproof vectors against indutny and noble hashes on BE and you will now. We are not guessing here - we need a real proof.

What is the create() and what should it return?

const sha256 = function(message) {return actualHash(message)}
sha256.create = sha256
sha256.blockLen = 32
sha256.outputLen = 32

[rposts]

@paulmillr IBM LinuxONE community provide access to big-endian hardware via this portal. Could this be a starting point towards helping you enable support on big-endian archs?

Thanks again for your consideration.

[paulmillr]

It could, however, there are time constraints.

Pull requests are welcome.

[jonathan-albrecht-ibm]

Hi @paulmillr I've been looking at this issue with @rposts. I've opened https://github.com/paulmillr/noble-hashes/pull/81 for adding support for big-endian platforms. Looking forward to your feedback when you get a chance and happy to make any changes necessary.

[bestbeforetoday]

@paulmillr What is your expected timeline for a new release of noble-hashes including this enhancement, and then a release of noble-curves that depends on the updated noble-hashes?

[paulmillr]

@bestbeforetoday I aim for quarterly cadence for the sake of simplified auditability for users. Last one was in December, so perhaps March or April. For now, it is possible to build a file by cloning a repository, and using build directory.

@jonathan-albrecht-ibm if you are willing to lend us a help with noble-ciphers, which also need BE support, we would be grateful!

[jonathan-albrecht-ibm]

@paulmillr, sorry, I'd love to work on noble-ciphers but I won't have time for the foreseeable future. I'm happy to answer any questions on the noble-hashes changes if someone else has time to pick it up.