Using Julia binary artifacts system for Random123?

[oschulz]

Are there plans to use the Julia binary artifacts system for Random123 (via BinaryBuilder.jl)?

I tried using Random123 within the official "julia:rc-alpine" container, but since it has no GCC, installation fails. Using artifacts would make Random123 available on all platforms supported by Julia without user-side compilation - I realize it would likely require some work to set it up, though.

[oschulz]

@giordano, you're the expert here, how complex would it be to make a Random123.jll?

[giordano]

Is it open source?

[dpsanders]

I thought this repo was already a re-implementation in Julia?

[oschulz]

I thought this repo was already a re-implementation in Julia?

I'm not sure - it definitely builds a binary dependency (old-style).

[oschulz]

I rely on Random123.jl pretty heavily in BAT.jl for reproducible random number generation in multithreaded/distributed code, but I don't know the internals of Random123 very well - the code that I have looked into so far was pure Julia, but the package won't install without building it's deps.

[dpsanders]

Oops, you're right, sorry.

[dpsanders]

The license is here: https://www.deshawresearch.com/downloads/download_random123.cgi/Random123_License.txt

Looks like MIT?

[oschulz]

At least it's definitely open source without copyleft, looks like.

[giordano]

But the code which is compiled is in https://github.com/sunoru/Random123.jl/tree/234208c4cbbc1fb961dcd1087fd86bfe5b282684/deps and it isn't very clear to me where it comes from.

[oschulz]

@sunoru , can you help us here?

[oschulz]

Or maybe the pure-Julia RNGs in here could be spun off as a separate package?

[aviks]

This package also does some build time checks for CPU features, so it may not be trivial to offload this into a jll -- will probably need some refactoring

https://github.com/sunoru/Random123.jl/blob/234208c4cbbc1fb961dcd1087fd86bfe5b282684/deps/build.jl#L28-L33

[ViralBShah]

Would be nice to have the JLL here. Anyone working on it?

[sunoru]

Would be nice to have the JLL here. Anyone working on it?

Hi, I replied in another issue under this repo that actually these binary dependencies are not necessary, and I have been working on a pure-Julia way (that directly uses LLVM intrinsics) to replace them.

[sunoru]

Done in v1.4+

[ViralBShah]

Can we remove the deps folder?

[sunoru]

Can we remove the deps folder?

The deps folder is used to determine if AES-NI is supported when the package is first built. If we remove it, the AESNI-related RNGs would make a process crash.

Or is there a better way to do this?

[ViralBShah]

Oh - is this something that is determined on the system it is run? How about in __init__()?

[sunoru]

Yes, it is determined by the platform. deps/build.jl will attempt to run deps/has_aesni.jl in a new process and see if it succeeded. If we use __init__(), something similar would be done every time the package is loaded?

[ViralBShah]

Yes, that would do it every time the package is loaded. I suppose deps/build.jl will only do it upon install. I suppose it takes a fraction of a second to do this - and doing it on package load may be ok.

This is a corner case - but in instances where people copy their package files to a different system or use on a shared filesystem (and across different computers), doing it on package load will be more robust.

[sunoru]

That makes sense. I'll change it.