Platform support branch.

[kayabaNerve]

Tracking issue for https://github.com/MerosCrypto/Meros/tree/platform-support.

Ubuntu 18.04 and 20.04 should work without issue. Arch Linux should work without issue.

• [ ] Windows isn't supported.
• [ ] Ubuntu 14.04 fails to compile mc_minisketch.

macOS has several issues.

• [ ] M1 machines don't support choosenim.
• [x] M1 machines don't support JIT RandomX. Meros will segfault when attempting this. https://github.com/MerosCrypto/mc_randomx should disable the JIT flag, which is automatically suggested, for us. We should also add checks to it for null pointers being returned and throw proper exceptions.
• [x] -z muldefs is used due to our multiple Rust libraries. This flag doesn't exist on macOS and isn't needed.
• [x] https://github.com/MerosCrypto/mc_wry links with -framework WebKit. CoreFoundation, CoreGraphics, CoreVideo, ColorSync, and AppKit are also needed frameworks.
• [x] The tests expect a .so file extension when not on Windows; macOS uses .dylib. I'm curious which BSD uses as well.
• [x] Pylint can't find the MultilineExpansionChecker. I have no idea why a scripted environment has difficulty here yet it does.
• [x] Python's Milagro needs to manually import libamcl_core.
• [x] argon2 Python package doesn't work due to _ffi not being available.
• [x] The WalletDB test fails due to an LMDB error; booting Meros causes 26 (invalid data!) to be printed which is another LMDB error.
• [x] Meros consistently prints 26 (invalid data!) on macOS.
• [x] ULimit code does not work, instead always returning 10/11 open FDs.
• [x] Pylint has several false positives for some reason.

And there may be more I have yet to find. I also needed to set a manual CPATH/LIBRARY_PATH so gmpy2 would build. I installed the dependencies with brew install gmp mpfr libmpc and it worked without issue, yet after rebooting, and deciding to rebuild on Python 3.8 instead of 3.9 as an experiment, I needed to set a CPATH for gmp and LIBRARY_PATH for all three...

Windows hasn't been supported in ages. I have no idea if we'll want to go with msvc or mingw when we add support for it. While it'd be great to say "MS ecosystem! msvc!", I do remember msvc has some quirks, such as not supporting C99, and I believe horrible CMake integration. That said, the last system I tried on had so much development cruft it's hard to actually say. Multiple libraries only have Linux build instructions. We also don't support tracking file handles on Windows, so Meros will fail to build there as well.

[kayabaNerve]

If we use the pr-applem1 branch from RandomX in mc_randomx, we'll have macOS support without issue; the library will also be ~6 months older than it currently is. That said, not only has no notable to us development happened in that time, it'd still be newer than our currently used commit. I'm personally hoping it's simply merged though and we can just update to the latest commit,

[kayabaNerve]

Pylint can't find the MultilineExpansionChecker. I have no idea why a scripted environment has difficulty here yet it does.

macOS and Windows use case insensitive file systems. The folder is named Pylint. It's trying to import from pylint proper instead of our custom folder.

[kayabaNerve]

For what it's worth, mc_lmdb expects msvc.

[kayabaNerve]

https://github.com/MerosCrypto/Meros/commit/8bf28f12c88503b68db813cd818b9e7f94e4cac8 fixed the WalletDB test failing. I'm hoping to get the other part of that checklist item via updating LMDB. Running every test on it now.

[kayabaNerve]

LMDB issues are all sorted.

[kayabaNerve]

Notably, running the Consensus test still prints 26 (invalid data!). On a second look, this doesn't appear to be from lmdb. It's printing next to RandomX debug statements I added, making that my next guess, yet I'm honestly not sure.

EDIT: It's Nim's system module. Apparently, a set is corrupted? I'm really not sure.

[kayabaNerve]

The issue was the RandomXFlags enum. Updating it now.

[kayabaNerve]

Updated mc_randomx, as well as the version of RandomX used, to solve all issues there. As far as macOS is concerned, it's solely the Python issues standing (and lack of choosenim which will not be considered a blocker).

[kayabaNerve]

Updating the argon2-cffi dependency fixed that issue. Possible new issue of pylint throwing plenty of no-member and c-extension-no-member errors.

[kayabaNerve]

ULimit didn't work properly, finding files 0 .. 9 and 10 if it exists, yet no others. Replaced it with a horrible shell execution which does work.

[kayabaNerve]

Except for choosenim on m1 Macs, macOS should work without issue now. I'd like to merge the platform support branch on that basis, yet keep it around pending further development on Windows. While I would like Ubuntu 14.04 support, as it's still a supported LTS branch, I understand it's 7 years old and not something we may want to officially support. I'm also just tired of tedious tasks, hence why I want to revisit Windows at a later date.