Open recanman opened 4 months ago
An extremely important task is to have tests for these cryptographic operations (to make sure results are consistent of course). If anyone seeing this is willing to give an expected input/output for each function in Cryptonote.php while I'm refactoring, that would be really great (if it isn't done, I'll do it myself. Just in case anyone else wants to speed up the migration process).
Rewriting it with sodium does not seem feasible at the moment as certain functions are not too well-documented. I've decided to add types, tests, and clean up the code for ed25519
and Cryptonote
.
The scalarmult
function in the ed25519
class is recursive and incredibly slow (15+ seconds). I'll fix that as well.
Here's my progress on Cryptonote.php
:
keccak_256
gen_new_hex_seed
sc_reduce
hash_to_scalar
derive_viewKey
gen_private_keys
pk_from_sk
gen_key_derivation
derivation_to_scalar
stealth_payment_id
txpub_from_extra
derive_public_key
is_output_mine
encode_address
verify_checksum
decode_address
integrated_addr_from_keys
address_from_seed
generate_subaddr_secret_key
generate_subaddress_spend_public_key
generate_subaddr_view_public_key
generate_subaddress
deserialize_block_header
Just added the publishing workflow. I'm about 80% done with the rest of the refactoring, but yet to polish it up and push it here.
@serhack, could you create a new repository under this organization? My name suggestion would be monerophp-rpc
.
Ed25519 (public methods in reference implementation:
H
expmod
inv
xrecover
edwards
scalarmult
encodeint
encodepoint
bit
Hint
publickey
signature
isoncurve
decodeint
decodepoint
checkvalid
@serhack, could you create a new repository under this organization? My name suggestion would be monerophp-rpc.
Sorry, I've missed a point, why shouldn't there be only one repo with all PHP related library?
I don't think Composer can manage a monorepo with multiple packages. Two packages: one for the cryptography operations, and one for the wallet/daemon RPC client
@serhack, any updates?
Tomorrow I'll create the second repository :)
@recanman I created the new repository
@recanman I'd like to help completing any remaining tasks, performing tests, adding unit tests, etc. ... I'll be reviewing this as it progresses and the other repo as well, if there's any areas still needing work please feel free to direct me towards it.
I really appreciate your help. I've been a bit busy.
I think that helping to implement the remaining functions would be great. I have struggled with signature
and checkvalid
.
Thanks to both of you.
I think that helping to implement the remaining functions would be great. I have struggled with
signature
andcheckvalid
.
@recanman I'm working on this. I'm syncing testnet and getting set up to run ... basing off of your recanman/monerophp/tree/refactor
phpunit
throws
1) BCMATH_BigIntegerTest::testCreate
Error: Class 'MoneroIntegrations\MoneroCrypto\BigInteger' not found
and so on due to the classes not being found / imported properly. I'm just running phpunit
in the root monerophp
folder, do I need any args or what else do I need to do? This is on a fresh Ubuntu 20.04 with XAMPP and PHP on the commandline and PHPUnit installed according to those linked docs
Later I can update the documentation. For now I'm going to poke around and any tips would be appreciated
Also, can you recommend an IDE? I usually use JetBrains IDEs but meh
Just documenting as I go for reference later... I installed Composer, ran composer update
, and composer install
in monerophp
. At this point I realized I had PHP 7.4 on the commandline (XAMPP installed 8.4 but only for itself), so I updated PHP to 8.3.10. Then I had to install PHP's bcmath extension as in sudo apt install php8.3-bcmath
sudo apt install php8.3-curl
speeds composer up
but I still get
PHP Fatal error: Cannot redeclare MoneroIntegrations\MoneroCrypto\Cryptonote::$network_prefixes in /home/user/src/monerophp/src/Cryptonote.php on line 61
which I'll work on massaging... Are you using a globally-installed PHPUnit or are you installing PHPUnit with Composer? And I see you're on Windows by your file format, I'll have to set my git to use Windows-style line endings... I also have a Windows dev box which I can work on for convenience sake. We should document the setup required for both Windows and Linux anyways. I also have a fresh macOS :D
Let me check my local machine, sorry about that.
After running git stash
I get the same error, fix is coming up.
By the way, try composer run test:unit
instead of composer run test
. That will run unit tests directly as there as some lint errors.
Ahh I was just running phpunit
. composer run test:unit
has much the same result tho even after this most recent commit
I have to take a break for awhile, if there are no more pushes by that time I'll start tinkering on what's here, thanks for this, love the approach to reimplement crypto fundamentals in PHP. I'll also contribute over in the rpc-centric repo. Cheers
I have pushed all of my local changes.
If you're able to figure out derive_viewKey
, signature
and checkvalid
, that would be great.
I'll post some resources related to how these functions are calculated and hopefully you'll have some luck.
Most of the remaining Cryptonote functions are dependent on derive_viewKey
.
(CI now passes! :smile:)
https://getmonero.dev/cryptography/asymmetric/private-key.html https://getmonero.dev/cryptography/asymmetric/public-key.html
I am rewriting the README to get it back up-to-date, and also to reduce confusion on how to run tests.
OK, just confirming that I got the tests working basically:
$ php -v
PHP 8.3.10 (cli) (built: Aug 2 2024 15:30:49) (NTS)
...
$ php -m
[PHP Modules]
bcmath
...
gmp
...
$ git clone https://github.com/recanman/monerophp.git
Cloning into 'monerophp'...
..
$ cd monerophp/
$ git checkout recanman/refactor
$ composer install
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Package operations: 17 installs, 0 updates, 0 removals
...
$ $ composer test:unit
> phpunit --testsuite unit
PHPUnit 10.5.29 by Sebastian Bergmann and contributors.
Runtime: PHP 8.3.10
Configuration: /home/user/src/monerophptesting/monerophp/phpunit.xml
.....W.....................WPHP Warning: Constant S_MATH_BIGINTEGER_MODE already defined in /home/user/src/monerophptesting/monerophp/tests/unit/GMP_BigIntegerTest.php on line 13
PHP Stack trace:
PHP 1. {main}() /home/user/src/monerophptesting/monerophp/vendor/bin/phpunit:0
PHP 2. include() /home/user/src/monerophptesting/monerophp/vendor/bin/phpunit:122
PHP 3. PHPUnit\TextUI\Application->run($argv = [0 => '/home/user/src/monerophptesting/monerophp/vendor/bin/phpunit', 1 => '--testsuite', 2 => 'unit']) /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/phpunit:104
PHP 4. PHPUnit\TextUI\TestRunner->run($configuration = class PHPUnit\TextUI\Configuration\Configuration { private readonly array $cliArguments = []; private readonly ?string $configurationFile = '/home/user/src/monerophptesting/monerophp/phpunit.xml'; private readonly ?string $bootstrap = NULL; private readonly bool $cacheResult = TRUE; private readonly ?string $cacheDirectory = '/home/user/src/monerophptesting/monerophp/.phpunit.cache'; private readonly ?string $coverageCacheDirectory = '/home/user/src/monerophptesting/monerophp/.phpunit.cache/code-coverage'; private readonly PHPUnit\TextUI\Configuration\Source $source = class PHPUnit\TextUI\Configuration\Source { private readonly ?string $baseline = NULL; private readonly bool $ignoreBaseline = FALSE; private readonly PHPUnit\TextUI\Configuration\FilterDirectoryCollection $includeDirectories = class PHPUnit\TextUI\Configuration\FilterDirectoryCollection { ... }; private readonly PHPUnit\TextUI\Configuration\FileCollection $includeFiles = class PHPUnit\TextUI\Configuration\FileCollection { ... }; private readonly PHPUnit\TextUI\Configuration\FilterDirectoryCollection $excludeDirectories = class PHPUnit\TextUI\Configuration\FilterDirectoryCollection { ... }; private readonly PHPUnit\TextUI\Configuration\FileCollection $excludeFiles = class PHPUnit\TextUI\Configuration\FileCollection { ... }; private readonly bool $restrictDeprecations = FALSE; private readonly bool $restrictNotices = FALSE; private readonly bool $restrictWarnings = FALSE; private readonly bool $ignoreSuppressionOfDeprecations = FALSE; private readonly bool $ignoreSuppressionOfPhpDeprecations = FALSE; private readonly bool $ignoreSuppressionOfErrors = FALSE; private readonly bool $ignoreSuppressionOfNotices = FALSE; private readonly bool $ignoreSuppressionOfPhpNotices = FALSE; private readonly bool $ignoreSuppressionOfWarnings = FALSE; private readonly bool $ignoreSuppressionOfPhpWarnings = FALSE }; private readonly bool $pathCoverage = FALSE; private readonly ?string $coverageClover = NULL; private readonly ?string $coverageCobertura = NULL; private readonly ?string $coverageCrap4j = NULL; private readonly int $coverageCrap4jThreshold = 30; private readonly ?string $coverageHtml = NULL; private readonly int $coverageHtmlLowUpperBound = 50; private readonly int $coverageHtmlHighLowerBound = 90; private readonly string $coverageHtmlColorSuccessLow = '#dff0d8'; private readonly string $coverageHtmlColorSuccessMedium = '#c3e3b5'; private readonly string $coverageHtmlColorSuccessHigh = '#99cb84'; private readonly string $coverageHtmlColorWarning = '#fcf8e3'; private readonly string $coverageHtmlColorDanger = '#f2dede'; private readonly ?string $coverageHtmlCustomCssFile = NULL; private readonly ?string $coveragePhp = NULL; private readonly ?string $coverageText = NULL; private readonly bool $coverageTextShowUncoveredFiles = FALSE; private readonly bool $coverageTextShowOnlySummary = FALSE; private readonly ?string $coverageXml = NULL; private readonly string $testResultCacheFile = '/home/user/src/monerophptesting/monerophp/.phpunit.cache/test-results'; private readonly bool $ignoreDeprecatedCodeUnitsFromCodeCoverage = FALSE; private readonly bool $disableCodeCoverageIgnore = FALSE; private readonly bool $failOnDeprecation = TRUE; private readonly bool $failOnEmptyTestSuite = FALSE; private readonly bool $failOnIncomplete = FALSE; private readonly bool $failOnNotice = FALSE; private readonly bool $failOnRisky = FALSE; private readonly bool $failOnSkipped = FALSE; private readonly bool $failOnWarning = FALSE; private readonly bool $stopOnDefect = FALSE; private readonly bool $stopOnDeprecation = FALSE; private readonly bool $stopOnError = FALSE; private readonly bool $stopOnFailure = FALSE; private readonly bool $stopOnIncomplete = FALSE; private readonly bool $stopOnNotice = FALSE; private readonly bool $stopOnRisky = FALSE; private readonly bool $stopOnSkipped = FALSE; private readonly bool $stopOnWarning = FALSE; private readonly bool $outputToStandardErrorStream = FALSE; private readonly int $columns = 80; private readonly bool $noExtensions = FALSE; private readonly ?string $pharExtensionDirectory = NULL; private readonly array $extensionBootstrappers = []; private readonly bool $backupGlobals = FALSE; private readonly bool $backupStaticProperties = FALSE; private readonly bool $beStrictAboutChangesToGlobalState = FALSE; private readonly bool $colors = TRUE; private readonly bool $processIsolation = FALSE; private readonly bool $enforceTimeLimit = FALSE; private readonly int $defaultTimeLimit = 1; private readonly int $timeoutForSmallTests = 1; private readonly int $timeoutForMediumTests = 10; private readonly int $timeoutForLargeTests = 60; private readonly bool $reportUselessTests = TRUE; private readonly bool $strictCoverage = FALSE; private readonly bool $disallowTestOutput = FALSE; private readonly bool $displayDetailsOnIncompleteTests = FALSE; private readonly bool $displayDetailsOnSkippedTests = FALSE; private readonly bool $displayDetailsOnTestsThatTriggerDeprecations = TRUE; private readonly bool $displayDetailsOnTestsThatTriggerErrors = FALSE; private readonly bool $displayDetailsOnTestsThatTriggerNotices = FALSE; private readonly bool $displayDetailsOnTestsThatTriggerWarnings = FALSE; private readonly bool $reverseDefectList = FALSE; private readonly bool $requireCoverageMetadata = FALSE; private readonly bool $registerMockObjectsFromTestArgumentsRecursively = FALSE; private readonly bool $noProgress = FALSE; private readonly bool $noResults = FALSE; private readonly bool $noOutput = FALSE; private readonly int $executionOrder = 0; private readonly int $executionOrderDefects = 0; private readonly bool $resolveDependencies = TRUE; private readonly ?string $logfileTeamcity = NULL; private readonly ?string $logfileJunit = NULL; private readonly ?string $logfileTestdoxHtml = NULL; private readonly ?string $logfileTestdoxText = NULL; private readonly ?string $logEventsText = NULL; private readonly ?string $logEventsVerboseText = NULL; private readonly ?array $testsCovering = NULL; private readonly ?array $testsUsing = NULL; private readonly bool $teamCityOutput = FALSE; private readonly bool $testDoxOutput = FALSE; private readonly ?string $filter = NULL; private readonly ?array $groups = []; private readonly ?array $excludeGroups = []; private readonly int $randomOrderSeed = 1723586420; private readonly bool $includeUncoveredFiles = TRUE; private readonly PHPUnit\TextUI\Configuration\TestSuiteCollection $testSuite = class PHPUnit\TextUI\Configuration\TestSuiteCollection { private readonly array $testSuites = [...] }; private readonly string $includeTestSuite = 'unit'; private readonly string $excludeTestSuite = ''; private readonly ?string $defaultTestSuite = NULL; private readonly array $testSuffixes = [0 => 'Test.php', 1 => '.phpt']; private readonly PHPUnit\TextUI\Configuration\Php $php = class PHPUnit\TextUI\Configuration\Php { private readonly PHPUnit\TextUI\Configuration\DirectoryCollection $includePaths = class PHPUnit\TextUI\Configuration\DirectoryCollection { ... }; private readonly PHPUnit\TextUI\Configuration\IniSettingCollection $iniSettings = class PHPUnit\TextUI\Configuration\IniSettingCollection { ... }; private readonly PHPUnit\TextUI\Configuration\ConstantCollection $constants = class PHPUnit\TextUI\Configuration\ConstantCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $globalVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $envVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $postVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $getVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $cookieVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $serverVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $filesVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $requestVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... } }; private readonly bool $controlGarbageCollector = FALSE; private readonly int $numberOfTestsBeforeGarbageCollection = 100; private readonly ?string $generateBaseline = NULL; private readonly bool $debug = FALSE }, $resultCache = class PHPUnit\Runner\ResultCache\DefaultResultCache { private readonly string $cacheFilename = '/home/user/src/monerophptesting/monerophp/.phpunit.cache/test-results'; private array $defects = []; private array $times = ['BCMATH_BigIntegerTest::testCreate' => 0.002, 'BCMATH_BigIntegerTest::testCreateSafe' => 0, 'BCMATH_BigIntegerTest::testSpaces' => 0, 'BCMATH_BigIntegerTest::testOp' => 0.001, 'BCMATH_BigIntegerTest::testBig' => 0.002, 'Base58Test::testEncode' => 0.001, 'Base58Test::testDecode' => 0.001, 'CryptonoteTest::testKeccak256' => 0.001, 'CryptonoteTest::testGenNewHexSeed' => 0, 'CryptonoteTest::testScReduce' => 0, 'CryptonoteTest::testEncodeAddress' => 0.002, 'CryptonoteTest::testVerifyChecksum' => 0.004, 'CryptonoteTest::testDecodeAddress' => 0.003, 'CryptonoteTest::testIntegratedAddrFromKeys' => 0.002, 'Ed25519Test::testPoint' => 0.001, 'Ed25519Test::testH' => 0, 'Ed25519Test::testExpMod' => 0, 'Ed25519Test::testInv' => 0, 'Ed25519Test::testXrecover' => 0.017, 'Ed25519Test::testEdwardsAndEncodePoint' => 0.001, 'Ed25519Test::testScalarMult' => 0.002, 'Ed25519Test::testPublicKey' => 0.873, 'Ed25519Test::testEncodeInt' => 0, 'Ed25519Test::testBit' => 0, 'Ed25519Test::testHint' => 0, 'Ed25519Test::testIsOnCurve' => 0.001, 'Ed25519Test::testDecodeInt' => 0, 'Ed25519Test::testDecodePoint' => 0.017] }, $suite = class PHPUnit\Framework\TestSuite { private string $name = '/home/user/src/monerophptesting/monerophp/phpunit.xml'; private array $groups = []; private ?array $requiredTests = NULL; private array $tests = []; private ?array $providedTests = NULL; private ?PHPUnit\Runner\Filter\Factory $iteratorFilter = NULL; private bool $wasRun = TRUE }) /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/TextUI/Application.php:202
PHP 5. PHPUnit\Framework\TestSuite->run() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:64
PHP 6. PHPUnit\Framework\TestSuite->run() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:380
PHP 7. PHPUnit\Framework\TestSuite->run() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:380
PHP 8. PHPUnit\Framework\TestSuite->invokeMethodsBeforeFirstTest($emitter = class PHPUnit\Event\DispatchingEmitter { private readonly PHPUnit\Event\Dispatcher $dispatcher = class PHPUnit\Event\DeferringDispatcher { private readonly PHPUnit\Event\SubscribableDispatcher $dispatcher = class PHPUnit\Event\DirectDispatcher { ... }; private PHPUnit\Event\EventCollection $events = class PHPUnit\Event\EventCollection { ... }; private bool $recording = FALSE }; private readonly PHPUnit\Event\Telemetry\System $system = class PHPUnit\Event\Telemetry\System { private readonly PHPUnit\Event\Telemetry\StopWatch $stopWatch = class PHPUnit\Event\Telemetry\SystemStopWatch { ... }; private readonly PHPUnit\Event\Telemetry\MemoryMeter $memoryMeter = class PHPUnit\Event\Telemetry\SystemMemoryMeter { ... }; private readonly PHPUnit\Event\Telemetry\GarbageCollectorStatusProvider $garbageCollectorStatusProvider = class PHPUnit\Event\Telemetry\Php83GarbageCollectorStatusProvider { ... } }; private readonly PHPUnit\Event\Telemetry\Snapshot $startSnapshot = class PHPUnit\Event\Telemetry\Snapshot { private readonly PHPUnit\Event\Telemetry\HRTime $time = class PHPUnit\Event\Telemetry\HRTime { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $memoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $peakMemoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\GarbageCollectorStatus $garbageCollectorStatus = class PHPUnit\Event\Telemetry\GarbageCollectorStatus { ... } }; private PHPUnit\Event\Telemetry\Snapshot $previousSnapshot = class PHPUnit\Event\Telemetry\Snapshot { private readonly PHPUnit\Event\Telemetry\HRTime $time = class PHPUnit\Event\Telemetry\HRTime { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $memoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $peakMemoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\GarbageCollectorStatus $garbageCollectorStatus = class PHPUnit\Event\Telemetry\GarbageCollectorStatus { ... } }; private bool $exportObjects = FALSE }, $testSuiteValueObjectForEvents = class PHPUnit\Event\TestSuite\TestSuiteForTestClass { private readonly string ${PHPUnit\Event\TestSuite\TestSuite}name = 'GMP_BigIntegerTest'; private readonly int ${PHPUnit\Event\TestSuite\TestSuite}count = 5; private readonly PHPUnit\Event\Code\TestCollection ${PHPUnit\Event\TestSuite\TestSuite}tests = class PHPUnit\Event\Code\TestCollection { private readonly array $tests = [...] }; private readonly string $className = 'GMP_BigIntegerTest'; private readonly string $file = '/home/user/src/monerophptesting/monerophp/tests/unit/GMP_BigIntegerTest.php'; private readonly int $line = 8 }) /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:357
PHP 9. GMP_BigIntegerTest::setUpBeforeClass() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:643
PHP 10. define($constant_name = 'S_MATH_BIGINTEGER_MODE', $value = 'gmp') /home/user/src/monerophptesting/monerophp/tests/unit/GMP_BigIntegerTest.php:13
........EEE....... 46 / 46 (100%)
Time: 00:00.962, Memory: 12.00 MB
There were 3 errors:
1) MnemonicTest::testEncode
Error: Call to undefined function MoneroIntegrations\MoneroCrypto\gmp_init()
/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:100
/home/user/src/monerophptesting/monerophp/tests/unit/MnemonicTest.php:35
2) MnemonicTest::testEncodeWithChecksum
Error: Call to undefined function MoneroIntegrations\MoneroCrypto\gmp_init()
/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:100
/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:120
/home/user/src/monerophptesting/monerophp/tests/unit/MnemonicTest.php:40
3) MnemonicTest::testDecode
Error: Call to undefined function MoneroIntegrations\MoneroCrypto\gmp_add()
/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:168
/home/user/src/monerophptesting/monerophp/tests/unit/MnemonicTest.php:45
ERRORS!
Tests: 46, Assertions: 228, Errors: 3, Warnings: 2.
Script phpunit --testsuite unit handling the test:unit event returned with error code 2
so I've begun working and will be back with updates as they come.
note that this doesn't imply any review and these sorts of changes will require thorough review
This pull request contains the final changes needed for the package migration. Package name will be changed to
monero-integrations/monero-crypto
(namespaceMoneroIntegrations\MoneroCrypto
)This will also bump the minimum PHP version to 8.1. A fork of refring's work. This will be called
monero-integrations/monero-rpc
(namespaceMoneroIntegrations\MoneroRpc
).Tasks left:
MoneroRpc
packageMost of my work is usually offline/on my local git server and I may not push to this PR's branch frequently.
I've spent some time with refring's code and tested it (it is very good). Merging it is just a matter of creating a new repository (I will CC serhack at the appropriate time)