monero-integrations / monerophp

Monero PHP library + JsonRPC Client
MIT License
116 stars 76 forks source link

draft: Final touches (closes #147, #135, #133, #132, #124, #111, #100, #92, #82, #81, #57, #54) #154

Open recanman opened 4 months ago

recanman commented 4 months ago

This pull request contains the final changes needed for the package migration. Package name will be changed to monero-integrations/monero-crypto (namespace MoneroIntegrations\MoneroCrypto)

This will also bump the minimum PHP version to 8.1. A fork of refring's work. This will be called monero-integrations/monero-rpc (namespace MoneroIntegrations\MoneroRpc).

Tasks left:

Most of my work is usually offline/on my local git server and I may not push to this PR's branch frequently.

I've spent some time with refring's code and tested it (it is very good). Merging it is just a matter of creating a new repository (I will CC serhack at the appropriate time)

recanman commented 3 months ago

An extremely important task is to have tests for these cryptographic operations (to make sure results are consistent of course). If anyone seeing this is willing to give an expected input/output for each function in Cryptonote.php while I'm refactoring, that would be really great (if it isn't done, I'll do it myself. Just in case anyone else wants to speed up the migration process).

recanman commented 3 months ago

Rewriting it with sodium does not seem feasible at the moment as certain functions are not too well-documented. I've decided to add types, tests, and clean up the code for ed25519 and Cryptonote. The scalarmult function in the ed25519 class is recursive and incredibly slow (15+ seconds). I'll fix that as well.

Here's my progress on Cryptonote.php:

recanman commented 3 months ago

Just added the publishing workflow. I'm about 80% done with the rest of the refactoring, but yet to polish it up and push it here. @serhack, could you create a new repository under this organization? My name suggestion would be monerophp-rpc.

recanman commented 3 months ago

Ed25519 (public methods in reference implementation:

serhack commented 3 months ago

@serhack, could you create a new repository under this organization? My name suggestion would be monerophp-rpc.

Sorry, I've missed a point, why shouldn't there be only one repo with all PHP related library?

recanman commented 3 months ago

I don't think Composer can manage a monorepo with multiple packages. Two packages: one for the cryptography operations, and one for the wallet/daemon RPC client

recanman commented 3 months ago

@serhack, any updates?

serhack commented 3 months ago

Tomorrow I'll create the second repository :)

serhack commented 3 months ago

@recanman I created the new repository

sneurlax commented 1 month ago

@recanman I'd like to help completing any remaining tasks, performing tests, adding unit tests, etc. ... I'll be reviewing this as it progresses and the other repo as well, if there's any areas still needing work please feel free to direct me towards it.

recanman commented 1 month ago

I really appreciate your help. I've been a bit busy.

recanman commented 1 month ago

I think that helping to implement the remaining functions would be great. I have struggled with signature and checkvalid.

serhack commented 1 month ago

Thanks to both of you.

sneurlax commented 1 month ago

I think that helping to implement the remaining functions would be great. I have struggled with signature and checkvalid.

@recanman I'm working on this. I'm syncing testnet and getting set up to run ... basing off of your recanman/monerophp/tree/refactor

phpunit throws

1) BCMATH_BigIntegerTest::testCreate
Error: Class 'MoneroIntegrations\MoneroCrypto\BigInteger' not found

and so on due to the classes not being found / imported properly. I'm just running phpunit in the root monerophp folder, do I need any args or what else do I need to do? This is on a fresh Ubuntu 20.04 with XAMPP and PHP on the commandline and PHPUnit installed according to those linked docs

Later I can update the documentation. For now I'm going to poke around and any tips would be appreciated

Also, can you recommend an IDE? I usually use JetBrains IDEs but meh

sneurlax commented 1 month ago

Just documenting as I go for reference later... I installed Composer, ran composer update, and composer install in monerophp. At this point I realized I had PHP 7.4 on the commandline (XAMPP installed 8.4 but only for itself), so I updated PHP to 8.3.10. Then I had to install PHP's bcmath extension as in sudo apt install php8.3-bcmath

sudo apt install php8.3-curl speeds composer up

but I still get

PHP Fatal error:  Cannot redeclare MoneroIntegrations\MoneroCrypto\Cryptonote::$network_prefixes in /home/user/src/monerophp/src/Cryptonote.php on line 61

which I'll work on massaging... Are you using a globally-installed PHPUnit or are you installing PHPUnit with Composer? And I see you're on Windows by your file format, I'll have to set my git to use Windows-style line endings... I also have a Windows dev box which I can work on for convenience sake. We should document the setup required for both Windows and Linux anyways. I also have a fresh macOS :D

recanman commented 1 month ago

Let me check my local machine, sorry about that.

recanman commented 1 month ago

@sneurlax I am not on Windows?

I use VSCodium mainly.

recanman commented 1 month ago

After running git stash I get the same error, fix is coming up.

By the way, try composer run test:unit instead of composer run test. That will run unit tests directly as there as some lint errors.

sneurlax commented 1 month ago

Ahh I was just running phpunit. composer run test:unit has much the same result tho even after this most recent commit

I have to take a break for awhile, if there are no more pushes by that time I'll start tinkering on what's here, thanks for this, love the approach to reimplement crypto fundamentals in PHP. I'll also contribute over in the rpc-centric repo. Cheers

recanman commented 1 month ago

I have pushed all of my local changes. If you're able to figure out derive_viewKey, signature and checkvalid, that would be great. I'll post some resources related to how these functions are calculated and hopefully you'll have some luck.

Most of the remaining Cryptonote functions are dependent on derive_viewKey.

(CI now passes! :smile:)

recanman commented 1 month ago

https://getmonero.dev/cryptography/asymmetric/private-key.html https://getmonero.dev/cryptography/asymmetric/public-key.html

I am rewriting the README to get it back up-to-date, and also to reduce confusion on how to run tests.

sneurlax commented 3 weeks ago

OK, just confirming that I got the tests working basically:

$ php -v
PHP 8.3.10 (cli) (built: Aug  2 2024 15:30:49) (NTS)
...
$ php -m
[PHP Modules]
bcmath
...
gmp
...
$ git clone https://github.com/recanman/monerophp.git
Cloning into 'monerophp'...
..
$ cd monerophp/
$ git checkout recanman/refactor
$ composer install
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Package operations: 17 installs, 0 updates, 0 removals
...
$ $ composer test:unit
> phpunit --testsuite unit
PHPUnit 10.5.29 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.10
Configuration: /home/user/src/monerophptesting/monerophp/phpunit.xml

.....W.....................WPHP Warning:  Constant S_MATH_BIGINTEGER_MODE already defined in /home/user/src/monerophptesting/monerophp/tests/unit/GMP_BigIntegerTest.php on line 13
PHP Stack trace:
PHP   1. {main}() /home/user/src/monerophptesting/monerophp/vendor/bin/phpunit:0
PHP   2. include() /home/user/src/monerophptesting/monerophp/vendor/bin/phpunit:122
PHP   3. PHPUnit\TextUI\Application->run($argv = [0 => '/home/user/src/monerophptesting/monerophp/vendor/bin/phpunit', 1 => '--testsuite', 2 => 'unit']) /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/phpunit:104
PHP   4. PHPUnit\TextUI\TestRunner->run($configuration = class PHPUnit\TextUI\Configuration\Configuration { private readonly array $cliArguments = []; private readonly ?string $configurationFile = '/home/user/src/monerophptesting/monerophp/phpunit.xml'; private readonly ?string $bootstrap = NULL; private readonly bool $cacheResult = TRUE; private readonly ?string $cacheDirectory = '/home/user/src/monerophptesting/monerophp/.phpunit.cache'; private readonly ?string $coverageCacheDirectory = '/home/user/src/monerophptesting/monerophp/.phpunit.cache/code-coverage'; private readonly PHPUnit\TextUI\Configuration\Source $source = class PHPUnit\TextUI\Configuration\Source { private readonly ?string $baseline = NULL; private readonly bool $ignoreBaseline = FALSE; private readonly PHPUnit\TextUI\Configuration\FilterDirectoryCollection $includeDirectories = class PHPUnit\TextUI\Configuration\FilterDirectoryCollection { ... }; private readonly PHPUnit\TextUI\Configuration\FileCollection $includeFiles = class PHPUnit\TextUI\Configuration\FileCollection { ... }; private readonly PHPUnit\TextUI\Configuration\FilterDirectoryCollection $excludeDirectories = class PHPUnit\TextUI\Configuration\FilterDirectoryCollection { ... }; private readonly PHPUnit\TextUI\Configuration\FileCollection $excludeFiles = class PHPUnit\TextUI\Configuration\FileCollection { ... }; private readonly bool $restrictDeprecations = FALSE; private readonly bool $restrictNotices = FALSE; private readonly bool $restrictWarnings = FALSE; private readonly bool $ignoreSuppressionOfDeprecations = FALSE; private readonly bool $ignoreSuppressionOfPhpDeprecations = FALSE; private readonly bool $ignoreSuppressionOfErrors = FALSE; private readonly bool $ignoreSuppressionOfNotices = FALSE; private readonly bool $ignoreSuppressionOfPhpNotices = FALSE; private readonly bool $ignoreSuppressionOfWarnings = FALSE; private readonly bool $ignoreSuppressionOfPhpWarnings = FALSE }; private readonly bool $pathCoverage = FALSE; private readonly ?string $coverageClover = NULL; private readonly ?string $coverageCobertura = NULL; private readonly ?string $coverageCrap4j = NULL; private readonly int $coverageCrap4jThreshold = 30; private readonly ?string $coverageHtml = NULL; private readonly int $coverageHtmlLowUpperBound = 50; private readonly int $coverageHtmlHighLowerBound = 90; private readonly string $coverageHtmlColorSuccessLow = '#dff0d8'; private readonly string $coverageHtmlColorSuccessMedium = '#c3e3b5'; private readonly string $coverageHtmlColorSuccessHigh = '#99cb84'; private readonly string $coverageHtmlColorWarning = '#fcf8e3'; private readonly string $coverageHtmlColorDanger = '#f2dede'; private readonly ?string $coverageHtmlCustomCssFile = NULL; private readonly ?string $coveragePhp = NULL; private readonly ?string $coverageText = NULL; private readonly bool $coverageTextShowUncoveredFiles = FALSE; private readonly bool $coverageTextShowOnlySummary = FALSE; private readonly ?string $coverageXml = NULL; private readonly string $testResultCacheFile = '/home/user/src/monerophptesting/monerophp/.phpunit.cache/test-results'; private readonly bool $ignoreDeprecatedCodeUnitsFromCodeCoverage = FALSE; private readonly bool $disableCodeCoverageIgnore = FALSE; private readonly bool $failOnDeprecation = TRUE; private readonly bool $failOnEmptyTestSuite = FALSE; private readonly bool $failOnIncomplete = FALSE; private readonly bool $failOnNotice = FALSE; private readonly bool $failOnRisky = FALSE; private readonly bool $failOnSkipped = FALSE; private readonly bool $failOnWarning = FALSE; private readonly bool $stopOnDefect = FALSE; private readonly bool $stopOnDeprecation = FALSE; private readonly bool $stopOnError = FALSE; private readonly bool $stopOnFailure = FALSE; private readonly bool $stopOnIncomplete = FALSE; private readonly bool $stopOnNotice = FALSE; private readonly bool $stopOnRisky = FALSE; private readonly bool $stopOnSkipped = FALSE; private readonly bool $stopOnWarning = FALSE; private readonly bool $outputToStandardErrorStream = FALSE; private readonly int $columns = 80; private readonly bool $noExtensions = FALSE; private readonly ?string $pharExtensionDirectory = NULL; private readonly array $extensionBootstrappers = []; private readonly bool $backupGlobals = FALSE; private readonly bool $backupStaticProperties = FALSE; private readonly bool $beStrictAboutChangesToGlobalState = FALSE; private readonly bool $colors = TRUE; private readonly bool $processIsolation = FALSE; private readonly bool $enforceTimeLimit = FALSE; private readonly int $defaultTimeLimit = 1; private readonly int $timeoutForSmallTests = 1; private readonly int $timeoutForMediumTests = 10; private readonly int $timeoutForLargeTests = 60; private readonly bool $reportUselessTests = TRUE; private readonly bool $strictCoverage = FALSE; private readonly bool $disallowTestOutput = FALSE; private readonly bool $displayDetailsOnIncompleteTests = FALSE; private readonly bool $displayDetailsOnSkippedTests = FALSE; private readonly bool $displayDetailsOnTestsThatTriggerDeprecations = TRUE; private readonly bool $displayDetailsOnTestsThatTriggerErrors = FALSE; private readonly bool $displayDetailsOnTestsThatTriggerNotices = FALSE; private readonly bool $displayDetailsOnTestsThatTriggerWarnings = FALSE; private readonly bool $reverseDefectList = FALSE; private readonly bool $requireCoverageMetadata = FALSE; private readonly bool $registerMockObjectsFromTestArgumentsRecursively = FALSE; private readonly bool $noProgress = FALSE; private readonly bool $noResults = FALSE; private readonly bool $noOutput = FALSE; private readonly int $executionOrder = 0; private readonly int $executionOrderDefects = 0; private readonly bool $resolveDependencies = TRUE; private readonly ?string $logfileTeamcity = NULL; private readonly ?string $logfileJunit = NULL; private readonly ?string $logfileTestdoxHtml = NULL; private readonly ?string $logfileTestdoxText = NULL; private readonly ?string $logEventsText = NULL; private readonly ?string $logEventsVerboseText = NULL; private readonly ?array $testsCovering = NULL; private readonly ?array $testsUsing = NULL; private readonly bool $teamCityOutput = FALSE; private readonly bool $testDoxOutput = FALSE; private readonly ?string $filter = NULL; private readonly ?array $groups = []; private readonly ?array $excludeGroups = []; private readonly int $randomOrderSeed = 1723586420; private readonly bool $includeUncoveredFiles = TRUE; private readonly PHPUnit\TextUI\Configuration\TestSuiteCollection $testSuite = class PHPUnit\TextUI\Configuration\TestSuiteCollection { private readonly array $testSuites = [...] }; private readonly string $includeTestSuite = 'unit'; private readonly string $excludeTestSuite = ''; private readonly ?string $defaultTestSuite = NULL; private readonly array $testSuffixes = [0 => 'Test.php', 1 => '.phpt']; private readonly PHPUnit\TextUI\Configuration\Php $php = class PHPUnit\TextUI\Configuration\Php { private readonly PHPUnit\TextUI\Configuration\DirectoryCollection $includePaths = class PHPUnit\TextUI\Configuration\DirectoryCollection { ... }; private readonly PHPUnit\TextUI\Configuration\IniSettingCollection $iniSettings = class PHPUnit\TextUI\Configuration\IniSettingCollection { ... }; private readonly PHPUnit\TextUI\Configuration\ConstantCollection $constants = class PHPUnit\TextUI\Configuration\ConstantCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $globalVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $envVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $postVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $getVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $cookieVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $serverVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $filesVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $requestVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... } }; private readonly bool $controlGarbageCollector = FALSE; private readonly int $numberOfTestsBeforeGarbageCollection = 100; private readonly ?string $generateBaseline = NULL; private readonly bool $debug = FALSE }, $resultCache = class PHPUnit\Runner\ResultCache\DefaultResultCache { private readonly string $cacheFilename = '/home/user/src/monerophptesting/monerophp/.phpunit.cache/test-results'; private array $defects = []; private array $times = ['BCMATH_BigIntegerTest::testCreate' => 0.002, 'BCMATH_BigIntegerTest::testCreateSafe' => 0, 'BCMATH_BigIntegerTest::testSpaces' => 0, 'BCMATH_BigIntegerTest::testOp' => 0.001, 'BCMATH_BigIntegerTest::testBig' => 0.002, 'Base58Test::testEncode' => 0.001, 'Base58Test::testDecode' => 0.001, 'CryptonoteTest::testKeccak256' => 0.001, 'CryptonoteTest::testGenNewHexSeed' => 0, 'CryptonoteTest::testScReduce' => 0, 'CryptonoteTest::testEncodeAddress' => 0.002, 'CryptonoteTest::testVerifyChecksum' => 0.004, 'CryptonoteTest::testDecodeAddress' => 0.003, 'CryptonoteTest::testIntegratedAddrFromKeys' => 0.002, 'Ed25519Test::testPoint' => 0.001, 'Ed25519Test::testH' => 0, 'Ed25519Test::testExpMod' => 0, 'Ed25519Test::testInv' => 0, 'Ed25519Test::testXrecover' => 0.017, 'Ed25519Test::testEdwardsAndEncodePoint' => 0.001, 'Ed25519Test::testScalarMult' => 0.002, 'Ed25519Test::testPublicKey' => 0.873, 'Ed25519Test::testEncodeInt' => 0, 'Ed25519Test::testBit' => 0, 'Ed25519Test::testHint' => 0, 'Ed25519Test::testIsOnCurve' => 0.001, 'Ed25519Test::testDecodeInt' => 0, 'Ed25519Test::testDecodePoint' => 0.017] }, $suite = class PHPUnit\Framework\TestSuite { private string $name = '/home/user/src/monerophptesting/monerophp/phpunit.xml'; private array $groups = []; private ?array $requiredTests = NULL; private array $tests = []; private ?array $providedTests = NULL; private ?PHPUnit\Runner\Filter\Factory $iteratorFilter = NULL; private bool $wasRun = TRUE }) /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/TextUI/Application.php:202
PHP   5. PHPUnit\Framework\TestSuite->run() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:64
PHP   6. PHPUnit\Framework\TestSuite->run() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:380
PHP   7. PHPUnit\Framework\TestSuite->run() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:380
PHP   8. PHPUnit\Framework\TestSuite->invokeMethodsBeforeFirstTest($emitter = class PHPUnit\Event\DispatchingEmitter { private readonly PHPUnit\Event\Dispatcher $dispatcher = class PHPUnit\Event\DeferringDispatcher { private readonly PHPUnit\Event\SubscribableDispatcher $dispatcher = class PHPUnit\Event\DirectDispatcher { ... }; private PHPUnit\Event\EventCollection $events = class PHPUnit\Event\EventCollection { ... }; private bool $recording = FALSE }; private readonly PHPUnit\Event\Telemetry\System $system = class PHPUnit\Event\Telemetry\System { private readonly PHPUnit\Event\Telemetry\StopWatch $stopWatch = class PHPUnit\Event\Telemetry\SystemStopWatch { ... }; private readonly PHPUnit\Event\Telemetry\MemoryMeter $memoryMeter = class PHPUnit\Event\Telemetry\SystemMemoryMeter { ... }; private readonly PHPUnit\Event\Telemetry\GarbageCollectorStatusProvider $garbageCollectorStatusProvider = class PHPUnit\Event\Telemetry\Php83GarbageCollectorStatusProvider { ... } }; private readonly PHPUnit\Event\Telemetry\Snapshot $startSnapshot = class PHPUnit\Event\Telemetry\Snapshot { private readonly PHPUnit\Event\Telemetry\HRTime $time = class PHPUnit\Event\Telemetry\HRTime { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $memoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $peakMemoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\GarbageCollectorStatus $garbageCollectorStatus = class PHPUnit\Event\Telemetry\GarbageCollectorStatus { ... } }; private PHPUnit\Event\Telemetry\Snapshot $previousSnapshot = class PHPUnit\Event\Telemetry\Snapshot { private readonly PHPUnit\Event\Telemetry\HRTime $time = class PHPUnit\Event\Telemetry\HRTime { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $memoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $peakMemoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\GarbageCollectorStatus $garbageCollectorStatus = class PHPUnit\Event\Telemetry\GarbageCollectorStatus { ... } }; private bool $exportObjects = FALSE }, $testSuiteValueObjectForEvents = class PHPUnit\Event\TestSuite\TestSuiteForTestClass { private readonly string ${PHPUnit\Event\TestSuite\TestSuite}name = 'GMP_BigIntegerTest'; private readonly int ${PHPUnit\Event\TestSuite\TestSuite}count = 5; private readonly PHPUnit\Event\Code\TestCollection ${PHPUnit\Event\TestSuite\TestSuite}tests = class PHPUnit\Event\Code\TestCollection { private readonly array $tests = [...] }; private readonly string $className = 'GMP_BigIntegerTest'; private readonly string $file = '/home/user/src/monerophptesting/monerophp/tests/unit/GMP_BigIntegerTest.php'; private readonly int $line = 8 }) /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:357
PHP   9. GMP_BigIntegerTest::setUpBeforeClass() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:643
PHP  10. define($constant_name = 'S_MATH_BIGINTEGER_MODE', $value = 'gmp') /home/user/src/monerophptesting/monerophp/tests/unit/GMP_BigIntegerTest.php:13
........EEE.......                    46 / 46 (100%)

Time: 00:00.962, Memory: 12.00 MB

There were 3 errors:

1) MnemonicTest::testEncode
Error: Call to undefined function MoneroIntegrations\MoneroCrypto\gmp_init()

/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:100
/home/user/src/monerophptesting/monerophp/tests/unit/MnemonicTest.php:35

2) MnemonicTest::testEncodeWithChecksum
Error: Call to undefined function MoneroIntegrations\MoneroCrypto\gmp_init()

/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:100
/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:120
/home/user/src/monerophptesting/monerophp/tests/unit/MnemonicTest.php:40

3) MnemonicTest::testDecode
Error: Call to undefined function MoneroIntegrations\MoneroCrypto\gmp_add()

/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:168
/home/user/src/monerophptesting/monerophp/tests/unit/MnemonicTest.php:45

ERRORS!
Tests: 46, Assertions: 228, Errors: 3, Warnings: 2.
Script phpunit --testsuite unit handling the test:unit event returned with error code 2

so I've begun working and will be back with updates as they come.

note that this doesn't imply any review and these sorts of changes will require thorough review