search

kint-php / kint

Kint - Advanced PHP dumper
https://kint-php.github.io/kint/
MIT License
2.78k stars 292 forks source link

Reproducible phar builds on Windows #425

Closed DRSDavidSoft closed 1 month ago

DRSDavidSoft commented 1 month ago 
diff --git a/build.php b/build.php
index 6ab0edf0..b06fe7a8 100644
--- a/build.php
+++ b/build.php
@@ -56,6 +56,7 @@ $filesToArchive = Finder::create()
     ->sortByName();

 foreach ($filesToArchive as $file) {
+    echo ((string) $file).PHP_EOL;
     $local = \substr((string) $file, $pathlen);
     $phar->addFile((string) $file, $local);
     $pi = new PharFileInfo('phar://'.$outpath.$local);

It looks like the files are being added to the phar in a different order on windows. Can you apply the above patch to build.php and paste me the output? Might be able to fix the reproducible build on windows too then.

Here's the output:

C:\Users\David\Desktop\kint (win-tput -> origin)
λ composer build
> npm run build:sass

> build:sass
> sass -s compressed --no-source-map resources/sass/:resources/compiled/

> npm run build:js

> build:js
> esbuild resources/js/main.js --bundle --minify --outfile=resources/compiled/main.js

  resources\compiled\main.js  14.8kb

⚡ Done in 10ms
> php ./build.php
C:\Users\David\Desktop\kint/init.php
C:\Users\David\Desktop\kint/init_helpers.php
C:\Users\David\Desktop\kint/init_phar.php
C:\Users\David\Desktop\kint/resources/compiled\aante-dark.css
C:\Users\David\Desktop\kint/resources/compiled\aante-light.css
C:\Users\David\Desktop\kint/resources/compiled\main.js
C:\Users\David\Desktop\kint/resources/compiled\original.css
C:\Users\David\Desktop\kint/resources/compiled\plain.css
C:\Users\David\Desktop\kint/resources/compiled\solarized-dark.css
C:\Users\David\Desktop\kint/resources/compiled\solarized.css
C:\Users\David\Desktop\kint/src\CallFinder.php
C:\Users\David\Desktop\kint/src\FacadeInterface.php
C:\Users\David\Desktop\kint/src\Kint.php
C:\Users\David\Desktop\kint/src\Parser\AbstractPlugin.php
C:\Users\David\Desktop\kint/src\Parser\ArrayLimitPlugin.php
C:\Users\David\Desktop\kint/src\Parser\ArrayObjectPlugin.php
C:\Users\David\Desktop\kint/src\Parser\Base64Plugin.php
C:\Users\David\Desktop\kint/src\Parser\BinaryPlugin.php
C:\Users\David\Desktop\kint/src\Parser\BlacklistPlugin.php
C:\Users\David\Desktop\kint/src\Parser\ClassHooksPlugin.php
C:\Users\David\Desktop\kint/src\Parser\ClassMethodsPlugin.php
C:\Users\David\Desktop\kint/src\Parser\ClassStaticsPlugin.php
C:\Users\David\Desktop\kint/src\Parser\ClassStringsPlugin.php
C:\Users\David\Desktop\kint/src\Parser\ClosurePlugin.php
C:\Users\David\Desktop\kint/src\Parser\ColorPlugin.php
C:\Users\David\Desktop\kint/src\Parser\ConstructablePluginInterface.php
C:\Users\David\Desktop\kint/src\Parser\DOMDocumentPlugin.php
C:\Users\David\Desktop\kint/src\Parser\DateTimePlugin.php
C:\Users\David\Desktop\kint/src\Parser\DomPlugin.php
C:\Users\David\Desktop\kint/src\Parser\EnumPlugin.php
C:\Users\David\Desktop\kint/src\Parser\FsPathPlugin.php
C:\Users\David\Desktop\kint/src\Parser\HtmlPlugin.php
C:\Users\David\Desktop\kint/src\Parser\IteratorPlugin.php
C:\Users\David\Desktop\kint/src\Parser\JsonPlugin.php
C:\Users\David\Desktop\kint/src\Parser\MicrotimePlugin.php
C:\Users\David\Desktop\kint/src\Parser\MysqliPlugin.php
C:\Users\David\Desktop\kint/src\Parser\Parser.php
C:\Users\David\Desktop\kint/src\Parser\PluginInterface.php
C:\Users\David\Desktop\kint/src\Parser\ProfilePlugin.php
C:\Users\David\Desktop\kint/src\Parser\ProxyPlugin.php
C:\Users\David\Desktop\kint/src\Parser\SerializePlugin.php
C:\Users\David\Desktop\kint/src\Parser\SimpleXMLElementPlugin.php
C:\Users\David\Desktop\kint/src\Parser\SplFileInfoPlugin.php
C:\Users\David\Desktop\kint/src\Parser\StreamPlugin.php
C:\Users\David\Desktop\kint/src\Parser\TablePlugin.php
C:\Users\David\Desktop\kint/src\Parser\ThrowablePlugin.php
C:\Users\David\Desktop\kint/src\Parser\TimestampPlugin.php
C:\Users\David\Desktop\kint/src\Parser\ToStringPlugin.php
C:\Users\David\Desktop\kint/src\Parser\TracePlugin.php
C:\Users\David\Desktop\kint/src\Parser\XmlPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\AbstractRenderer.php
C:\Users\David\Desktop\kint/src\Renderer\AssetRendererTrait.php
C:\Users\David\Desktop\kint/src\Renderer\CliRenderer.php
C:\Users\David\Desktop\kint/src\Renderer\ConstructableRendererInterface.php
C:\Users\David\Desktop\kint/src\Renderer\PlainRenderer.php
C:\Users\David\Desktop\kint/src\Renderer\RendererInterface.php
C:\Users\David\Desktop\kint/src\Renderer\RichRenderer.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\AbstractPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\ArrayLimitPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\BinaryPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\BlacklistPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\CallablePlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\ClosurePlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\ColorPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\DepthLimitPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\FilesizePlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\MethodDefinitionPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\MicrotimePlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\PluginInterface.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\ProfilePlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\RecursionPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\SimpleXMLElementPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\SourcePlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\TabPluginInterface.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\TablePlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\TimestampPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\TraceFramePlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Rich\ValuePluginInterface.php
C:\Users\David\Desktop\kint/src\Renderer\TextRenderer.php
C:\Users\David\Desktop\kint/src\Renderer\Text\AbstractPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Text\ArrayLimitPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Text\BlacklistPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Text\DepthLimitPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Text\EnumPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Text\IteratorPrimaryPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Text\MicrotimePlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Text\PluginInterface.php
C:\Users\David\Desktop\kint/src\Renderer\Text\RecursionPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Text\SplFileInfoPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Text\StreamPlugin.php
C:\Users\David\Desktop\kint/src\Renderer\Text\TracePlugin.php
C:\Users\David\Desktop\kint/src\Utils.php
C:\Users\David\Desktop\kint/src\Zval\BlobValue.php
C:\Users\David\Desktop\kint/src\Zval\ClosureValue.php
C:\Users\David\Desktop\kint/src\Zval\DateTimeValue.php
C:\Users\David\Desktop\kint/src\Zval\EnumValue.php
C:\Users\David\Desktop\kint/src\Zval\InstanceValue.php
C:\Users\David\Desktop\kint/src\Zval\MethodValue.php
C:\Users\David\Desktop\kint/src\Zval\ParameterHoldingTrait.php
C:\Users\David\Desktop\kint/src\Zval\ParameterValue.php
C:\Users\David\Desktop\kint/src\Zval\Representation\ColorRepresentation.php
C:\Users\David\Desktop\kint/src\Zval\Representation\MethodDefinitionRepresentation.php
C:\Users\David\Desktop\kint/src\Zval\Representation\MicrotimeRepresentation.php
C:\Users\David\Desktop\kint/src\Zval\Representation\ProfileRepresentation.php
C:\Users\David\Desktop\kint/src\Zval\Representation\Representation.php
C:\Users\David\Desktop\kint/src\Zval\Representation\SourceRepresentation.php
C:\Users\David\Desktop\kint/src\Zval\Representation\SplFileInfoRepresentation.php
C:\Users\David\Desktop\kint/src\Zval\ResourceValue.php
C:\Users\David\Desktop\kint/src\Zval\SimpleXMLElementValue.php
C:\Users\David\Desktop\kint/src\Zval\StreamValue.php
C:\Users\David\Desktop\kint/src\Zval\ThrowableValue.php
C:\Users\David\Desktop\kint/src\Zval\TraceFrameValue.php
C:\Users\David\Desktop\kint/src\Zval\TraceValue.php
C:\Users\David\Desktop\kint/src\Zval\Value.php
jnvsor commented 1 month ago

TIL slash comes before letters in ascii and backslash comes after... Ok let me think of how to fix this

DRSDavidSoft commented 1 month ago

So weird, TIL this also! 😄

The Finder component docs apparently doesn't have a built-in sorter to deal with this, they sadly suggest using something like this:

$filesToArchive->sort(function (\SplFileInfo $a, \SplFileInfo $b): int {
    return strcmp($a->getRealPath(), $b->getRealPath());
});

instead of:

$finder->sortByName();

Alternatively, you can use DIRECTORY_SEPARATOR instead of using hard-coded /:

    ->in([__DIR__.DIRECTORY_SEPARATOR.'src', __DIR__.DIRECTORY_SEPARATOR.'resources'.DIRECTORY_SEPARATOR.'compiled'])
    ->append([
        __DIR__.DIRECTORY_SEPARATOR.'init_phar.php',
        __DIR__.DIRECTORY_SEPARATOR.'init.php',
        __DIR__.DIRECTORY_SEPARATOR.'init_helpers.php',
    ])

instead of:

    ->in([__DIR__.'/src', __DIR__.'/resources/compiled'])
    ->append([
        __DIR__.'/init_phar.php',
        __DIR__.'/init.php',
        __DIR__.'/init_helpers.php',
    ])

But this will look ugly and hard to read. So maybe you can wrap the __DIR__.'/some/path statements in a realpath() call.

I personally feel like apologizing for the / vs \ inconsistency in Windows systems. 😔

jnvsor commented 1 month ago

Alternatively, you can use DIRECTORY_SEPARATOR

That's not the problem. We're telling the finder to grab all the files in src and the finder is giving them backslashes. Actually the finder is weird to begin with since those paths you echoed have a mixture of slashes and backslashes in a single file.

I don't think getRealPath would work because even if it's all backslashes on windows you'll still end up with a different ordering from linux where it produces slashes. I think we're going to have to replace the backslashes in the paths for a consistent sort. Let me whip up a prototype and you can try it out

jnvsor commented 1 month ago

@DRSDavidSoft Can you try this?

diff --git a/build.php b/build.php
index b4c941f8..0436bdca 100644
--- a/build.php
+++ b/build.php
@@ -60,6 +60,14 @@ $filesToArchive = Finder::create()
     ])
     ->sortByName();

+if (KINT_WIN) {
+    $filesToArchive->sort(static function (SplFileInfo $a, SplFileInfo $b) {
+        $a = strtr($a->getRealPath() ?: $a->getPathname(), '\\', '/');
+        $b = strtr($b->getRealPath() ?: $b->getPathname(), '\\', '/');
+        return strcmp($a, $b);
+    });
+}
+
 foreach ($filesToArchive as $file) {
     $local = \substr((string) $file, $pathlen);
     $phar->addFile((string) $file, $local);
DRSDavidSoft commented 1 month ago

This definitely fixes it

C:\Users\David\Desktop\kint>sha1sum build\kint.phar
\d1a8d851ad71f2ba207809af7e666061c91cf210 *build\\kint.phar

C:\Users\David\Desktop\kint>composer clean

C:\Users\David\Desktop\kint>composer build
....

C:\Users\David\Desktop\kint>sha1sum build\kint.phar
\d1a8d851ad71f2ba207809af7e666061c91cf210 *build\\kint.phar

Notice the first file is the one already built correctly.

jnvsor commented 1 month ago

Great!