Closed rustdevbtw closed 3 weeks ago
Here is another benchmark, comparing node.unsafe
(installed through --unsafe
), node.safe
(normally installed), pkgx node
, and the system-installed (in this case, installed from Arch repos) node
:
I believe (though I could be wrong) that the 1.1x diff is because of the shell overhead, and overhead of checking files.
Changed the mkdir -p
(on the first run, or when the cache doesn't exist) to do that on $(dirname ~/.cache/pkgx/envs/gnu.org/emacs.env)
so it creates the gnu.org
dir also (the install also takes care of this, but if the cache is missing, it'll cause error, as mentioned in 1st caveat). It handles the first caveat.
@jhheider It seems to be okay, but I'm not in PC rn (it's 1:02AM here lol). Can you try it now please, if possible?
@jhheider It seems to be okay, but I'm not in PC rn (it's 1:02AM here lol). Can you try it now please, if possible?
bugs in the script, but i can add a commit to fix them.
but the speed looks good:
pkgx hyperfine "node --version"
Benchmark 1: node --version
Time (mean ± σ): 26.4 ms ± 2.1 ms [User: 18.8 ms, System: 3.5 ms]
Range (min … max): 25.0 ms … 46.3 ms 103 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
@jhheider It seems to be okay, but I'm not in PC rn (it's 1:02AM here lol). Can you try it now please, if possible?
bugs in the script, but i can add a commit to fix them.
Thanks. Sucks to be without laptop/PC, lol. Anyways, thanks again.
but the speed looks good:
pkgx hyperfine "node --version" Benchmark 1: node --version Time (mean ± σ): 26.4 ms ± 2.1 ms [User: 18.8 ms, System: 3.5 ms] Range (min … max): 25.0 ms … 46.3 ms 103 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Is that right after doing the install, or after doing an initial run? It caches only at the first run, so subsequent runs should be faster than the first one. And the first one might interfere with the rest.
You mean, the env should take priority?
No, I mean if either is set, it should be on. Which is the way you have it, I believe. Setting the envvar to zero shouldn't do anything.
You mean, the env should take priority?
No, I mean if either is set, it should be on. Which is the way you have it, I believe. Setting the envvar to zero shouldn't do anything.
By shouldn't do anything, do you mean it should ignore that? So, if that's set, regardless of its value, it should do an unsafe install?
You mean, the env should take priority?
No, I mean if either is set, it should be on. Which is the way you have it, I believe. Setting the envvar to zero shouldn't do anything.
By shouldn't do anything, do you mean it should ignore that? So, if that's set, regardless of its value, it should do an unsafe install?
If yes, it gets simpler:
function is_unsafe(unsafe: boolean): boolean {
return Deno.env.get("PKGX_UNSAFE_INSTALL") || unsafe;
}
So, if that's set, regardless of its value, it should do an unsafe install?
No. If it's non-zero, or the --unsafe flag is passed, it should do an unsafe install. If the envvar is zero, it has no effect one way or the other.
The default behaviour (without any env or flag) is to install normally. So, if the env is set to 0, it should do that? Sorry for sounding like an idiot.
On Wed, 15 May, 2024, 8:14 pm Jacob Heider, @.***> wrote:
So, if that's set, regardless of its value, it should do an unsafe install?
No. If it's non-zero, or the --unsafe flag is passed, it should do an unsafe install. If the envvar is zero, it has no effect one way or the other.
— Reply to this email directly, view it on GitHub https://github.com/pkgxdev/pkgx/pull/1008#issuecomment-2112741688, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATXDOIZKEBPBKQ6ZCQFUVFDZCNYEJAVCNFSM6AAAAABHSDC4DWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJSG42DCNRYHA . You are receiving this because you authored the thread.Message ID: @.***>
The default behaviour (without any env or flag) is to install normally. So, if the env is set to 0, it should do that?
Exactly. Setting it 0 is the same as not setting it. So, it doesn't affect behavior.
If it's set to 0 and the flag is passed it's unsafe. If it's 0 and no flag, then safe. Set to 0 is a no-op.
The default behaviour (without any env or flag) is to install normally. So, if the env is set to 0, it should do that?
Exactly. Setting it 0 is the same as not setting it. So, it doesn't affect behavior.
If it's set to 0 and the flag is passed it's unsafe. If it's 0 and no flag, then safe. Set to 0 is a no-op.
So, this:
function is_unsafe(unsafe: boolean): boolean {
const env = parseInt(Deno.env.get("PKGX_UNSAFE_INSTALL") || "0") ? true : false;
return env || unsafe;
}
With this, there are six possible combinations:
@jhheider implemented it, is it okay? If so, can you please test that the env works as expected? (It probably does, but still a test is a good idea)
right, i don't think that changed anything. your original implementation checked the truth value of the env || the flag. so, it should still be working correctly.
So @jhheider anything else that's left? (Maybe check if it's by PKGX part? We can improve that)
Also @mxcl can you please share thoughts?
looks like good work
- pkgx install now checks for pkgx (instead of exec pkgx) in a script to consider it as installed by PKGX (because unsafe installations don't use exec pkgx)
I've got to say this approach is just as not future-proof as the previous. Perhaps we should fix it for good by adding some known comment to the stubs? Like:
#!/bin/sh
# managed by pkgx
as proposed by @jhheider (thanks!)
Guys, I don't want to be silly, but I had proposed this on Apr 11. @jhheider is there a chance you missed my comment and later came up with the same proposal?
Don't get me wrong, all I want is pkgx
to be improved. I'm asking for pure curiosity.
- pkgx install now checks for pkgx (instead of exec pkgx) in a script to consider it as installed by PKGX (because unsafe installations don't use exec pkgx)
I've got to say this approach is just as not future-proof as the previous. Perhaps we should fix it for good by adding some known comment to the stubs? Like:
#!/bin/sh # managed by pkgx
That is possible, but it'll break compatibility with the stubs installed by older versions. PKGX wouldn't detect them as installed by PKGX if we completely move to this approach. As for unsafe install, I've had the idea to see and check for .env files (generated for unsafe installations only). That's not as reliable, though, because it would fail if the cache was somehow deleted, or not yet generated.
That is possible, but it'll break compatibility with the stubs installed by older versions. PKGX wouldn't detect them as installed by PKGX if we completely move to this approach.
Perhaps we can keep searching for pkgx
while still in pkgx 1.x, and maybe drop support for it in pkgx 2.x. Anyway, this can be done in another PR, no need to have it done here.
as proposed by @jhheider (thanks!)
Guys, I don't want to be silly, but I had proposed this on Apr 11. @jhheider is there a chance you missed my comment and later came up with the same proposal?
Don't get me wrong, all I want is
pkgx
to be improved. I'm asking for pure curiosity.
Ha. I don't specifically recall your comment, but, yes, we were talking in discord about what and how to cache. I certainly don't deserve the credit for the idea, but we came to the same conclusion as you.
maybe i read that proposal before coming up with this PR, and forgot about it before doing it. it is certainly inspired by your proposal. so, i think you deserve the credit!
On Sat, Jun 8, 2024 at 1:42 AM Felipe Santos @.***> wrote:
as proposed by @jhheider (thanks!)
Guys, I don't want to be silly, but I had proposed this on Apr 11. @jhheider is there a chance you missed my comment and later came up with the same proposal?
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
That is possible, but it'll break compatibility with the stubs installed by older versions. PKGX wouldn't detect them as installed by PKGX if we completely move to this approach.
Perhaps we can keep searching for
pkgx
while still in pkgx 1.x, and maybe drop support for it in pkgx 2.x. Anyway, this can be done in another PR, no need to have it done here.
That sounds like a good option. What do you think @jhheider?
We can also add that to unsafe install now (which is in scope of this PR), because backwards compatibility doesn't exist for it.
sure, sounds like a good idea.
does #MANAGED BY PKGX
sound good for that comment? (it's unique enough
to not cause conflicts)
go for it!
go for it!
done!
Also, updated the example to reflect the current diff
(as of now)
Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | ||
---|---|---|---|---|---|
src/parse-args.ts | 2 | 5 | 40.0% | ||
<!-- | Total: | 2 | 5 | 40.0% | --> |
Files with Coverage Reduction | New Missed Lines | % | ||
---|---|---|---|---|
src/utils/execve.ts | 10 | 81.72% | ||
<!-- | Total: | 10 | --> |
Totals | |
---|---|
Change from base Build 10740131780: | -1.5% |
Covered Lines: | 1475 |
Relevant Lines: | 1603 |
Nice feature. Additive so easy merge.
closes #997 closes #991
This PR adds support for the
--unsafe
install flag to PKGX, which, as proposed by @jhheider and @felipecrs (thanks!), creates additional env stubs for a package on the first run, making subsequent runs make use of the already installed binaries, thus making it faster[1].The stub
As for the stub, it makes it a bit different, for instance, here's a "safe"
node
wrapper (installed normally):And here's the "unsafe" (as in experimental)
node
wrapper:Additional changes
Additionally, this PR also makes the following behavioural changes to PKGX:
pkgx uninstall
now also removes the env stub of the packagepkgx install
now checks for bothexec pkgx
(old behaviour) and#MANAGED BY PKGX
(comment) to determine whether that file was created by PKGX.Usage
To do an unsafe installation, there are two options:
--unsafe
flag (e.gpkgx install --unsafe node
)PKGX_UNSAFE_INSTALL
environmental variable.In the presence of both, the
--unsafe
flag takes precedence.Caveats
This approach doesn't come without any caveat. It's necessary to document them upon having such a functionality. As per my knowledge, these are the known issues:
Most (if not all) of them error correctly and offer (potentially) helpful error messages.
[1]