background compile scripts.

[bsutton]

Currently our start up time for non-compiled scripts is slower than bash.

We could improve this by doing a background compile of a script when it is launched.

On first launch we run the script using the standard jit launch. However we also start a background compile.

Next time the script is launched we check the timestamp on the script against the time stamp of the compile. If the compile is more recent then we launch from the compiled version.

We still have the over head of dshell being loaded so this may not end up improving the launch time. Worth an experiment.

An AOT snapshot doesn’t include the Dart runtime. Consider using snapshots if you’re distributing multiple programs and disk space is limited.

[whoizit]

~ > time dcli scripts/footq.dart
dcli scripts/footq.dart  13.12s user 2.94s system 72% cpu 22.127 total
~ > time dcli scripts/footq.dart
dcli scripts/footq.dart  13.34s user 2.74s system 121% cpu 13.190 total
~ > time dcli scripts/footq.dart
dcli scripts/footq.dart  13.11s user 2.58s system 135% cpu 11.604 total

~ > time .dcli/bin/footq
.dcli/bin/footq  0.08s user 0.05s system 46% cpu 0.296 total

13 seconds is too long for my small script https://codeberg.org/dnkl/foot/issues/604#issuecomment-249112

[bsutton]

So why are you not simply using the compiled versions?

dcli install
dcli compile --install --overwrite scripts/footq.dart
footq

Running 'dcli install' adds the ~/.dcli/bin directory to you PATH.

Here is a slightly simplified version of your script using dcli's existing functions:

#! /usr/bin/env dcli
import 'package:dcli/dcli.dart';

void main() {
  var sock_path =
      '${env["XDG_RUNTIME_DIR"]}/foot-${env["WAYLAND_DISPLAY"]}.sock';

  if (processExist('foot')) {
    'footclient --no-wait'.run;
  } else {
    if (exists(sock_path)) {
      delete(sock_path);
    }
    'foot --server --hold'.start(detached: true);

    while (true) {
      if (processExist('foot')) {
        'footclient --no-wait'.run;
        break;
      } else {
        sleep(100, interval: Interval.millseconds);
      }
    }
  }
}

bool processExist(String name) {
  var ph = ProcessHelper();

  for (var pd in ph.getProcesses()) {
    if (pd.name == name) {
      return true;
    }
  }

  return false;
}

I've added a method ProcessHelper().isProcessRunning(String name) which will be part of the next release of dcli.

[bsutton]

Also my times are much faster:

time ./footq.dart

real 0m3.309s user 0m4.616s sys 0m0.686s

try adding a setVerbose(enabled: true) at the top of your script to see where it might be stalling.

[whoizit]

Thank you very much for your fixes. The code has become very nice. With your fixes it is 5 seconds faster. I have old CPU (AMD FX-4300) Of course I am using the compiled version and it's very fast. Thaks for isProcessRunning()

~ > time dcli ./footq2.dart
info: client.c:420: exit-code=0
dcli ./footq2.dart  8.78s user 1.77s system 150% cpu 6.992 total

~ > time footq
info: client.c:420: exit-code=0
footq  0.06s user 0.05s system 9% cpu 1.088 total

[whoizit]

@bsutton quick question. Сall function ProcessHelper().isProcessRunning(String name) is too long, how-to do it shorter? I do:

  var ph = ProcessHelper();
  if (ph.isProcessRunning('foot')) {

But I'm not sure this is a good way to do it

[bsutton]

You suggested technique is fine.

On Wed, 25 Aug 2021, 12:42 am whoami, @.***> wrote:

@bsutton https://github.com/bsutton quick question. Сall function ProcessHelper().isProcessRunning(String name) is too long, how-to do it shorter? I do:

var ph = ProcessHelper();

if (ph.isProcessRunning('foot')) {

But I'm not sure this is a good way to do it

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/noojee/dcli/issues/96#issuecomment-904704674, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OC7DOCC3LQK45U4KQLT6OVWLANCNFSM4P7APT7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[whoizit]

practically the same thing dmypy does for mypy.

I think this should be implemented as an optional daemon that would compile scripts in the background if they change.

https://mypy.readthedocs.io/en/stable/mypy_daemon.html https://manpages.debian.org/testing/mypy/dmypy.1.en.html https://command-not-found.com/dmypy

[bsutton]

I don't think that is quite the same thing. However I can see a path forward where:

1) the first time we run a script we run it in jit mode but also launch a background compile. 2) we compile the script to a cache directory under ~/.dcli. The directory under dcli would need to use the full path of the script so we don't get conflicts if the script lives in /home/user/myscript.dart then the compiled exe would live in e.g. /home/user/.dcli/home/user/myscript.dart We will need to translate the drive letter to something like c_drive on windows.

3) the second time we run the script we see the compiled exe and compare its creation date/time with the last modified time on the script If the compiled script creation date/time is after the script modification time we run the compiled version, if not we go back to 1).

There is already a fair bit of code written to support this.

DartProject.warmup - does a pub get on a project in the background DartScript.compile - compiles a script

The run.dart code would need to be modified to implement the above steps.

If you want to have a crack at doing this I'm happy to answer questions as you go.

S. Brett Sutton Noojee Contact Solutions 03 8320 8100

On Wed, 25 Aug 2021 at 09:29, whoami @.***> wrote:

practically the same thing dmypy does for mypy.

I think this should be implemented as an optional daemon that would compile scripts in the background if they change.

https://mypy.readthedocs.io/en/stable/mypy_daemon.html https://manpages.debian.org/testing/mypy/dmypy.1.en.html https://command-not-found.com/dmypy

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/noojee/dcli/issues/96#issuecomment-905042311, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OCM4K6YIQ2XM7XOILTT6QTMNANCNFSM4P7APT7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[whoizit]

on step 1 we should start compiling after running (in background), not in the same time. because it can slow down startup time for jit.

on step 2, we should store the timestamp when the original file (for example /home/user/myscript.dart) was modified. For example into /home/user/.dcli/home/user/myscript.dart.timestamp.

on step 3 we should compare modified time for script /home/user/myscript.dart with timestamp stored in /home/user/.dcli/home/user/myscript.dart.timestamp

[whoizit]

sounds pretty simple

[bsutton]

I don't think we need to store the timestamp as we can directly check each files date/time. This will be more authoritative.

The dcli 'stat' function will give you the details.

S. Brett Sutton Noojee Contact Solutions 03 8320 8100

On Wed, 25 Aug 2021 at 10:25, whoami @.***> wrote:

also on step 2, we should store the timestamp when the original file (for example /home/user/myscript.dart) was modified. For example into /home/user/.dcli/home/user/myscript.dart.timestamp.

And on step 3 we should compare modified time for script /home/user/myscript.dart with timestamp stored in /home/user/.dcli/home/user/myscript.dart.timestamp

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/noojee/dcli/issues/96#issuecomment-905064333, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OCLDAW3F2RDOSGQV2DT6QZ7ZANCNFSM4P7APT7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[whoizit]

the time of modification of the original file will be different from the time of creation/modification of the compiled file. I don't know what you mean.

[bsutton]

We need to detect if the original script had been modified since we last complied it. If it has them we need to recompile it.

We can check if the script needs to be recompiled by comparing the scripts last modified date with the compiled exes creation date.

On Wed, 25 Aug 2021, 10:37 am whoami, @.***> wrote:

the time of modification of the original file will be different from the time of creation/modification of the compiled file. I don't know what you mean.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/noojee/dcli/issues/96#issuecomment-905070010, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OBA7ZAYYNMINQXKZB3T6Q3NBANCNFSM4P7APT7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[whoizit]

Ok, now I understand

On Wed, Aug 25, 2021, 5:34 AM Brett Sutton @.***> wrote:

We need to detect if the original script had been modified since we last complied it. If it has them we need to recompile it.

We can check if the script needs to be recompiled by comparing the scripts last modified date with the compiled exes creation date.

On Wed, 25 Aug 2021, 10:37 am whoami, @.***> wrote:

the time of modification of the original file will be different from the time of creation/modification of the compiled file. I don't know what you mean.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/noojee/dcli/issues/96#issuecomment-905070010, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAG32OBA7ZAYYNMINQXKZB3T6Q3NBANCNFSM4P7APT7A

. Triage notifications on the go with GitHub Mobile for iOS < https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android < https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/noojee/dcli/issues/96#issuecomment-905130016, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACXW3HCL76SLHXHOVP5EETT6RJCTANCNFSM4P7APT7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .