Have DCLI scan the incoming script when compiling or running to look for embedded pubspec.yaml content

[RandalSchwartz]

Now that I can finally use DCLI, I'm finding that having the same default pubspec.yaml for all compiled scripts to be somewhat restrictive. My scripts vary wildly in their application. Perhaps I'd like Riverpod in this one, Dio in that one.

I thought at one time that one of these "dart as shebang" frameworks scanned the script-to-be-compiled looking for things to add to pubspec. They would be in specially demarked comments, so the syntax of Dart wouldn't be affected.

Can we get that for DCLI? It would really open up new applications.

[bsutton]

Originally DCli had the ability to specify a 'per' script set of imports.

The problem with this approach was that it was proprietary to dcli which mean that the IDE's didn't recognize the imports. This made editing your script harder as auto completes etc no longer worked.

When we looked out our internal usage of dcli (we have over 100K lines of scripts that use dcli) we realised that we really didn't need the the proprietary import mechanism.

Generally we are grouping related scripts into a single project directory (e.g. all of our production deployment scripts live together) which then share a single pubspec.yaml.

The key reason for keeping a pubspec.yaml 'trimmed' down is about the resulting exe size as it make little difference to how you develop a script. With cli scripts, size is not really an issue. A typical compiled script is around 7MB with around 15 dependencies.

Removing the proprietary approach also greatly simplified the dcli tooling.

At this point I don't expect to go back to that approach unless some compelling reason is presented.

[RandalSchwartz]

I don't understand where that "common" pubspec.yaml would live. The only one I've seen is the one in ~/.dcli/pubspec.yaml.

[bsutton]

So we have a number couple of projects each in its own directory:

provision\ pubspec.yaml bin\ deploy.dart upgrade.dart delete.dart

we then use the dcli compile -install option to put them on the path

cd provision\bin dcli compile -i *.dart

We can no run any of the scripts such as: deploy

I have another project where I store a number of helper functions:

dscripts\ pubspec.yaml bin\ dwhich.dart dfind.dart dipaddr.dart

Again I compile these so I can use them from anywhere.

When deploying a script to a remote system we generally compile it locally and then scp it to the remote.

Does that make sense?

On Mon, 4 Jan 2021 at 07:41, Randal L. Schwartz notifications@github.com wrote:

I don't understand where that "common" pubspec.yaml would live. The only one I've seen is the one in ~/.dcli/pubspec.yaml.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bsutton/dcli/issues/125#issuecomment-753673276, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OEOCIMBMR6EBEI7OLTSYDJBHANCNFSM4VRIQPHQ .

[RandalSchwartz]

Yeah, it makes perfect sense. So mkdir with pubspec.yaml, bin/* (and nothing else?) and then dcli compile -i bin/*.dart? Can that be added into the docs? :)

[bsutton]

@RandalSchwartz are you using 'dcli create'?

The create command creates an initial script, pubspec.yaml and directory.

It doesn't actually create the bin folder and just places the script in the root of the directory it creates.

This is somewhat due to history where the original idea was to allow a simple script to be created without a pubspec.

This proved impractical as you could no longer use an ide.

Now that we require a pubspec I'm wondering if we should create the full directory structure with a bin folder from scratch.

Thoughts?

[RandalSchwartz]

I like bin folder idea.

[RandalSchwartz]

Any progress on this? shall I work on a fork for you?

[bsutton]

If you look through the commit history dcli originally had the concept of s virtual project which allowed you to add a @pubspec annotation in the day dart script.

I remove support for multiple reasons.

1) it was hard to maintain 2) it used symlinks to link in a virtual pubspec which caused multiple problems on windows including an issue with dart which is still an open issue (list I looked). 3) it didn't work with an ide. This last one was the main reason.

At this point I'm not inclined to add the feature back in.

[RandalSchwartz]

No, on this part (which I would find very useful, and am still waiting for):

Now that we require a pubspec I'm wondering if we should create the full directory structure with a bin folder from scratch.

I concur with your reasons of not scanning the file, but putting my script into bin of a created directory would be awesome.

[Timmmm]

The problem with this is it makes distributing scripts much more awkward. I agree IDE support is fairly critical. Would it not be possible to have both for IDE development? So while developing a script you can have decent IDE support but you can still post a single file in a Github Gist or Slack or whatever.

That does mean you have to keep them in sync but that's a price I'd be willing to pay.

[bsutton]

What about a pack command.

It adds the deps as annotations/comments.

To run there script you have to unpack it which recreates the original directory.

On Tue, 20 Apr 2021, 8:43 pm Tim, @.***> wrote:

The problem with this is it makes distributing scripts much more awkward. I agree IDE support is fairly critical. Would it not be possible to have both for IDE development? So while developing a script you can have decent IDE support but you can still post a single file in a Github Gist or Slack or whatever.

That does mean you have to keep them in sync but that's a price I'd be willing to pay.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bsutton/dcli/issues/125#issuecomment-823173229, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OCSKKQGOPBIF5OKANTTJVLDVANCNFSM4VRIQPHQ .

[daniel-v]

When compiling a packed script, the dcli could detect that fact, create a temporary project directory for it and compile from there. The user experience would be that packed scripts compile just like project script based ones do.

[bsutton]

So one of my concerns is start up times if we have to scan every .dart file before we can run it.

So what about this.

We create a different extension, something like .dartp.

This would allow dcli to run existing .dart scripts without having to first scan the script.

The .dartp would be a 'packed dart script'. The packed file would still be a simple text file so it can be edited with vi.

When we detect a .dartp file we unpack the script into a .dartp/ directory under the current directory. We then run the project as a normal dart script.

This is essentially a slight simpler take on the original virtual projects. This would avoid needing symlinks (which is a problem under windows) as we would create a full copy of the script when unpacking it. Potential problem that comes to mind:

• what the current directory would be?
• We need to NOT change the current directory so the script cwd works as expected.
• Methods like DartSdk.packageRoot are possibly problematic.
• I don't have the time at the moment to undertake this.

[RandalSchwartz]

Any progress on this?

Now that we require a pubspec I'm wondering if we should create the full directory structure with a bin folder from scratch.

It's sooo close, but it puts the dart file at the pubspec level, not in bin. Please make a repo that complies with the rules.

[bsutton]

I'm still in two minds over this. One the one hand I'm a big fan of supporting the standard directory structure. On the other hand I like the concept of the script being in the top directory as it provides a more simplistic starting point.

[bsutton]

so we moved to supporting the standard dart directory structure with scripts in the bin directory.

This does work fairly well and its been a long time since anyone has asked about some novel architecture so will close this one out for now.