search

onepub-dev / dcli

An extensive library and tooling for building console/cli applications and scripts using the Dart programming language.
242 stars 28 forks source link

dcli should not presume pub is in PATH or any particular directory #120

Closed RandalSchwartz closed 3 years ago

RandalSchwartz commented 3 years ago

Describe the bug dcli_install fails by presuming pub is in PATH.

To Reproduce

dart pub global activate dcli
sudo dcli_install # fails by not finding pub

Expected behavior I expected it to install the rest of dcli.

Logs

Run dcli doctor: From the cli run dcli doctor

Locohost.local:~ +% dcli doctor
DCli version      0.34.6                                                 

OS                macos                                                  
OS Version        10.15.7
Path separator    /                                                      

dart version      2.12.0                                                 

dcli path         /<HOME>/.pub-cache/bin/dcli                            
dart exe path     /<HOME>/Flutter/bin/dart                               
dart path         /<HOME>/Flutter/bin/dart                                which: /<HOME>/Flutter/bin/dart                        
dart2Native path  Not Found                                              

pub path          /<HOME>/Flutter/bin/pub                                 which: /<HOME>/bin/pub                                 
Pub cache         /<HOME>/.pub-cache                                     

Package Config    Not Passed                                             

PATH
                  /Library/Frameworks/Python.framework/Versions/3.6/bin  
                  /<HOME>/bin                                            
                  /opt/git/bin                                           
                  /<HOME>/MIRROR/rakudo-GIT/install/bin                  
                  /<HOME>/MIRROR/rakudo-GIT/install/share/perl6/site/bin 
                  /opt/perl/bin                                          
                  /opt/local/bin                                         
                  /opt/local/sbin                                        
                  /<HOME>/Flutter/.pub-cache/bin                         
                  /<HOME>/Flutter/bin                                    
                  /<HOME>/.pub-cache/bin                                 
                  /<HOME>/MIRROR/brew-GIT/bin                            
                  /usr/local/bin                                         
                  /usr/bin                                               
                  /bin                                                   
                  /usr/sbin                                              
                  /sbin                                                  
                  /Applications/VMware Fusion.app/Contents/Public        
                  /usr/local/share/dotnet                                
                  /opt/X11/bin                                           
                  /<HOME>/~/.dotnet/tools                                
                  /Library/Apple/usr/bin                                 
                  /Library/Frameworks/Mono.framework/Versions/Current/Com
                  /<HOME>/Library/Android/sdk/platform-tools             

$SHELL            /bin/zsh                                               
Detected SHELL    zsh                                                    
Shell Start Scrip Not Found                                              

Dart location(s)
                  /Users/merlyn/Flutter/bin/dart                         

Permissions
HOME              rwxr-xr-x <user>: /<HOME> 
.dcli             /<HOME>/.dcli does not exist                           
cache             /<HOME>/.dcli/cache does not exist                     
templates         /<HOME>/.dcli/templates does not exist

Additional context If you're expecting dart2native in the path, that has been replaced with dart compile exe in the latest releases.

bsutton commented 3 years ago

Can you posted the actual error that dcli generated.

On Sun, 20 Dec 2020, 4:16 am Randal L. Schwartz, notifications@github.com wrote:

Describe the bug dcli_install fails by presuming pub is in PATH.

To Reproduce

dart pub global activate dcli sudo dcli_install # fails by not finding pub

Expected behavior I expected it to install the rest of dcli.

Logs

Run dcli doctor: From the cli run dcli doctor

Locohost.local:~ +% dcli doctor DCli version 0.34.6

OS macos OS Version 10.15.7 Path separator /

dart version 2.12.0

dcli path //.pub-cache/bin/dcli dart exe path //Flutter/bin/dart dart path //Flutter/bin/dart which: //Flutter/bin/dart dart2Native path Not Found

pub path //Flutter/bin/pub which: //bin/pub Pub cache //.pub-cache

Package Config Not Passed

PATH /Library/Frameworks/Python.framework/Versions/3.6/bin //bin /opt/git/bin //MIRROR/rakudo-GIT/install/bin //MIRROR/rakudo-GIT/install/share/perl6/site/bin /opt/perl/bin /opt/local/bin /opt/local/sbin //Flutter/.pub-cache/bin //Flutter/bin //.pub-cache/bin //MIRROR/brew-GIT/bin /usr/local/bin /usr/bin /bin /usr/sbin /sbin /Applications/VMware Fusion.app/Contents/Public /usr/local/share/dotnet /opt/X11/bin //~/.dotnet/tools /Library/Apple/usr/bin /Library/Frameworks/Mono.framework/Versions/Current/Com //Library/Android/sdk/platform-tools

$SHELL /bin/zsh Detected SHELL zsh Shell Start Scrip Not Found

Dart location(s) /Users/merlyn/Flutter/bin/dart

Permissions HOME rwxr-xr-x : / .dcli //.dcli does not exist cache //.dcli/cache does not exist templates //.dcli/templates does not exist

Additional context If you're expecting dart2native in the path, that has been replaced with dart compile exe in the latest releases.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bsutton/dcli/issues/120, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OFISF2SFJSZIRSHBMLSVTNXVANCNFSM4VCNWXQQ .

lohanidamodar commented 3 years ago

@bsutton I was having the similar problem. dcli install was a success however, dcli compile failed because of the assumed path of the pub. When I checked dcli doctor the path for pub and dart2native assumed by dcli was different from the actual in my system, that was showing by the dcli doctor on the right side where:..., so I created a soft link from assumed path to the actual path, only then I was able to successfully compile and install my scripts.

lohanidamodar commented 3 years ago

@bsutton @RandalSchwartz Me thinking, If dart is in the path, can we not simply

  1. instead of pub get use dart pub get
  2. instead of dart2native use dart compile exe ?
bsutton commented 3 years ago

Can you post details of your path and what dcli found it will make it easier when I go digging into the root cause.

Also are you in osx as well?

I will update the code to use dart compile. Do you know from what version that changes from dart2native?

I wasn't aware of the dart pub get option, I assume that is also new option. There is certainly an argument to only rely on dart rather than dart and pub.

lohanidamodar commented 3 years ago

@bsutton

  1. I am on Linux
  2. Looks like dart cli was available from dart 2.10 (based on this https://dart.dev/tools/dart-tool)
  3. Here is the output of my dcli doctor before i created the softlink 
    
DCli version      0.34.6

OS linux
OS Version Linux 5.4.0-58-generic #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020 Path separator /

dart version 2.10.4

dcli path /opt/flutter/bin/dcli
dart exe path /opt/flutter/bin/dart
dart path /opt/flutter/bin/dart which: /opt/flutter/bin/dart
dart2Native path /opt/flutter/bin/dart2native which: /opt/flutter/bin/cache/dart-sdk/bin/dart2native

pub path /opt/flutter/bin/pub which: /opt/flutter/bin/cache/dart-sdk/bin/pub
Pub cache //.pub-cache

Package Config Not Passed

PATH /usr/local/go/bin
//.npm/bin
//Documents/projects/flutter/bin
//.local/bin
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin
/usr/games
/usr/local/games
/snap/bin
//Android/Sdk/tools
//Android/Sdk/platform-tools
/opt/flutter/bin
/opt/flutter/.pub-cache/bin
/opt/flutter/bin/cache/dart-sdk/bin
//.pub-cache/bin
//.dcli/bin

$SHELL /usr/bin/zsh
Detected SHELL zsh
Shell Start Scrip Not Found

Dart location(s) /opt/flutter/bin/dart

Permissions HOME rwxr-xr-x : / .dcli rwxrwxr-x : //.dcli cache rwxrwxr-x : //.dcli/cache templates rwxrwxr-x : //.dcli/templates

bsutton commented 3 years ago

@lohanidamodar @RandalSchwartz

I'm still struggling to see what is going on here.

From your dcli doctor output it looks like dart is on the path: //Documents/projects/flutter/bin /opt/flutter/bin

dcli_install is meant to search for dart on your path. If it doesn't find it then it will install dart and add it to your path.

Clearly from the doctor output you both had dart and pub installed (via flutter) and @lohanidamodar has dart2native on his path while @RandalSchwartz doesn't. So I'm not even certain if you are both having the same problem.

I really need to see a trace of the error from dcli_install so I can see the stack trace.

I do note that the one thing in common here is that you are both running zsh.

When you run dcli_install please pass in the -v switch:

dcli_install -v

Please post the complete trace.

lohanidamodar commented 3 years ago

@bsutton For me as i described and you can see in the output of dcli doctor the path for pub and dart2native are not in the path assumed by the dcli. And for me the problem is not during the dcli install its with dcli compile where it's unable to find the pub and pubGetException is raised as below

Setting flag: verbose
DCli Version: 0.34.6
Found command Instance of 'CompileCommand'
Setting flag: install

Compiling ./pcreate.dart...

exists: ./pcreate.dart followLinks: true
env:  SHELL:/usr/bin/zsh
env:  SHELL:/usr/bin/zsh
env:  SUDO_USER:null
path: /<path>/dart/scripts/pcreate.dart
exists: /<path>/dart/scripts/pubspec.yaml followLinks: true
lockcount = 0
Hardlock taken
exists: /<path>/dart/scripts followLinks: true
find: pwd: /home/dlohani/Documents/dart/scripts root: /<path>/dart/scripts pattern: *.script.lock caseSensitive: false recursive: false types: [file] 
found [] lock files
Taking lock .486796.2833298015332279.script.lock for isolates/2833298015332279
Lock Source: named_lock.dart : NamedLock.withLock : 104
touch: /<path>/dart/scripts/.486796.2833298015332279.script.lock create: true
exists: /<path>/dart/scripts followLinks: true
Hardlock released
Waiting for warmup to complete...
Incremented lock: 1
lockcount = 1
Incremented lock: 2
env:  SUDO_USER:null
env:  PATH:/usr/local/go/bin:/opt/flutter/bin:/opt/flutter/.pub-cache/bin:/opt/flutter/bin/cache/dart-sdk/bin:/home/dlohani/.pub-cache/bin:/home/dlohani/.dcli/bin
exists: /usr/local/go/bin/dart followLinks: true
exists: /opt/flutter/bin/dart followLinks: true
Process.start: cmdLine /opt/flutter/bin/pub get --no-precompile
Process.start: runInShell: false workingDir: /<path>/dart/scripts mode: normal cmd: /opt/flutter/bin/pub args: get, --no-precompile
exists: /home/dlohani/Documents/dart/scripts followLinks: true
pub get exeception: /opt/flutter/bin/pub get --no-precompile 
exit: 2
reason: Could not find /opt/flutter/bin/pub on the path.
Decremented lock: 1
Decremented lock: 0
Releasing lock: /<path>/dart/scripts/.486796.2833298015332279.script.lock
Hardlock taken
delete:  /<path>/dart/scripts/.486796.2833298015332279.script.lock ask: false
exists: /<path>/dart/scripts/.486796.2833298015332279.script.lock followLinks: true
Hardlock released
DCliException: Instance of 'PubGetException' 
Stacktrace: named_lock.dart : NamedLock.withLock.<anonymous closure> : 127
zone.dart : runZonedGuarded : 1620
zone.dart : runZoned : 1547
named_lock.dart : NamedLock.withLock : 104
dart_project.dart : DartProject.warmup : 126
compile.dart : CompileCommand.compileScript : 108
compile.dart : CompileCommand.run : 65
command_line_runner.dart : CommandLineRunner.process : 86
entry_point.dart : EntryPoint._parseCmdLine : 40
entry_point.dart : EntryPoint.process : 34
bsutton commented 3 years ago

Damodar Lohani, can you open this as a separate issue.

I should be able to push a fix for this one tomorrow.

On Mon, 21 Dec 2020 at 10:48, Damodar Lohani notifications@github.com wrote:

@bsutton https://github.com/bsutton For me as i described and you can see in the output of dcli doctor the path for pub and dart2native are not in the path assumed by the dcli. And for me the problem is not during the dcli install its with dcli compile where it's unable to find the pub and pubGetException is raised as below

Setting flag: verbose DCli Version: 0.34.6 Found command Instance of 'CompileCommand' Setting flag: install

Compiling ./pcreate.dart...

exists: ./pcreate.dart followLinks: true env: SHELL:/usr/bin/zsh env: SHELL:/usr/bin/zsh env: SUDO_USER:null path: //dart/scripts/pcreate.dart exists: //dart/scripts/pubspec.yaml followLinks: true lockcount = 0 Hardlock taken exists: //dart/scripts followLinks: true find: pwd: /home/dlohani/Documents/dart/scripts root: //dart/scripts pattern: *.script.lock caseSensitive: false recursive: false types: [file] found [] lock files Taking lock .486796.2833298015332279.script.lock for isolates/2833298015332279 Lock Source: named_lock.dart : NamedLock.withLock : 104 touch: //dart/scripts/.486796.2833298015332279.script.lock create: true exists: //dart/scripts followLinks: true Hardlock released Waiting for warmup to complete... Incremented lock: 1 lockcount = 1 Incremented lock: 2 env: SUDO_USER:null env: PATH:/usr/local/go/bin:/opt/flutter/bin:/opt/flutter/.pub-cache/bin:/opt/flutter/bin/cache/dart-sdk/bin:/home/dlohani/.pub-cache/bin:/home/dlohani/.dcli/bin exists: /usr/local/go/bin/dart followLinks: true exists: /opt/flutter/bin/dart followLinks: true Process.start: cmdLine /opt/flutter/bin/pub get --no-precompile Process.start: runInShell: false workingDir: //dart/scripts mode: normal cmd: /opt/flutter/bin/pub args: get, --no-precompile exists: /home/dlohani/Documents/dart/scripts followLinks: true pub get exeception: /opt/flutter/bin/pub get --no-precompile exit: 2 reason: Could not find /opt/flutter/bin/pub on the path. Decremented lock: 1 Decremented lock: 0 Releasing lock: //dart/scripts/.486796.2833298015332279.script.lock Hardlock taken delete: //dart/scripts/.486796.2833298015332279.script.lock ask: false exists: //dart/scripts/.486796.2833298015332279.script.lock followLinks: true Hardlock released DCliException: Instance of 'PubGetException' Stacktrace: named_lock.dart : NamedLock.withLock. : 127 zone.dart : runZonedGuarded : 1620 zone.dart : runZoned : 1547 named_lock.dart : NamedLock.withLock : 104 dart_project.dart : DartProject.warmup : 126 compile.dart : CompileCommand.compileScript : 108 compile.dart : CompileCommand.run : 65 command_line_runner.dart : CommandLineRunner.process : 86 entry_point.dart : EntryPoint._parseCmdLine : 40 entry_point.dart : EntryPoint.process : 34

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bsutton/dcli/issues/120#issuecomment-748689533, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32ODLOWTH7TM3UGWQ4ULSV2ELJANCNFSM4VCNWXQQ .

lohanidamodar commented 3 years ago

@bsutton i have created #121

bsutton commented 3 years ago

@RandalSchwartz I'm in the process of releasing 0.35 which now checks for dart compile as uses that rather than dart2native.

I'm still rather perplexed by your issue:

dcli_install fails by presuming pub is in PATH.

your dcli doctor shows that pub is on the path:

pub path //Flutter/bin/pub which: //bin/pub

However it is in an unusual spot.

Can you provide more details and ideally a trace of dcli install:

dcli_install -v

Of have I mis-understood and this is just the dart2native problem?

bsutton commented 3 years ago

I've just published 0.37 which I now believe fixes the pub path issue.

I'm now using the path returned by which rather than assuming that pub is in the sdk directory.

RandalSchwartz commented 3 years ago

Nice! Keep in mind there are two pubs because there's "flutter pub" and "dart pub". And they install into different .pub-cache dirs. Madness, I tell you.

RandalSchwartz commented 3 years ago

Yup. Failed out of the box. Again you're assuming pub is in PATH. That... is... going... away! Just like dart2native going away. You need to use "dart pub" for this. The only things you can count on is "dart", and "flutter" if flutter has been installed.

RandalSchwartz commented 3 years ago

And can't do dcli_install:

Locohost.local:~/bin % dcli_install
Hang on a tick whilst we install DCli 0.37.0

RunException: /usr/lib/dart/bin/pub global activate dcli 
exit: 2
reason: Could not find /usr/lib/dart/bin/pub on the path. 
Stacktrace: wait_for_ex.dart : waitForEx : 39
runnable_process.dart : RunnableProcess._waitForStart : 254
runnable_process.dart : RunnableProcess.start : 230
runnable_process.dart : RunnableProcess.run : 146
run.dart : start : 210
string_as_process.dart : StringAsProcess.run : 67
macosx_installer.dart : MacOsxDCliInstaller.install : 17
posix_mixin.dart : PosixMixin.install : 81
install.dart : InstallCommand.run : 105
command_line_runner.dart : CommandLineRunner.process : 86
bsutton commented 3 years ago

I'm not actually see the pub path issue and I'm running the beta.

No matter, I wasn't aware that pub was moving into dart.

Give me a day or so and I will update that as well.

Thanks for raising the issue.

bsutton commented 3 years ago

@RandalSchwartz I've just pushed 0.38.0 which has fixes for the pub issue.

As I'm writing this the unit tests are still running so I've pushed it a little prematurely but enough have passed that I think it should be fine.

Give it a go and let me know how things go.

RandalSchwartz commented 3 years ago

Still fails on dcli_install, presuming dart is in...

Locohost.local:~ % dcli_install
Hang on a tick whilst we install DCli 0.39.2

RunException: /usr/lib/dart/bin/dart pub global activate dcli 
exit: 2
reason: Could not find /usr/lib/dart/bin/dart on the path. 
Stacktrace: wait_for_ex.dart : waitForEx : 39
runnable_process.dart : RunnableProcess._waitForStart : 254
runnable_process.dart : RunnableProcess.start : 230
runnable_process.dart : RunnableProcess.run : 146
run.dart : start : 210
string_as_process.dart : StringAsProcess.run : 67
macosx_installer.dart : MacOsxDCliInstaller.install : 17
posix_mixin.dart : PosixMixin.install : 81
install.dart : InstallCommand.run : 105
command_line_runner.dart : CommandLineRunner.process : 86
Locohost.local:~ %

Where does it ever get /usr/lib/dart from?

bsutton commented 3 years ago

where is dart located on your system?

On Thu, 24 Dec 2020 at 11:44, Randal L. Schwartz notifications@github.com wrote:

Still fails on dcli_install, presuming pub is in...

Locohost.local:~ % dcli_install Hang on a tick whilst we install DCli 0.39.2

RunException: /usr/lib/dart/bin/dart pub global activate dcli exit: 2 reason: Could not find /usr/lib/dart/bin/dart on the path. Stacktrace: wait_for_ex.dart : waitForEx : 39 runnable_process.dart : RunnableProcess._waitForStart : 254 runnable_process.dart : RunnableProcess.start : 230 runnable_process.dart : RunnableProcess.run : 146 run.dart : start : 210 string_as_process.dart : StringAsProcess.run : 67 macosx_installer.dart : MacOsxDCliInstaller.install : 17 posix_mixin.dart : PosixMixin.install : 81 install.dart : InstallCommand.run : 105 command_line_runner.dart : CommandLineRunner.process : 86 Locohost.local:~ %

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/bsutton/dcli/issues/120#issuecomment-750654799, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OGILUDLW42SF7RIUNTSWKFHPANCNFSM4VCNWXQQ .

bsutton commented 3 years ago

Sorry silly question: //Flutter/bin/dart

bsutton commented 3 years ago

I've just pushed 0.39.3 which I think will fix this.

After xmas I will try and find to to spool up a mac. but in the mean time if you want to give this ago I think there is a reasonable chance it will work.

lohanidamodar commented 3 years ago

@bsutton If i remove pub from path, even in 0.39.3, dcli install fails.

bsutton commented 3 years ago

Dcli expects pub to exist if you are running dart versions earlier than 2.10

On Thu, 24 Dec 2020, 10:48 pm Damodar Lohani, notifications@github.com wrote:

@bsutton https://github.com/bsutton If i remove pub from path, even in 0.39.3, dcli install fails.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bsutton/dcli/issues/120#issuecomment-750859895, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OHRBXSSNF23FWNTLNLSWMTABANCNFSM4VCNWXQQ .

lohanidamodar commented 3 years ago

I am running dart 2.10, here

dart --version
Dart SDK version: 2.10.4 (stable) (Wed Nov 11 13:35:58 2020 +0100) on "linux_x64"
bsutton commented 3 years ago

Can you post a copy of the error stack and will have a look at it.

On Fri, 25 Dec 2020, 9:58 am Damodar Lohani, notifications@github.com wrote:

I am running dart 2.10, here

dart --version Dart SDK version: 2.10.4 (stable) (Wed Nov 11 13:35:58 2020 +0100) on "linux_x64"

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bsutton/dcli/issues/120#issuecomment-751127501, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32ODCNSDBQ4WY5ZPS5WLSWPBRRANCNFSM4VCNWXQQ .

lohanidamodar commented 3 years ago

@bsutton I simply get

/opt/flutter/bin/dcli: line 7: pub: command not found
bsutton commented 3 years ago

That is a different issue.

It means you are trying to run a dart script when dart isn't in the path.

This isn't actually a dcli issue.

Your dart install is a little pickled.

On Fri, 25 Dec 2020, 12:44 pm Damodar Lohani, notifications@github.com wrote:

@bsutton https://github.com/bsutton I simply get

/opt/flutter/bin/dcli: line 7: pub: command not found

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bsutton/dcli/issues/120#issuecomment-751144033, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OBVIUABX2YNXK6I5P3SWPVAJANCNFSM4VCNWXQQ .

bsutton commented 3 years ago

Actually you said you deleted pub. The pub global activate command still requires pub to be in the path.

This is still a dart requirement not a dcli issue.

On Fri, 25 Dec 2020, 12:44 pm Damodar Lohani, notifications@github.com wrote:

@bsutton https://github.com/bsutton I simply get

/opt/flutter/bin/dcli: line 7: pub: command not found

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bsutton/dcli/issues/120#issuecomment-751144033, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OBVIUABX2YNXK6I5P3SWPVAJANCNFSM4VCNWXQQ .

lohanidamodar commented 3 years ago

The error is pub not found, dart is still in path, and for pub global activate , I use dart pub global activate

bsutton commented 3 years ago

It doesn't matter which way you do it.

The problem is that the global activate command creates a bash script which still expects pub to be in the path.

It is that script which is causing the error.

Check line 7 of the dcli script in pub cache bin.

This is generated by global activate.

On Fri, 25 Dec 2020, 6:03 pm Damodar Lohani, notifications@github.com wrote:

The error is pub not found, dart is still in path, and for pub global activate , I use dart pub global activate

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bsutton/dcli/issues/120#issuecomment-751191837, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG32OAPLGYCSVSHACRE37TSWQ2MJANCNFSM4VCNWXQQ .