search

compas-dev / compas_fab

Robotic fabrication package for the COMPAS Framework.
https://compas.dev/compas_fab/
MIT License
108 stars 32 forks source link

External file loading failed in Grasshopper after install #297

Closed tetov closed 3 years ago

tetov commented 3 years ago

Describe the bug

Grasshopper breaks after install of compas_fab, probably due to Grasshopper userobjects.

To reproduce

  1. Rhino 6 SR34, compas_fab==0.18.0, python==3.8.8, Win 10, empty Libraries and UserObjects folders.

  2. conda create -n repro compas_fab python==3.8.8 -y
conda activate repro
python -m compas_rhino.install

  3. Launch Rhino and Grasshopper. image

  4. image

  5. python -m compas_rhino.uninstall

  6. Error in post-uninstall steps

    
Uninstalling COMPAS packages from Rhino 6.0 scripts folder:
Location scripts folder: C:\Users\a\AppData\Roaming\McNeel\Rhinoceros\6.0\scripts

The following packages have been detected and will be uninstalled: compas_fab OK compas OK roslibpy OK compas_rhino OK compas_ghpython OK compas_bootstrapper OK

Running post-uninstallation steps...

One or more errors occurred:

Uninstall completed.


7. Launch Rhino and Grasshopper.
![image](https://user-images.githubusercontent.com/14882117/117283175-d269a180-ae65-11eb-901f-4212d67978f5.png)
8. See 4
9.  The `.ghuser` files remain in `%APPDATA%\Grasshopper\UserObjects` after install, but are interpreted as directories by explorer. (Location not available. Directory name invalid).
![image](https://user-images.githubusercontent.com/14882117/117283381-05ac3080-ae66-11eb-957c-43c694d631fa.png)
10. After deleting files in UserObjects Grasshopper starts normally.
gonzalocasas commented 3 years ago

oh, thanks for the report! will look into it!

gonzalocasas commented 3 years ago

I just tried your repro steps, and it worked for me. I tried the uninstall and got the same error in the post-steps (TypeError('lstat: path should be string, bytes or os.PathLike, not tuple')), but besides that, the components worked properly.

Could you please try again the python -m compas_rhino.install and post the output of that? Then, I would ask you to check if you can add more details about the symlinked ghuser files which you say point to a folder (strange! they should point to a file).

gonzalocasas commented 3 years ago

And a last bit: do you run any of these commands as admin?

tetov commented 3 years ago 
$ python -m compas_rhino.install
Installing COMPAS packages to Rhino 6.0 scripts folder:
Location scripts folder: C:\Users\a\AppData\Roaming\McNeel\Rhinoceros\6.0\scripts

   compas               OK
   compas_fab           OK
   compas_ghpython      OK
   compas_rhino         OK
   roslibpy             OK
   compas_bootstrapper  OK

Running post-installation steps...

   compas_fab           OK: Installed 11 GH User Objects

Completed.
tetov commented 3 years ago

And a last bit: do you run any of these commands as admin?

As user. Tried as admin but didn't notice any difference. I'll test that a bit more.

gonzalocasas commented 3 years ago

try dir /a in the AppData\Roaming\Grasshopper\UserObjects folder and then check what's the situation at the target of those links.

gonzalocasas commented 3 years ago

And a last bit: do you run any of these commands as admin? As user. Tried as admin but didn't notice any difference. I'll test that a bit more.

User is good. Admin is bound to cause problems.

tetov commented 3 years ago 
$ dir /a
 Volume in drive C is Windows
 Volume Serial Number is 5407-2AD5

 Directory of C:\Users\a\AppData\Roaming\Grasshopper\UserObjects

2021-05-06  13:04    <DIR>          .
2021-05-06  13:04    <DIR>          ..
2021-05-06  13:04    <SYMLINKD>     Cf_AttachedCollisionMesh.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_AttachedCollisionMesh.ghuser]
2021-05-06  13:04    <SYMLINKD>     Cf_CollisionMesh.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_CollisionMesh.ghuser]
2021-05-06  13:04    <SYMLINKD>     Cf_ConstraintsFromPlane.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_ConstraintsFromPlane.ghuser]
2021-05-06  13:04    <SYMLINKD>     Cf_InverseKinematics.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_InverseKinematics.ghuser]
2021-05-06  13:04    <SYMLINKD>     Cf_PlanCartesianMotion.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_PlanCartesianMotion.ghuser]
2021-05-06  13:04    <SYMLINKD>     Cf_PlanMotion.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_PlanMotion.ghuser]
2021-05-06  13:04    <SYMLINKD>     Cf_PlanningScene.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_PlanningScene.ghuser]
2021-05-06  13:04    <SYMLINKD>     Cf_RosConnect.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_RosConnect.ghuser]
2021-05-06  13:04    <SYMLINKD>     Cf_RosRobot.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_RosRobot.ghuser]
2021-05-06  13:04    <SYMLINKD>     Cf_VisualizeRobot.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_VisualizeRobot.ghuser]
2021-05-06  13:04    <SYMLINKD>     Cf_VisualizeTrajectory.ghuser [C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\lib\site-packages\compas_fab\ghpython\components\Cf_VisualizeTrajectory.ghuser]
               0 File(s)              0 bytes
              13 Dir(s)  71 076 491 264 bytes free

Files at symlink target are recognized as ghuser files.

I'll try it in a non scoop install just to make sure.

gonzalocasas commented 3 years ago

Are the files at target correct? Are they really there? Do they have any permissions that might limit access to them?

tetov commented 3 years ago

Powershell

$ Dir | Get-Acl

    Directory: C:\Users\a\scoop\apps\miniconda3\4.9.2\envs\biomimfab\Lib\site-packages\compas_fab\ghpython\components

Path                            Owner             Access                                                                                                               
----                            -----             ------                                                                                                               
Cf_AttachedCollisionMesh        UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_CollisionMesh                UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_ConstraintsFromPlane         UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_InverseKinematics            UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_PlanCartesianMotion          UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_PlanMotion                   UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_PlanningScene                UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_RosConnect                   UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_RosRobot                     UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_VisualizeRobot               UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_VisualizeTrajectory          UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
__pycache__                     UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_AttachedCollisionMesh.ghuser UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_CollisionMesh.ghuser         UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_ConstraintsFromPlane.ghuser  UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_InverseKinematics.ghuser     UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_PlanCartesianMotion.ghuser   UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_PlanMotion.ghuser            UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_PlanningScene.ghuser         UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_RosConnect.ghuser            UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_RosRobot.ghuser              UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_VisualizeRobot.ghuser        UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
Cf_VisualizeTrajectory.ghuser   UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
install.py                      UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...                                                                            
__init__.py                     UNIVERSAL-FRIEN\a NT AUTHORITY\SYSTEM Allow  FullControl...
tetov commented 3 years ago

And for those of us including me who don't understand Windows permissions: image

tetov commented 3 years ago 
$ Dir | Get-Acl

    Directory: C:\Users\a\AppData\Roaming\Grasshopper\UserObjects

Path                            Owner                  Access
----                            -----                  ------
Cf_AttachedCollisionMesh.ghuser BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
Cf_CollisionMesh.ghuser         BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
Cf_ConstraintsFromPlane.ghuser  BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
Cf_InverseKinematics.ghuser     BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
Cf_PlanCartesianMotion.ghuser   BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
Cf_PlanMotion.ghuser            BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
Cf_PlanningScene.ghuser         BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
Cf_RosConnect.ghuser            BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
Cf_RosRobot.ghuser              BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
Cf_VisualizeRobot.ghuser        BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
Cf_VisualizeTrajectory.ghuser   BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl...
tetov commented 3 years ago

Same issue using latest installer from anaconda.org.

gonzalocasas commented 3 years ago

There's a weird situation there with ownership, the files in site-packages are owned by your user, while the ones in UserObjects are owned by BUILTIN\Administrators. This might point to the fact that you've ran python -m compas_rhino.install as administrator, if that's the case, please remove these links first and run again without admin.

tetov commented 3 years ago

(Used net session to confirm that I was running as user)

(base) C:\Users\a>conda activate repro

(repro) C:\Users\a>net session
System error 5 has occurred.

Access is denied.

(repro) C:\Users\a>python -m compas_rhino.install
Installing COMPAS packages to Rhino 6.0 scripts folder:
Location scripts folder: C:\Users\a\AppData\Roaming\McNeel\Rhinoceros\6.0\scripts

   compas               OK
   compas_fab           OK
   compas_ghpython      OK
   compas_rhino         OK
   roslibpy             OK
   compas_bootstrapper  OK

Running post-installation steps...

   compas_fab           OK: Installed 11 GH User Objects

Completed.

And then in powershell

$ Dir | Get-Acl
Directory: C:\Users\a\AppData\Roaming\Grasshopper\UserObjects

Path                            Owner                  Access
----                            -----                  ------
Cf_AttachedCollisionMesh.ghuser BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
Cf_CollisionMesh.ghuser         BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
Cf_ConstraintsFromPlane.ghuser  BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
Cf_InverseKinematics.ghuser     BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
Cf_PlanCartesianMotion.ghuser   BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
Cf_PlanMotion.ghuser            BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
Cf_PlanningScene.ghuser         BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
Cf_RosConnect.ghuser            BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
Cf_RosRobot.ghuser              BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
Cf_VisualizeRobot.ghuser        BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
Cf_VisualizeTrajectory.ghuser   BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow  FullControl…
gonzalocasas commented 3 years ago

mmm... there are some code paths that in fact, lead to us doing privilege escalation during install to be able to create symlink (see here), but only as a fallback for the normal flow which uses more python-native methods. During install, are you asked to confirm the privilege escalation (the annoying UAC dialog on Windows)?

tetov commented 3 years ago

There are two UAC prompts (just enabled it again, haha!).. Symlinks will always require escalation, right?

gonzalocasas commented 3 years ago

Not always, but I cannot really recall all the details of the (various) possible code paths. I remember that at least on some configuration, symlink works on the "happy path" (prolly if you are the admin user of the machine).

Did you try to manually delete the symlinks and then run again python -m compas_rhino.install? Does it result in the same "admin" links? (I'm wondering because the uninstall is reporting issues.

tetov commented 3 years ago

Yes, same admin links.

I'm testing if creating them in CMD works better. 

set link_dir=C:\Users\a\AppData\Roaming\Grasshopper\UserObjects
set target_dir=C:\Users\a\scoop\apps\miniconda3\current\envs\biomimfab\Lib\site-packages\compas_fab\ghpython\components

set files=Cf_AttachedCollisionMesh.ghuser Cf_CollisionMesh.ghuser Cf_ConstraintsFromPlane.ghuser Cf_InverseKinematics.ghuser Cf_PlanCartesianMotion.ghuser Cf_PlanMotion.ghuser Cf_PlanningScene.ghuser Cf_RosConnect.ghuser Cf_VisualizeRobot.ghuser Cf_VisualizeTrajectory.ghuser

(for %%f in (%files%) do ( 
   mklink /H %link_dir%\%%f %target_dir%\%%f
))

Running this as admin creates hard symlinks pointing to files owned by my user account. (And works in Grasshopper)

gonzalocasas commented 3 years ago

Oh... Of course..i know what it is...I'll send a PR

tetov commented 3 years ago

This got me thinking about admin privs, and as was mentioned in yesterdays OS Arch meeting there are potential users without admin privs (due to company policies).

I hope to be teaching compas next term and if students are back on site they won't be able to run the rhino_install on the computers in the computer labs here..

We might need to work out non-admin ways to do this, even if it means manual intervention by the user...

Have this been discussed before?

tetov commented 3 years ago

@gonzalocasas @brgcode

This got me thinking about admin privs, and as was mentioned in yesterdays OS Arch meeting there are potential users without admin privs (due to company policies).

I hope to be teaching compas next term and if students are back on site they won't be able to run the rhino_install on the computers in the computer labs here..

We might need to work out non-admin ways to do this, even if it means manual intervention by the user...

Has this been discussed before?

tomvanmele commented 3 years ago

maybe ETH computers should have compas pre-installed :)

gonzalocasas commented 3 years ago

@brgcode not a bad idea, at least, for D-ARCH we should have Anaconda pre-installed as a minimum.

tomvanmele commented 3 years ago

but so i guess the problem with the institute's computers is that Rhino is installed on the roaming profile and the students have no access to that?

gonzalocasas commented 3 years ago

We might need to work out non-admin ways to do this, even if it means manual intervention by the user...

based on the experience with previous workshops and so on, I think currently we have very few cases in which compas cannot be installed due to permissions. I haven't tested myself what happens in a totally unprivileged user account thou.

tomvanmele commented 3 years ago

@gonzalocasas we have a had few occasions recently where Rhino was installed in a non-typical location because of limited user permissions. perhaps we could start by adding a few more search locations to the installation process?

gonzalocasas commented 3 years ago

good point, i'll create a follow up

tetov commented 3 years ago

The issue is not access to roaming profile, but admin reqs for making symlinks.

I could probably work with the IT dept here (LTH) to symlink packages to a central location..

I would assume however that a lot of (potential) users in industry don’t have admin privs on their work computers.

tomvanmele commented 3 years ago

do you need admin privileges for making symlinks on your local profile?

tetov commented 3 years ago

All symlinks except directory junctions.

Maybe there’s some workarounds: https://stackoverflow.com/questions/58038683/allow-mklink-for-a-non-admin-user#58048888

gonzalocasas commented 3 years ago

as a "3rd level fallback", we could copy the files instead of symlinking them (native -> polyfill -> copy), but if we can avoid it, I feel it would create less maintenance in the future.

gonzalocasas commented 3 years ago

Maybe there’s some workarounds

the "Development mode" is a good option! Maybe we need to test it and document it and we're done.

tetov commented 3 years ago

Hopefully it can be enabled by non-admins. I'll have a look.