aws-samples / sagemaker-ssh-helper

A helper library to connect into Amazon SageMaker with AWS Systems Manager and SSH (Secure Shell)
MIT No Attribution
222 stars 33 forks source link

SSH-ing into Sagemaker studio (new) apps #67

Open mario-at-intercom opened 2 months ago

mario-at-intercom commented 2 months ago

Hello,

I've been trying to use this to SSH into SageMaker studio apps, but the setup fails. Main issues are hitting various permission errors that sagemaker-user doesn't hold.

I managed to get around it by installing as root, which brings a problem of root user not having AWS credentials exposed (possible to pass them from environment variables though).

Local scripts don't support it either, but that's something we can fix too and I managed to go around it.

mario-at-intercom commented 2 months ago

For transparency, these are hacky instructions I assembled this morning to make it work. Ideally it'd be just one script call

Running as a sagemaker-user (probably should figure out a better way)
- chown -R $(whoami):users ~ (to give root ability to see /home/sagemaker-users)
- export | grep AWS
    - copy the output of this

Now open a sudo shell with "sudo su"
- paste the output of AWS credentials so sudo can work
- go to /home/sagemaker-user/sm-ssh-helper
- python -m ensurepip
- python -m pip install -e .
- sm-ssh-ide configure --ssh-only
- LOCAL_USER_ID="{org_id}:mario" 
- sm-ssh-ide set-local-user-id "$LOCAL_USER_ID"
- sm-ssh-ide init-ssm
- sm-ssh-ide stop
- sm-ssh-ide start
- cat /var/log/amazon/ssm/amazon-ssm-agent.log  | grep "Managed instance-id" (this will show you instance id, starts with mi-xxxx)
- sm-ssh-ide ssm-agent (keep this terminal opened)

Now on your laptop run

INSTANCE_ID="mi-xxxx"
sm-local-start-ssh "$INSTANCE_ID" \
        -L localhost:10022:localhost:22 \
        $*

For more ssh options look into sm-local-ssh-ide script.
mariokostelac commented 2 months ago

Oh I see there's https://github.com/aws-samples/sagemaker-ssh-helper/issues/51.

andreasbolstad commented 1 month ago

How did you get this to work? sm-ssh-ide init-ssm doesn't work for me, as there is no UserProfileName in the /opt/ml/metadata/resource-metadata.json for the new studio (both code editor and jupyter).

mariokostelac commented 1 month ago

Have you followed the instructions above?