search

jupyterhub / mybinder.org-deploy

Deployment config files for mybinder.org
https://mybinder-sre.readthedocs.io/en/latest/index.html
BSD 3-Clause "New" or "Revised" License
76 stars 74 forks source link

Crypto miner xmrig running on GESIS node #2623

Closed rgaiacs closed 1 year ago

rgaiacs commented 1 year ago

List of processes showes xmrig:

$ ps -ef | grep xmrig
adminis+ 3854693 3854513 99 15:33 pts/0    02:26:23 ./xmrig -ostratum+tcp://gulf.moneroocean.stream:80 -a rx -k -u 42VsRJPATEujKXhQATp8D9GNJLBNZCQBh7v92wK9Lz3AFSdkipZMTFWCkwisiNAmhWZ4LN85FRirzbo5HtcKE5qwNTTzEM6.aa1

The parent process is python:

$ ps aux | grep 3854513
adminis+ 3854513  2.3  0.0 771172 63240 ?        Ssl  15:32   0:30 /srv/conda/envs/notebook/bin/python -m ipykernel_launcher -f /home/jovyan/.local/share/jupyter/runtime/kernel-93042ae6-41c1-495a-87f9-9c006b92bfcd.json

The hostname points to a pod using BinderHub git repository.

$ sudo nsenter -t 3854693 -u hostname
jupyter-jupyterhub-2dbinderhub-2d0v3uud9a

that is confirmed by

$ kubectl describe pod -n gesis jupyter-jupyterhub-2dbinderhub-2d0v3uud9a
Name:                 jupyter-jupyterhub-2dbinderhub-2d0v3uud9a
Namespace:            gesis
Priority:             0
Priority Class Name:  binderhub-default-priority
Service Account:      default
Node:                 REDACTED
Start Time:           Fri, 05 May 2023 15:31:42 +0200
Labels:               app=jupyterhub
                      chart=jupyterhub-2.0.0
                      component=singleuser-server
                      heritage=jupyterhub
                      hub.jupyter.org/network-access-hub=true
                      hub.jupyter.org/servername=
                      hub.jupyter.org/username=jupyterhub-2dbinderhub-2d0v3uud9a
                      release=binderhub
Annotations:          hub.jupyter.org/username: jupyterhub-binderhub-0v3uud9a
Status:               Running
IP:                   10.244.3.203
IPs:
  IP:  10.244.3.203
Init Containers:
  block-cloud-metadata:
    Container ID:  containerd://85328cd5c96d0e8b8d38b1fa11cb68ef113a3c10768bb56973a475585f4e7ea7
    Image:         jupyterhub/k8s-network-tools:2.0.0
    Image ID:      docker.io/jupyterhub/k8s-network-tools@sha256:ab4172a025721495c0c65bd2a6165a6cd625bae39e0e5231c06e149c2ffc5dab
    Port:          <none>
    Host Port:     <none>
    Command:
      iptables
      -A
      OUTPUT
      -d
      169.254.169.254
      -j
      DROP
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 05 May 2023 15:31:43 +0200
      Finished:     Fri, 05 May 2023 15:31:43 +0200
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:         <none>
Containers:
  notebook:
    Container ID:  containerd://e17c240d5ba1ec7e1d02eb4464ff693c04b32f510dff093d385d9ece5dff7f8c
    Image:         gesiscss/binder-r2d-g5b5b759-jupyterhub-2dbinderhub-b26558:d6dd45ac12218c6a6912ce6375cad41cf3042c15
    Image ID:      docker.io/gesiscss/binder-r2d-g5b5b759-jupyterhub-2dbinderhub-b26558@sha256:72bebf079dad90db11d85d493f2c5a8c3ccda0053b8c9c0028f33afa57142ac7
    Port:          8888/TCP
    Host Port:     0/TCP
    Args:
      python3
      -c
      import os
      import sys

      try:
          import jupyterlab
          import jupyterlab.labapp
          major = int(jupyterlab.__version__.split(".", 1)[0])
      except Exception as e:
          print("Failed to import jupyterlab: {e}", file=sys.stderr)
          have_lab = False
      else:
          have_lab = major >= 3

      if have_lab:
        # technically, we could accept another jupyter-server-based frontend
        print("Launching jupyter-lab", file=sys.stderr)
        exe = "jupyter-lab"
      else:
        print("jupyter-lab not found, launching jupyter-notebook", file=sys.stderr)
        exe = "jupyter-notebook"

      # launch the notebook server
      os.execvp(exe, sys.argv)

      --ip=0.0.0.0
      --port=8888
      --NotebookApp.base_url=/binder/jupyter/user/jupyterhub-binderhub-0v3uud9a/
      --NotebookApp.token=REDACTED
      --NotebookApp.trust_xheaders=True
      --NotebookApp.allow_origin=*
      --NotebookApp.allow_origin_pat=.*
      --ServerApp.base_url=/binder/jupyter/user/jupyterhub-binderhub-0v3uud9a/
      --ServerApp.token=REDACTED
      --ServerApp.trust_xheaders=True
      --ServerApp.allow_origin=*
      --ServerApp.allow_origin_pat=.*
    State:          Running
      Started:      Fri, 05 May 2023 15:31:55 +0200
    Ready:          True
    Restart Count:  0
    Requests:
      memory:  1073741824
    Environment:
      BINDER_LAUNCH_HOST:                      https://mybinder.org/
      BINDER_PERSISTENT_REQUEST:               v2/gh/jupyterhub/binderhub/d6dd45ac12218c6a6912ce6375cad41cf3042c15
      BINDER_REF_URL:                          https://github.com/jupyterhub/binderhub/tree/d6dd45ac12218c6a6912ce6375cad41cf3042c15
      BINDER_REPO_URL:                         https://github.com/jupyterhub/binderhub
      BINDER_REQUEST:                          v2/gh/jupyterhub/binderhub/HEAD
      JPY_API_TOKEN:                           REDACTED
      JUPYTERHUB_ACTIVITY_URL:                 http://hub:8081/binder/jupyter/hub/api/users/jupyterhub-binderhub-0v3uud9a/activity
      JUPYTERHUB_ADMIN_ACCESS:                 1
      JUPYTERHUB_API_TOKEN:                    REDACTED
      JUPYTERHUB_API_URL:                      http://hub:8081/binder/jupyter/hub/api
      JUPYTERHUB_BASE_URL:                     /binder/jupyter/
      JUPYTERHUB_CLIENT_ID:                    jupyterhub-user-jupyterhub-binderhub-0v3uud9a
      JUPYTERHUB_HOST:                         
      JUPYTERHUB_OAUTH_ACCESS_SCOPES:          ["access:servers!server=jupyterhub-binderhub-0v3uud9a/", "access:servers!user=jupyterhub-binderhub-0v3uud9a"]
      JUPYTERHUB_OAUTH_CALLBACK_URL:           /binder/jupyter/user/jupyterhub-binderhub-0v3uud9a/oauth_callback
      JUPYTERHUB_OAUTH_CLIENT_ALLOWED_SCOPES:  []
      JUPYTERHUB_OAUTH_SCOPES:                 ["access:servers!server=jupyterhub-binderhub-0v3uud9a/", "access:servers!user=jupyterhub-binderhub-0v3uud9a"]
      JUPYTERHUB_SERVER_NAME:                  
      JUPYTERHUB_SERVICE_PREFIX:               /binder/jupyter/user/jupyterhub-binderhub-0v3uud9a/
      JUPYTERHUB_SERVICE_URL:                  http://0.0.0.0:8888/binder/jupyter/user/jupyterhub-binderhub-0v3uud9a/
      JUPYTERHUB_USER:                         jupyterhub-binderhub-0v3uud9a
      JUPYTER_IMAGE:                           gesiscss/binder-r2d-g5b5b759-jupyterhub-2dbinderhub-b26558:d6dd45ac12218c6a6912ce6375cad41cf3042c15
      JUPYTER_IMAGE_SPEC:                      gesiscss/binder-r2d-g5b5b759-jupyterhub-2dbinderhub-b26558:d6dd45ac12218c6a6912ce6375cad41cf3042c15
      MEM_GUARANTEE:                           1073741824
    Mounts:
      /etc/jupyter from etc-jupyter (rw)
      /etc/jupyter/templates from etc-jupyter-templates (rw)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  etc-jupyter:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      user-etc-jupyter
    Optional:  false
  etc-jupyter-templates:
    Type:        ConfigMap (a volume populated by a ConfigMap)
    Name:        user-etc-jupyter-templates
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  binderhub=true
Tolerations:     hub.jupyter.org/dedicated=user:NoSchedule
                 hub.jupyter.org_dedicated=user:NoSchedule
                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From                      Message
  ----    ------     ----  ----                      -------
  Normal  Scheduled  30m   binderhub-user-scheduler  Successfully assigned gesis/jupyter-jupyterhub-2dbinderhub-2d0v3uud9a to spko-css-app03
  Normal  Pulled     30m   kubelet                   Container image "jupyterhub/k8s-network-tools:2.0.0" already present on machine
  Normal  Created    30m   kubelet                   Created container block-cloud-metadata
  Normal  Started    30m   kubelet                   Started container block-cloud-metadata
  Normal  Pulling    30m   kubelet                   Pulling image "gesiscss/binder-r2d-g5b5b759-jupyterhub-2dbinderhub-b26558:d6dd45ac12218c6a6912ce6375cad41cf3042c15"
  Normal  Pulled     30m   kubelet                   Successfully pulled image "gesiscss/binder-r2d-g5b5b759-jupyterhub-2dbinderhub-b26558:d6dd45ac12218c6a6912ce6375cad41cf3042c15" in 951.027521ms (10.987990196s including waiting)
  Normal  Created    30m   kubelet                   Created container notebook
  Normal  Started    30m   kubelet                   Started container notebook

https://github.com/yuvipanda/cryptnono/ doesn't show any activity.

Can I have some help to have cryptnono to kill this crypt miners? cc @yuvipanda

yuvipanda commented 1 year ago

@rgaiacs can you ssh into the node and run the detector script manually (https://github.com/pixie-io/pixie-demos/tree/main/detect-monero-demo#bpftrace-cli) to see what it says? What kernel / distro are you running?

rgaiacs commented 1 year ago

Thanks for the reply. I will run it next time that I see the miner.

rgaiacs commented 1 year ago

@yuvipanda when I run sudo bpftrace detectrandomx.bt, I get the following error:

/lib/modules/5.15.0-69-generic/build/arch/x86/include/asm/fpu/types.h:341:15: error: use of undeclared identifier 'PAGE_SIZE'
rgaiacs commented 1 year ago

Thanks for the help! It is fixed now.