Feature request - Jupyter for Juju

[aviditas-security]

Background:

Post-killchain activities, IR and Insider Threat would like to have some form of Jupyter notebooks for our case management tool. JupyterHub for the internal project obsidian workstations to use would be the ideal solution. https://jupyter.org/hub While the need is not until after the KC activities are complete, the sooner we have this setup, the sooner we can knock out the work for Defcon, and the less work is needed to revise the analysis for realism.

This request is for internal to Project Obsidian only. Would have a larger conversation about Defcon and workshops if this is successful.

Request for us to build

IR and InsiderThreat would like to have the ability to make and use Jupyter notebooks to interact with the SIEMs. Preferably JupyterHub to enable multiple user int

CRITERIA FOR SUCCESS

Jupyter notebooks that can run queries against the SIEMs

DRIs

juju43#9004 a.skye (aviditas)#6832

[juju4]

If local, no issue, easy to setup Jupyter notebook If inside the case management of security onion, I don't know as not much familiar with current evolution of SO. If server instance of jupyter hub, need to review. the little jupyterhub https://tljh.jupyter.org/en/latest/ seems the way to go. Quick search and found an ansible role doing setup and adding github SSO https://galaxy.ansible.com/cyverse-ansible/ansible_jupyterhub_docker - testing needed.

On my side, and like last year, I would use and recommend msticpy but few limitations

• splunk connection did not work likely because free splunk

  # if free splunk, 
  #  * enable the 'allowRemoteLogin' setting in your server.conf file.
  #  * need to patch splunk_driver.py and remove password from _SPLUNK_REQD_ARGS.
  # you must keep username as 'admin' in msticpyconfig.yaml
  # FIXME! HTTPError: HTTP 400 Bad Request -- Missing or malformed messages.conf stanza for SEARCHFACTORY:UNKNOWN_OP__index
  # works in UI, maybe a free splunk limitation

  from https://github.com/blueteamvillage/obsidian-ir/blob/main/ooda_loop/obsidian-kc3-ir-notebook.ipynb

• elastic support for msticpy is in the work and not sure how usable it is https://github.com/microsoft/msticpy/search?q=elastic&type=code
• some direct options per https://marcusedmondson.com/2020/08/14/threat-hunting-with-jupyter-notebooks-part-1-connect-to-elasticsearch/ https://github.com/walterra/jupyter2kibana

But outside of jupyter/pandas/msticpy, what other modules would the team want?

[zveroboy152]

Hi @aviditas-security ,

We wanted to follow up on this request. Can you comment back to Juju to see if that is what you need?

[juju4]

https://github.com/blueteamvillage/DC31-obsidian-sec-eng/pull/149

[juju4]

Please check PR Currently working with local authentication (user ubuntu + creds spreadsheet) msticpy+splunk working this time outside of some output rendering that seems slightly different than web UI

still need to confirm what python modules would want depending on activities to be done.

[juju4]

jupyterhub hardening mostly at systemd level per /lib/systemd/system/jupyterhub.service. Network side

# This will prevent any new pip module install or public TI enrichment
IPAddressAllow=localhost link-local multicast 10.0.0.0/8 192.168.0.0/16

From today seceng meeting, to review possible extras

• aws nsg on outbound
• web proxy

[juju4]

Ben to review with team leads if still want to use jupyterhub this year. Else to keep for next one.

[juju4]

May 13th status is still current @CptOfEvilMinions