search

jupyter-incubator / sparkmagic

Jupyter magics and kernels for working with remote Spark clusters
Other
1.33k stars 447 forks source link

How to capture various events like livy session created, etc #582

Open devender-yadav opened 5 years ago

devender-yadav commented 5 years ago

Is there any way to capture sparkmagic or livy session specific events from the notebook itself?

itamarst commented 4 years ago

Can you explain a bit more what you're trying to achieve? What events would you expect to be able to capture, and what would you do with them?

devender-yadav commented 4 years ago

Hi @itamarst,

I want to see the usage of notebooks (with sparkmagic kernel). So that I know how many livy sessions are created, etc.

I can do it on spark level as livy sessions correspond to spark applications. I was just checking if sparkmagic is emitting any events like livy session started, stopped, etc.