Integration of Results with Jupyter Notebooks #221

Closed stv0g closed 1 year ago

In GitLab by @skolen on Nov 23, 2020, 15:00

We need a new feature that allows to download and post-process Results from VILLASweb in Jupyter Notebooks. A first solution: Auto-create a Python code snippet based on the requested results that downloads the data and prepares it for example as numpy array for further use in Jupyter. For the download, the VILLASweb API + authentication token is required. A Jupyter user can copy and execute the code snippet in his/her notebook and start post-processing the data.

In a more advanced solution we could think of directly opening a Jupyter notebook from VILLASweb.

CC @jan.dinkelbach

Generating Jupyter Notebooks is fairly easy. Here is the content of an example notebook file (with ending .ipynb). The notebooks file simply contain JSON like the following:

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('here goes our python code')\n",
    "print('and a second line')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}

So we simply need to generate a file and insert the Python code which Sonja mentioned at .cells[0].source[] (JSONpath).

I imagine the Python snippet for the user to look like the following:

import pandas
import Results from villas.web.results

results = Results(
    124523523,                         # Project ID
    1,                                 # Run/Result ID
    'neich9IeX6Taif4Koh1oyohraiph9baK' # Token
)

with results.open('sim_01.csv') as f:
    df = pandas.load_csv(f)

print(df)

All the magic happens inside the villas.web.results.Results.open() function:

Returns a Python file-like object
Downloads the ZIP in the background
Extracts single file from ZIP and provide file-like object

I just released the villas-dataoprocessing v0.3 package on pypi.org: https://pypi.org/project/villas-dataprocessing/

It includes the required Python code for fetching results from VILLASweb. This code is also tested.

Changes are also merged into master:

In GitLab by @skolen on Jun 17, 2021, 10:55

Completed with !89.

VILLASframework / web

Integration of Results with Jupyter Notebooks #221