geigr / ipytone

Interactive audio in Jupyter
https://ipytone.readthedocs.io
BSD 3-Clause "New" or "Revised" License
60 stars 8 forks source link

altair integration #74

Open mattijn opened 2 years ago

mattijn commented 2 years ago

I saw some awesome videos at your twitter-announcement. Congrats on this package.

I would love to hear the simplification of spatial data using the package topojson and ipywidgets, see animated gif here https://mattijn.github.io/topojson/example/output-types.html#to_widget.

I've no idea what it takes;)

benbovy commented 2 years ago

Thanks!

I'm not sure how would work altair integration as I don't think that the altair figure elements are Jupyter widgets (https://github.com/altair-viz/altair/issues/290).

That said, the example that you mention seems to use basic widgets (sliders) to control the simplification, so if you have access to those widgets it would be straightforward to link them to any ipytone widget. Unfortunately, it seems that topojson doesn't allow accessing the underlying slider widgets as it directly returns an interact instance.

Assuming that you manage to get access to those sliders, I would probably try something like this (i.e., cut higher to lower frequencies of an oscillator as the geometry gets more simplified):

import ipytone

# the toposimplifly factor widget
eps = widgets.FloatSlider(...)

filtr = ipytone.Filter(frequency=10_000)
osc = ipytone.Oscillator(type="sawtooth")
osc.chain(filtr, ipytone.destination)

def filter_osc(change):
    # scale factor between the simplify factor value and the filter frequency
    factor = 1e3
    filtr.frequency.value = change["new"] * factor

eps.observe(filter_osc, names="value")
osc.start()

But there are many other possible ways to do it (e.g., using distortion, oscillator frequencies, etc.).

mattijn commented 2 years ago

Ah, good points. That is a great starting point. Will start from there. Once the XY works on canvas, one should be able to listen to country borders.

Fwiw, altair produces vegalite json, that compiles into vega, which renders as canvas in browser (or svg), https://github.com/vega/vega/tree/main/packages/vega-scenegraph

A chart object is accessible using the View API including the scenegraph and provide a way to render and interact. https://vega.github.io/vega/docs/api/view.

benbovy commented 2 years ago

Once the XY works on canvas, one should be able to listen to country borders.

If you want to somehow listen to the country borders, I think that you can already do that now without the need to do something specific on the JS (canvas) side.

I guess that you already have access to the border coordinates from the Python side via the topojson object. If so, you could for example use the set_value_curve_at_time or any other scheduling method of an ipytone Param, Signal or Envelope instance to automate the modulation of some audio parameter based on the country border coordinate values.

joelostblom commented 11 months ago

Just a note here that it is now possible to use the JupyterChart class to create Altair charts that can interact with jupyter/ipywidgets directly. See this section in the docs for details https://altair-viz.github.io/user_guide/jupyter_chart.html