Make capabilities consume parameters

[kthakore]

• Accelerometer => *.csv
• Audio => *.wav (mono) / bufferstream f32[]
• Image => *.png/jpg / bufferstream f32[]

Need to make these configurable

[Michael-F-Bryan]

I think this ticket comes in two pieces.

The first piece is internal and involves adding a mechanism that transforms Runefile arguments into method calls on the generated code.

For example, this line

CAPABILITY<I16[24000]> audio SOUND --hz 16000 --samples 150 --sample-size-ms 1500

Would need to be converted into a bunch of calls which tell the host to configure the SOUND capability so that we get 1500ms worth of samples taken at 16kHz. This would be a once-off step which gets run from _manifest().

---

The second part will be identifying the various knobs and levers that we want to expose and wiring them into the runtime. At the moment the Rust runtime is very bare-bones and doesn't let you configure anything.

I'm not 100% sure what these would be, but I'm guessing @meelislootus and @kthakore will have a list.

[Michael-F-Bryan]

With #72 things are set up so you can set a capability object's properties. Now we just need to figure out which properties to implement and their semantics for the PC and mobile runtimes.

@meelislootus, @Mi1ind, and @kthakore I'm completely spitballing here, so feel free to post suggestions or alterations.

Random

Fill a buffer with random data.

(nothing needed, number of bytes can be inferred from the buffer's size)

Accelerometer

A capability which gives you an array of [f32; 3] samples containing the acceleration in the X/Y/Z direction.

General:

• n | samples - the number of samples we want (could also be inferred from the destination buffer size?)

Phone:

• sample_rate - How frequently to sample the accelerometer
• duration - How long we should be taking samples for (note: samples = duration * sample_rate so this could be unnecessary)

Image

Fill a buffer with a single image.

General:

• grayscale - convert the image's 3 RGB channels into one
• width and height - ask the capability to resize the image to the desired dimensions

Video

(note: I've split this out from the image capability)

A capability which fills the provided buffer with a series of images (e.g. u8[num_samples x height x width x channels]).

General:

• samples - the number of images to use
• (all the other Image properties)

Phone:

• sample_frequency - when recording video in real time

Sound

Fill a buffer with PCM formatted audio samples.

General:

• samples - the number of samples (could be inferred from the buffer size?)

Phone:

• hz | frequency - the frequency to record audio at
• sample_duration | sample_duration_ms - how long to record samples for

[kthakore]

Another would be get GPIO from a device input pin.

Some other phone sensors are here:

https://github.com/SensingKit/SensingKit-iOS

[kthakore]

@meelislootus @Ge-te the mobile runtime may not be using the parameters for the capabilities to massage in the inputs for the Rune.