WebAudio / web-audio-api-v2

The Web Audio API v2.0, developed by the W3C Audio WG
Other
120 stars 11 forks source link

Define TextEncoderStream(), and TextDecoderStream() in AudioWorkletGlobalScope #94

Closed guest271314 closed 4 years ago

guest271314 commented 4 years ago

Describe the feature Define TextEncoderStream(), and TextDecoderStream() (and TextEncoder and TextDecoder) in AudioWorkletGlobalScope.

Is there a prototype?

TextEncoder, TextDecoder, TextEncoderStream(), and TextDecoderStream() are defined in Window and Worker scope.

Describe the feature in more detail

Consider a (potentially infinite) raw PCM stream source from fetch() to set as outputs within AudioWorletProcessor.process(). We need to convert text to a TypedArray which can be achieved with

const controller;
const readable = new ReadableStream({
  start(_) {
    return controller = _;
  }
});

readable
.pipeThrough(new TextEncoderStream())
.pipeTo(new WritableStream({
  write(value) {
    console.log(value);
  },
  close() {
    console.log('stream closed');
  }
}));
port.onMessage.addListener(async message => {
  if (message !== 'done') {
    controller.enqueue(message);
  }
  else {
    controller.close();
  }
});

at Window, however throws an error in AudioWorkletGlovbalScope

constructor(options) {
    super(options);
    this.byteSize = 512 * 344 * 60 * 50;
    this.memory = new Uint8Array(this.byteSize);
    Object.assign(this, options.processorOptions);
    console.log(this);
    this.port.onmessage = this.appendBuffers.bind(this);
  }
  async appendBuffers({ data: readable }) {
    Object.assign(this, { readable });
    const source = {
      write: (value, c) => {
        if (this.totalBytes < this.byteSize) {
          this.memory.set(value, this.readOffset);
          this.readOffset = this.readOffset + value.buffer.byteLength;
          this.totalBytes = this.readOffset;
        } else {
          console.log(
            value.buffer.byteLength,
            this.readOffset,
            this.totalBytes
          );
        }
        if (this.totalBytes >= (344 * 512) / 2 && !this.started) {
          this.started = true;
          this.port.postMessage({ started: this.started });
        }
      },
      close() {
        console.log('stopped');
      },
    };
    try {
      // throws error
      await this.readable
        .pipeThrough(new TextEncoderStream())
        .pipeTo(new WritableStream(source));
    } catch (e) {
      console.warn(e);
      console.log(this.writeOffset, this.totalBytes);
      this.endOfStream();
    }
  }
a9bc72ac-b7ac-43ba-96b0-9f8e58322a2b:37 ReferenceError: TextEncoderStream is not defined
    at AudioWorkletProcessor.appendBuffers (a9bc72ac-b7ac-43ba-96b0-9f8e58322a2b:35)
guest271314 commented 4 years ago

Solved by converting Uint8Array to JSON in one context then back to Uint8Array in a different context

            readable
            .pipeTo(
              new WritableStream({
                write: async value => {
                  externalPort.postMessage(JSON.stringify(value));
                },
              })
            )
            if (readable.locked) {
              controller.enqueue(Uint8Array.from(Object.values(JSON.parse(message))));
            }