Turbo Stream rendering plain text html instead of replacing content in targeted turbo frame

[rorykoehler]

I am building an LLM backed app on Rails 7.1.3.4 where I want to continuously replace the main content based on the next prompt response. I start in home/index.html.erb which looks like this (some javascript which isn't shown, records audio and inserts it into the audio form field).

<div class="d-flex flex-column align-items-center" style="height: 100vh;">
    <div class="mb-3" data-controller="speak-button">
        <span class="speak-text me-2">Tell me something interesting you found out recently</span>
        <button class="btn btn-primary speak-button">
        <i class="fas fa-volume-up"></i>
        </button>
    </div>
    <button id="record-button" class="btn btn-danger rounded-circle d-flex justify-content-center align-items-center" style="width: 100px; height: 100px;">
        <i class="fas fa-microphone"></i>
    </button>

    <%= form_with url: prompts_path, method: :post, data: { turbo_frame: :responses}, id: "audio-form", multipart: true do %>
        <%= file_field_tag :audio, style: 'display: none;', id: 'audio-file' %>
    <% end %>

    <%= turbo_frame_tag 'responses' do %>
        <!-- The last prompt responses will be dynamically loaded here -->

        <!-- Subject specific responses will be dynamically loaded here -->
    <% end %>
</div>

<%= javascript_include_tag 'audio_recorder' %>

So far so good. I click the record button and record my question, it stops recording and auto submits the form to /prompts.

My prompts controller looks like this:

class PromptsController < ApplicationController
    def create
        prompt_service = PromptService.new

        if params[:audio]
            audio = params[:audio]
            @transcription = prompt_service.transcribe_audio(audio)
            @subject = "all"
        else
            @subject = params[:title].downcase
            @transcription = params[:text]
        end

        gpt4_response = prompt_service.get_response_from_gpt4(@transcription, @subject)
        @responses = JSON.parse(gpt4_response)

        respond_to do |format|
            format.turbo_stream
        end
    end
end  

It renders the prompts/create.turbo_stream.erb no problem and I see the responses I expect rendered correctly.

 <%= turbo_stream.update "responses" do %>
        <%= render "prompts/transcription", transcription: @transcription %>

        <%= render "prompts/responses", responses: @responses %>
<% end %> 

_transcription.html.erb:

<div>
        <h3>Transcription:</h3>
        <p><%= transcription %></p>
</div>

_responses.html.erb:

<div class="container my-5">
    <div class="row row-cols-1 row-cols-md-6 g-4">
      <% responses.each do |key, value| %>
        <div class="col">
          <div class="card" data-controller="speak-button">
            <div class="card-body">
              <%= form_with url: prompts_path(format: :turbo_stream), method: :post, class: "clickable-form" do |form| %>
                <h5 class="card-title">
                    <%= form.hidden_field :title, value: key %>
                    <%= form.hidden_field :text, value: value %>
                    <a href="#" onclick="this.closest('form').submit(); return false;"><%= key %></a>
                </h5>
                <p class="card-text speak-text">
                  <a href="#" onclick="this.closest('form').submit(); return false;"><%= value %></a>
                </p>
              <% end %>
              <button class="btn btn-primary speak-button">
                <i class="fas fa-volume-up"></i>
              </button>
            </div>
          </div>
        </div>
      <% end %>
    </div>
</div>

The html looks like this after the first render of the turbo stream

<turbo-frame id="responses">

    <div>
        <h3>Transcription:</h3>
        <p>Blue whales are the biggest animal to have ever lived.</p>
    </div>

  <div class="container my-5">
    <div class="row row-cols-1 row-cols-md-6 g-4">
        <div class="col">
          <div class="card" data-controller="speak-button">
            <div class="card-body">
              <form class="clickable-form" action="/prompts.turbo_stream" accept-charset="UTF-8" method="post"><input type="hidden" name="authenticity_token" value="QLvB_NtoFVpgQRGQmVtXrTHhpduPKUYTn4RTCQ0KozVb6dao7h-zL_JstuJ4KPutaTgUg" autocomplete="off">
                <h5 class="card-title">
                    <input value="Language and Literature" autocomplete="off" type="hidden" name="title" id="title">
                    <input value="Herman Melville's classic novel 'Moby-Dick' features a sperm whale as its central character, but blue whales are even larger than the fictional Moby-Dick and any sperm whale in reality." autocomplete="off" type="hidden" name="text" id="text">
                    <a href="#" onclick="this.closest('form').submit(); return false;">Language and Literature</a>
                </h5>
                <p class="card-text speak-text">
                  <a href="#" onclick="this.closest('form').submit(); return false;">Herman Melville's classic novel 'Moby-Dick' features a sperm whale as its central character, but blue whales are even larger than the fictional Moby-Dick and any sperm whale in reality.</a>
                </p>
                </form>              
                <button class="btn btn-primary speak-button">
                <i class="fas fa-volume-up"></i>
              </button>
            </div>
          </div>
        </div>
      <!-- A FEW MORE OF THESE REMOVED FOR BREVITY -->
    </div>
  </div>
</turbo-frame>

The issue starts when I submit the form above from _responses.html.erb for a second roundtrip of the turbo stream. It renders plain text html of only the turbo frame content to the browser as in this image:

I have tried using various other turbo stream methods such as replace and advance. I have also tried using turbo frames with a prompt/create.html.erb. This has similar behaviour in that the second time around it loads a whole new html page with just the turbo frame content. Headers and response formats etc all look correct to me (this is incorrect... see new comment).

If I copy the form structure from the working initial audio capturing form i.e adding data: { turbo_frame: :responses} and removing format: :turbo_stream I get ActionController::UnknownFormat in PromptsController#create

The audio capturing form continues to work across multiple submissions and re-renders of the turbo frame. I've just come across to Turbo from Stimulus Reflex and feel like I've read half of the internet trying to understand what's going on at this stage. I am not really sure where to go from here. Any advice or insights would be very much appreciated.

[rorykoehler]

I've had a bit of time and headspace to look at this again and on second look realised the Request Header on the second form is not being set correctly. It is Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/ png,image/svg +xml,*/*:q=0.8 instead of Accept: text/vnd.turbo-stream.html, text/html, application/xhtml+xml.

Unless I am misunderstanding the implementation I suspect what is causing the issue is events on new forms being rendered from within a turbo stream are not being caught. Adding the format: :turbo_stream attribute doesn't seem to change this. This issue lead me here: https://github.com/hotwired/turbo/issues/1069 .

For now I have worked around this issue by populating the original audio form with extra hidden fields on click and submitting that instead of having an alternative form as in the original code submitted.

[4lllex]

don't add format: turbo_stream to your form url: prompts_path(format: :turbo_stream) should just be prompts_path. also you need to use requestSubmit() when submitting a form this.closest('form').requestSubmit().