ClinicianFOCUS / freescribe-copilot

Chrome and Firefox Plugin to assist healthcare providers interacting with patients with transcribing and preparing notes, and other active assistance; under development.
GNU Affero General Public License v3.0
0 stars 0 forks source link

real time transcription implementation #6

Open n41tik opened 2 days ago

n41tik commented 2 days ago

Summary by Sourcery

Implement real-time transcription by periodically restarting the media recorder and refactor code for improved readability and API key handling.

New Features:

Enhancements:

sourcery-ai[bot] commented 2 days ago

Reviewer's Guide by Sourcery

This pull request implements real-time transcription functionality, improves the configuration handling, and refactors the audio recording and SOAP note generation process. The changes focus on enhancing the user experience and streamlining the code.

Sequence diagram for real-time transcription process

sequenceDiagram
    actor User
    participant UI as User Interface
    participant MR as MediaRecorder
    participant WS as Whisper Server
    participant GUI as GUI Update

    User->>UI: Click Record Button
    UI->>MR: Start Recording
    loop Every 5 seconds
        MR->>WS: Send Audio Blob
        WS-->>MR: Return Transcription
        MR->>GUI: Update GUI with Transcription
    end
    User->>UI: Click Stop Button
    UI->>MR: Stop Recording

Updated class diagram for index.js

classDiagram
    class Config {
        +String WHISPER_URL
        +String WHISPER_API_KEY
        +String AI_SCRIBE_URL
        +String AI_SCRIBE_MODEL
        +String AI_SCRIBE_CONTEXT_BEFORE
        +String AI_SCRIBE_CONTEXT_AFTER
    }
    class MediaRecorder {
        +ondataavailable(event)
        +onstop()
        +start()
        +stop()
    }
    class UIElements {
        +Element audioInputSelect
        +Element recordButton
        +Element stopButton
        +Element userInput
        +Element soapNotesElement
    }
    class Functions {
        +toggleConfigView()
        +convertAudioToText(audioBlob)
        +updateGUI(text)
        +generateSoapNotes(text)
    }
    Config --> MediaRecorder : uses
    UIElements --> MediaRecorder : interacts
    Functions --> MediaRecorder : controls
    Functions --> Config : accesses
    Functions --> UIElements : updates

File-Level Changes

Change Details Files
Implement real-time transcription
  • Add interval-based recording to continuously process audio chunks
  • Modify the mediaRecorder to start and stop at regular intervals
  • Update the GUI with transcribed text in real-time
index.js
Improve configuration handling
  • Add toggle functionality for configuration visibility
  • Implement validation for Whisper URL and AI Scribe URL
  • Auto-hide configuration section after saving
index.js
Refactor audio recording process
  • Remove audio playback functionality
  • Simplify the audio blob handling and conversion to text
  • Update the user interface elements' states during recording
index.js
Enhance SOAP note generation
  • Simplify the SOAP note generation and display process
  • Use consistent variable names for DOM elements
  • Remove redundant displaySoapNotes function
index.js
Update API authentication method
  • Change Whisper API authentication from X-API-Key to Bearer token
index.js

Possibly linked issues


Tips and commands #### Interacting with Sourcery - **Trigger a new review:** Comment `@sourcery-ai review` on the pull request. - **Continue discussions:** Reply directly to Sourcery's review comments. - **Generate a GitHub issue from a review comment:** Ask Sourcery to create an issue from a review comment by replying to it. - **Generate a pull request title:** Write `@sourcery-ai` anywhere in the pull request title to generate a title at any time. - **Generate a pull request summary:** Write `@sourcery-ai summary` anywhere in the pull request body to generate a PR summary at any time. You can also use this command to specify where the summary should be inserted. #### Customizing Your Experience Access your [dashboard](https://app.sourcery.ai) to: - Enable or disable review features such as the Sourcery-generated pull request summary, the reviewer's guide, and others. - Change the review language. - Add, remove or edit custom review instructions. - Adjust other review settings. #### Getting Help - [Contact our support team](mailto:support@sourcery.ai) for questions or feedback. - Visit our [documentation](https://docs.sourcery.ai) for detailed guides and information. - Keep in touch with the Sourcery team by following us on [X/Twitter](https://x.com/SourceryAI), [LinkedIn](https://www.linkedin.com/company/sourcery-ai/) or [GitHub](https://github.com/sourcery-ai).