Automated Surgical Report Generation Using In-context Learning with Scene Labels from Surgical Videos

[u-keisuke]

:school_satchel: Poster submission

Welcome to Poster submission for 🎒 Open-Source Software for Surgical Technologies :tada:

Personal Details

• First author:
  • [ ] family-names: Ueda
  • [ ] given-names: Keisuke
  • [ ] email: u.keisuke1111@gmail.com
  • [ ] orcid: 0009-0006-8132-4008
  • [ ] Department/Faculty:
  • [ ] affiliation: Medical Dataway
• Second author:
  • [ ] family-names: Fujisawa
  • [ ] given-names: Mineto
  • [ ] email: mifujisawa-tky@umin.ac.jp
  • [ ] orcid: 0009-0006-5064-1879
  • [ ] Department/Faculty: Faculty of Medicine
  • [ ] affiliation: The University of Tokyo / Medical Dataway
• Third author:
  • [ ] family-names: Hiratsuka
  • [ ] given-names: Daiki
  • [ ] email: hiratsuka.daiki@gmail.com
  • [ ] orcid: 0000-0001-7128-2859
  • [ ] Department/Faculty:
  • [ ] affiliation: Medical Dataway
• Fourth author:
  • [ ] family-names: Ruggeri
  • [ ] given-names: Giovanni
  • [ ] email: giovanni.ruggeri@insel.ch
  • [ ] orcid: optional
  • [ ] Department/Faculty: Department of Gynecology and Obstetrics
  • [ ] affiliation: University of Bern and Bern University Hospital
• Fifth author:
  • [ ] family-names: Matsumoto
  • [ ] given-names: Takashi
  • [ ] email: t_matsumoto@medicaldataway.io
  • [ ] orcid: optional
  • [ ] Department/Faculty
  • [ ] affiliation: Medical Dataway
• Sixth author:
  • [ ] family-names: Mueller
  • [ ] given-names: Michael
  • [ ] email: michael.mueller@insel.ch
  • [ ] orcid: optional
  • [ ] Department/Faculty: Department of Gynecology and Obstetrics
  • [ ] affiliation: University of Bern and Bern University Hospital

Which subtheme most closely connects to your poster?

• [ ] Open-source software libraries and frameworks,
• [ ] Medical and surgical software innovations,
• [ ] Sustainability in open-source software,
• [ ] Case studies showcasing novel applications and combinations of existing software,
• [ ] Protocols for managing clinical data in computer-assisted software for surgical technologies, and
• [x] Project summaries, methodological approaches, research findings, and initiatives related to Open-Source Software for Surgical Technologies.
• [ ] Other (please add topic).

Poster title

Automated Surgical Report Generation Using In-context Learning with Scene Labels from Surgical Videos

Briefly describe your poster proposal

We propose a method for generating surgical reports from surgical video scene labels and demonstrate the effectiveness of In-context Learning (ICL) in this process. Writing surgical reports is a significant burden for surgeons. Utilizing the open-source language model Llama 3 (8b), we generate surgical reports from scene labels of surgical videos through few-shot learning, comparing the performance of 1-shot, 2-shot, and 3-shot scenarios. Gynecologists wrote reference surgical reports for ten videos, and the generated reports were evaluated based on the number of errors compared to these references. The results indicate that increasing the number of shots reduces errors in the generated reports, confirming the effectiveness of ICL in surgical report generation. This approach has the potential to alleviate the documentation workload for surgeons, improving efficiency and accuracy in medical reporting.

[mxochicale]

Dear @u-keisuke

Thank you for your submission! We are pleased to inform you that your poster has been accepted.

Please refer to Zenodo DOI's for accepted poster for further details on Zenodo submission! We have sent you an email with further details of our workshop, please do not hesitate to reach via this issue or email if there is anything else we can help!

Looking forward to meeting you! Thanks, @evaherbst @thompson318, and @mxochicale

[u-keisuke]

Dear @mxochicale

I have successfully submitted my poster to Zenodo! Here is the DOI for my submission: 10.5281/zenodo.12518729.

Thank you very much, @u-keisuke

[mxochicale]

Thanks @u-keisuke for submitting your poster in Zenodo. We will be updating the main readme soon and share a summary of the outcomes of the workshop to bring more visibility of your work!