Surgical Video Summarization: Multifarious Uses, Summarization Process and Ad-Hoc Coordination

While surgical videos are valuable support material for activities around surgery, their summarization demands great amounts of time from surgeons, limiting the production of videos. Through fieldwork, we show current practices around surgical videos. First, we highlight the critical role of video summaries in support of surgery, being used for self-improvement, education, discussing cases, scientific research, patient communication, and as legal resources. Second, we describe a video summarization process that is highly impacted by the loss of information originating during the surgery. Third, we shed light on ad-hoc coordination strategies which involve using the video itself for articulation work, making it both the field of work and coordination artifact. These results motivate and inform the future design of tools for their summarization.

1. The importance of video summaries: Multifarious uses

Summarized surgical videos have different uses.

1. Personal: Self-Improvement
2. Education: Teaching Residents
3. Professional: Discussing Cases
4. Scientific: Conferences and Studies
5. Public: Information and Communication
6. Legal: Litigation

The most notable uses are for personal, educational and scientific purposes. However, current video-editing tools are inefficient for summarizing surgical videos, and surgeons would increase the use of summarized videos if they had more efficient and faster tools at their disposal.

2. The summarization of surgical videos

Summarizing surgical videos is a complex multi-step process, it involves watching the video several times, and iteratively removing frames until obtaining a summary of the desired length. Although summarization strategies are personal, we observed that there were common strategies.

1. Overviewing
2. Cleaning
3. Coarse Sequencing
4. Fine Sequencing
5. Contracting

We find that existing video editing tools impose a working paradigm, as participants adopted a strategy consisting of removing sequences and frames with the goal of keeping only what they consider most relevant. Also, we observe that the summarization process is hindered by the fact that knowledge is lost from the live action to the summarization step. As a result, surgeons need to watch the entire video and recreate this information to identify the important moments. We identify that surgeons would like to record Key Moments during surgery for later use during summarization. These Key Moments can refer to 1) Surgical Steps, 2) Patient Anatomy, 3) Particular Surgical Gestures, and, 4) Unexpected or Critical Moments.

3. The collaborative nature of summarization: ad hoc coordination

Surgeons perform video summarization with colleagues for efficiency reasons. To collaboratively summarize videos, surgeons do not rely on functionalities in video-editing tools, simply because these tools do not provide any, they are not designed to support collaboration. They developed instead ad-hoc strategies to reify information used as articulation work when handing off a partially-done summarization to another surgeon. We observe three strategies in particular.

1. Strategy 1: Segmenting the Live Recording
2. Strategy 2: Capturing Moments on External Media
3. Strategy 3: Engraving Marks on a Partial Video Summary

There are two implications of the collaborative nature of summarization. First, coordination happens across two stages disjoint in their temporality and aim: a data-generation stage during a domain-specific activity, and later a data-processing stage during a non domain-specific activity (summarization). We are particularly focused here on supporting in the future the transferring of information that supports articulation work from one stage to the next as a component of this trajectory. Second, our study shows a particular case for the conceptualization of coordinating artifacts in CSCW where the coordination artifact is the common field object. This inseparability is a consequence of the lack of tools to deal with it, resulting in a protocol with implicit conventions and unprescribed procedures, contrary to the classic notion of coordination artifacts.

This study is published in the Proceedings of the ACM on Human-Computer Interaction.

The publication has an accompanying video:

Reference :
Ignacio Avellino, Sheida Nozari, Geoffroy Canlorbe, et Yvonne Jansen. 2021. Surgical Video Summarization: Multifarious Uses, Summarization Process and Ad-Hoc Coordination. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 140 (avril 2021), 23 pages. https://doi.org/10.1145/3449214