AMIA 2024

Generative AI (GenAI) is flooding our world with a dizzying amount of synthetic media, and there is little guidance on how to responsibly navigate this new reality. The Archival Producers Alliance will present the Best Practices Guidelines we have developed for use of GenAI in documentary films. In this panel, we will explore the potential risks that GenAI presents to the Archive, to preservation, to the historical record, and to documentary film--as well as ways of mitigating these risks when using it. We will also discuss the methodology that went into developing the guidelines and how we see the Archival and Documentary spaces evolving in light of this powerful new technology.

Since the 2022 debut of Whisper, OpenAI’s automatic speech recognition software, an ecosystem of complementary tools and modifications has evolved, now offering organizations stewarding AV materials unprecedented opportunities to leverage this tool to increase the accessibility and discoverability of their digital collections. In this session, panelists from Emory, GBH, and Indiana will discuss their experiences and research implementing Whisper into their media management and preservation workflows, discussing Whisper’s performance across varied AV collections as well as the human impacts of working with Whisper. These insights from three organizations with varied scopes of collections and distinct but connected avenues of research may be helpful to a variety of organizations engaged in similar work or interested in starting up such a project. Attendees will gain insight into the strengths and weaknesses of leveraging Whisper, the challenges and opportunities presented by the technology, and practical guidance on implementing or expanding AV accessibility-related projects.

This panel will present a human-centered approach to AI-assisted cataloging. Panelists from GBH Archives, working on the American Archive of Public Broadcasting, will describe the development, implementation, testing, and use of AI-based media analysis tools within workflows for item-level cataloging digital videos. Panelists include cataloging, digital ingest, and metadata operations staff who have been involved in the creation and use of AI-based cataloging tools. We will discuss the CLAMS (Computational Linguistics Applications for Multimedia Services) suite of open source AI tools, post-processing CLAMS output for use in cataloging, questions we sought to answer regarding cataloging ease and efficiency, and the results of our experiments with tool integration. We will explain how our approach and roadmap differs from initiatives seeking to make cataloging fully automatic.

This presentation explores research on integrating AI in various organisational contexts, with a focus on audiovisual archiving. AI's potential is evident in search & exploration, preservation, artistic expression, and big-data analysis. Policies like the AI Act and Ethical Guidelines for Trustworthy AI shape strategies for responsible AI use. However, integrating AI in audiovisual domains poses challenges, such as choosing between off-the-shelf and bespoke solutions, aligning AI with legacy systems, considering public values in procurement, ensuring scalability and long-term viability, and fostering AI literacy. The audiovisual domain's specificity necessitates developing good practices. The AI4Media Network of Excellence engaged media practitioners to capture their experiences. This presentation shares these insights, offering practical guidance on sustainable, responsible AI integration into workflows and formulating policies for AI technology selection and use.

In this session, we will explore the cutting-edge applications of machine learning in film restoration, addressing not only color recovery and frame replacement but also the intricate processes of colorization, source/gauge matching, and nitrate decay recovery. Through a series of detailed case studies, I will demonstrate how AI tools are revolutionizing the field, making advanced restoration techniques more accessible. Drawing from a rich background in digital intermediates and a history of international collaboration, I will guide attendees through the ethical and practical considerations of integrating AI into film preservation workflows. The session is designed for those with an intermediate understanding of the field, but beginners will also find the discussions enlightening and informative. Participants will leave with a deeper appreciation of the capabilities of machine learning in film restoration. We will delve into how these tools can be leveraged to overcome traditional challenges, ensuring the longevity and integrity of our cinematic heritage.

AI companies are eager to amass collections of content to train both large language and niche models. In some cases, companies turn to archives and pay handsomely for what, for them, is valuable and unique raw material. Only the largest and most well-resourced archives have been able to take advantage of this opportunity. We are exploring models that better highlight the value of archival content, give institutions more opportunity for partnerships, and try to give more balance in negotiations. We outline how rights issues can be resolved with more technical knowledge and better understanding of intended use of archival material. Importantly, we propose how, working collectively, archives can foster a new revenue stream for digitization, preservation, and improved access to their collections.

Join us for an insightful discussion on the transformative impact of AI on audiovisual archiving. Traditional methods of transcription, indexing, and cataloguing have been laborious and costly, yet essential for enhancing the usability of collections. Over the past decade, AI has promised a revolution in these processes, though its effectiveness has been debated. Recent advancements, including OpenAI's Whisper and ChatGPT, have significantly enhanced AI transcription accuracy and mainstreamed AI technologies. Concurrent developments in AI Named Entity Recognition (NER) further facilitate the automatic extraction of key entities. Organizations like the USC Shoah Foundation, University of Kentucky's Louie B. Nunn Center for Oral History, and the TheirStory oral history platform have embraced these innovations, integrating them with tools. Join Sandra Aguilar (USC), Doug Boyd (University of Kentucky), and Zack Ellis (TheirStory) as they share their experiences, explore the benefits and challenges, and discuss the ethical considerations shaping the future of audiovisual archiving.

This presentation will introduce video content summarization (VCS) and its real-world applications. We'll explore how artificial intelligence (AI), particularly large language models (LLMs) and large vision-language models (LVLMs), can be used to understand and summarize video content, and then we will show how pipelined approaches with proper pre- and post-processing elements can improve the results. Additionally, we'll cover the effectiveness of AI in video summarization and the challenges that remain. Our goal is to provide the audience with a broad understanding of VCS and how AI is transforming this field. We'll also showcase a practical implementation of VCS pipelines in the CLAMS project, an open-source AI-assisted metadata extraction platform developed via collaboration between Brandeis University and AAPB (GBH MLA).

This presentation discusses the National Archives of Singapore’s experience using ICA’s new archival description standard, Records-in-Contexts (RiC) to integrate archival descriptions for audio-visual (AV) and paper-based archives. With a specific focus on broadcast and sound archives, the presentation will assess how RiC addresses perennial integration and data modelling challenges for AV resources, evaluate how other AV-centric ontologies may be used to complement and/or extend RiC, and propose design patterns that may be referenced by other institutions seeking to use RiC for archival description of AV materials.