About

A New Standard for Video Analytics

The Video Analysis Tableau (VAT) is an online toolkit created for automated video comparison, annotation and visualization.

Individual Files Viewed from The VAT, running on Clowder
Individual Files Viewed from The VAT, running on Clowder.

Built on the Clowder Framework and supported by the NSF’s XSEDE Program (Extreme Science and Engineering Discovery Environment), The VAT employs both current and experimental discovery methods to query videos against entries in its database. Imagining video frames as a three dimensional volume, The VAT applies seventeen computer vision algorithms to match a selected video with other assets in its data set, enabling users to uncover previously unknown thematic, formal or technical connections within both related and diverse sources.

Visualizing the Movie Cube.

Filmic media
is one of the most compelling
big data
issues of our time.

Its formats are diverse, rapidly transmitted, and boundlessly large in number. As such, it demands scholarly attention within and beyond the field of cinema studies.

The VAT joins the emergent field of cultural analytics, an approach that deploys computer technologies to analyze the formal features of art and culture, making them available to interpretive methods. Moving image media is particularly ripe for computational analysis given its ubiquity in contemporary culture. Indeed, digital video—whether recorded digitally or digitized from film—is a rapidly expanding form of cultural production, one made possible by the proliferation of personal recording technologies and hosting platforms like YouTube, Vimeo and the Internet Archive.

Yet despite its scale and importance, video remains a daunting object for sustained analysis for reasons that are technological, institutional and conceptual in nature. The VAT’s goal is to fill existing gaps in scholars’ ability to ask cultural questions about filmic archives using computers, while also experimenting with transformative methods in research and analysis. The long term goal is to allow researchers to move with agility from textual description and collection management, to manual inspection, to automated analysis, to visualization of discrete films as well as whole collections.

Publications

Press

Project Team

Virginia Kuhn

Principal Investigator

School of Cinematic Arts
University of Southern California

Alan Craig

Co-Principal Investigator

XSEDE

Michael Simeone

Co-Principal Investigator

Nexus Lab, Arizona State University

Luigi Marini

Senior Research Programmer

NCSA, University of Illinois

Sandeep Puthanveetil Satheesan

Research Programmer

NCSA, University of Illinois

Dave Bock

Senior Visualization Programmer

NCSA, University of Illinois

Jonathan Fudem

Research Assistant

MA+P, School of Cinematic Arts, University of Southern California

Alexis Bradby

Research Assistant

MA+P, School of Cinematic Arts, University of Southern California

Jonah Sanchez

Research Assistant

MA+P, School of Cinematic Arts, University of Southern California