Objectives

Audiovisual collections in transition

Audiovisual content collections are transforming from archives of analogue materials to very large stores of digital data. As time based digital media and their related metadata are edited, reused and reformatted in a continuously evolving environment, the concept of the unique original loses its meaning. We require new dynamic preservation processes.

The purpose of PrestoPRIME was therefore to research and develop practical solutions for the long term preservation of digital audiovisual objects, programmes and collections. It also sought ways to increase access by making media archives available within the framework of European online digital libraries. The result was a range of tools and services made available through a networked Competence Centre.

Audiovisual recordings preserve the history of the 20th Century so that events and personalities can be seen and heard. Unlike documents which may last for centuries, audiovisual recording materials have a life expectancy measured at best in decades. The EC sponsored projects Presto and PrestoSpace addressed the first stage of the problem by developing a preservation factory approach and methods for digitising audiovisual collections.

The processes and technology for digitising analogue holdings are now well known. Although the process is far from complete, around 10 million hours of European audiovisual content have already been digitised. Audiovisual media is now born digital and will also be kept as electronic files. The new challenge is how to ensure the long term viability of the content within these files.

Why copying is not preservation

Digital preservation requires the indefinite error free storage of digital information, with means for its retrieval and interpretation, irrespective of changes in technologies, support and data formats, or changes in the requirements of the user community.

It is wrong to think that simply copying or migrating media files to a new format will solve the problem. Technical obsolescence, transcoding, reformatting, data movement and physical media deterioration all introduce threats and the risk of loss of quality and content. Checking integrity is not straightforward, and files, being invisible, are easily lost or mislaid. It is often supposed that digital copies are necessarily identical, but the size of audiovisual files increases the probability of read errors.

Transcoding artefacts in the digital audiovisual domain are an obvious danger when lossy codecs and formats are used. More subtle risks are emerging in the audio domain, which reveals artefacts as a result of multiple transcoding with professional digital equipment that supports supposedly lossless formats. A new everlasting storage medium would not solve the problem either, because even standard file formats disappear. MPEG 1 video, introduced as a digital equivalent to VHS videotape in the early 1990s, is already effectively obsolete.

Archives and libraries therefore need to store not just the audiovisual files, but all the information needed to keep file content readable and usable.

From OAIS to everyday practice

Many standards, reference models and frameworks have been suggested for managing content in digital libraries. The OAIS Reference Model defines the processes required for long term preservation and access to information objects, establishing a common language to describe them. Although OAIS discusses preservation through migration, it does not specify how to monitor objects or the systems they are stored in, identify when migration should take place, how it should take place, or to what an object should be migrated.

Research is in progress to create a theory of preservation that extends the concept from sending records into the future to preserving a description of the environment used to manage and read the records. The entire information context should be described so well that records can be migrated into an independent preservation environment without losing authenticity or integrity.

The use and reuse of content in different territories adds a further level of metadata in the form of rights. A study by RAI identified 800 different types of rights associated with its archived assets alone. This number increases when scaled at the European level, and the ability to manage rights has a bearing on public access to audiovisual archive content.

In the digital universe, broadcast archives are part of a wider system of library, museum and private sector media collections, each with its own media creation and preservation practices. Changing technologies and consumption patterns are also blurring the boundaries between personal and public media. A growing volume of user generated content is entering the public space through the internet and through mainstream broadcasters. Any general solution for long term audiovisual preservation needs to accommodate the different approaches of media creators, the archive and library communities, and support unified online access.

The PrestoPRIME approach

PrestoPRIME researched methods for applying long term preservation practice and preservation theory to the practical requirements of sound and moving image content, so that it can be preserved in affordable distributed and federated environments. The work recognised:

• The complexity of audiovisual file formats and wrapper files that include multiple signals and time dependent metadata, in formats largely unknown to digital library preservation technology.
• The need to preserve the signal. Audiovisual files encode an approximation of a signal, which is reconstituted in playback. Migration of the optimal representation of the signal is an absolute requirement for public access.
• The problem of large numbers of large files. Mass storage works by reducing failure rates and using redundancy and maintenance. When moving from terabytes to petabytes to exabytes, this approach is expensive and data losses can be catastrophic. Storage management technology needs to understand the audiovisual signal and perform a best effort playback of data that is still readable.
• The cost of safety. The trade offs between quality, quantity, cost and safety must be optimised so that archives can afford to preserve huge volumes of digital audiovisual content to acceptable standards.

Four objectives

1. Ensure the permanence of digital audiovisual content

Research and develop means of ensuring the permanence of digital audiovisual content in archives, libraries, museums and other collections. This was achieved by:

• Comparing strategies for audiovisual preservation, including multivalent, emulation and migration approaches, and creating a standard data model for audiovisual content preservation.
• Researching metadata solutions for long term audiovisual preservation to manage migration between data models, metadata enrichment, the maintenance of correct relationships between audiovisual objects, the integrity of files, and a model of the preservation process itself.
• Modelling storage and processing as a set of services with well defined interfaces, service level agreements and quality of service metrics appropriate to preservation.
• Producing software tools and a framework for implementing and combining both migration and multivalent preservation strategies optimised for audiovisual content.
• Developing a service oriented storage infrastructure that enables multivalent and migration preservation strategies to be deployed on local and federated storage and processing.
• Creating a quality assessment and risk management framework for monitoring and optimising the long term safety and integrity of audiovisual content.

2. Ensure long term future access

Research and develop means of ensuring long term future access to audiovisual content in dynamically changing contexts. This was achieved by:

• Establishing metadata interoperability between audiovisual archives, cultural heritage institutions, the Semantic Web and content portals, with services for metadata conversion and deployment.
• Developing tools and services to enrich archive content with user generated metadata and align it with existing metadata.
• Developing services for tracking audiovisual content, using fingerprinting techniques and documenting the provenance of audiovisual content items.
• Modelling rights associated with audiovisual content and developing standard compliant services for archive rights management.

3. Integrate, evaluate and demonstrate

Integrate, evaluate and demonstrate tools and processes for audiovisual digital permanence and access. This was achieved by:

• Integrating the software modules and metadata tools in a functioning testbed.
• Validating the preservation actions and tools in the testbed.
• Developing test material and proving the ability of the integrated testbed to carry out the required actions for long term preservation with very large volumes of audiovisual content.
• Integrating the PrestoPRIME software with existing market leading digital library software.

4. Establish a European networked Competence Centre

Establish a European networked Competence Centre to gather the knowledge created by PrestoPRIME and deliver advanced digital preservation advice and services in conjunction with the European Digital Library Foundation and other projects. This was achieved by:

• Setting up the legal and operational structure of the networked Competence Centre for audiovisual preservation and a European community of audiovisual practitioners.
• Defining and initiating activities for monitoring and registering technology and best practice.
• Issuing guidance on standards relating to audiovisual content and its preservation.
• Establishing a communication platform for the Competence Centre to manage its outputs.
• Publishing economic guidance and business models for digital media preservation activities.