Two years ago we presented the early results of applying artificial intelligence and machine learning to image restoration. We showed the results that preliminary forms of AI could deliver in automatically detecting and removing scratches. Since then, there has been steady stream of major breakthroughs in AI that have enabled some truly transformational new capabilities in the restoration of and image processing of moving picture sequences.
In this presentation we will discuss the capabilities of the latest AI breakthroughs (specifically in the area of deep neural networks), the practical results they are delivering in moving image restoration, and what the future holds. We will highlight our most recent research and results using AI in the following application areas:
– Dust removal: Today’s deep learning models have increased the reliability of dust recognition by 30-50%, which drives big cost savings by heavily cutting the need for manual retouching.
– Frame interpolation: A new AI-based algorithm can generate very accurate, entirely synthetic interpolated frames that can be used in a variety of scenarios. These computed interpolated frames can be used to replace frames that are entirely missing, badly torn, or severely damaged. These computed frames can also be used as a source to seamlessly replace dust, stains, scratches and mold within otherwise good original frames. And, interpolated frames can even be used to produce slow motion, or otherwise re-timed sequences.
– Video de-interlacing: Carefully trained deep neural networks are now delivering visually better results in this crucial video restoration step than previous manmade algorithms. This raises the potential of repurposing standard definition video footage into very good quality HD programming through AI driven de-interlacing, detail enhanced up-rezing, and tone scale expansion.
– Optical Flow (motion estimation): Accurately tracking what is moving and where from frame to frame is essential to determining which image features don’t belong (this is key to restoration.) Recent AI driven optical flow methods developed to compress modern video images have been delivering terrific results- with better tracking of fine details and better identification of object boundaries. But, these AI methods are often less stable on older movies because the high grain levels and flickering. We will review our research into different neural network designs and tuning methods to leverage these new optical flow approaches in the more difficult environments we all find in restoration.
Looking at a higher level, we will review some of the collaborative approaches being used in the exciting world of Edge Computing, and how they can be applied to movie and video restoration. If we, as a community can collaborate like the edge computing community, we can substantially accelerate the pace of machine learning and drive even faster improvements in the efficiency and power of digital restoration algorithms.
Finally, we will demonstrate an exciting new technical approach to restoration enabled by the collective power of all these AI imaging advancements. This new approach gives today’s archivist important new abilities to deliver both historically accurate restorations and masters that satisfy the demand for high quality, highly compressed masters for broadcast and streaming, at the same time, with very little additional cost.
AlgoSoft-Tech presentation at The Reel Thing, August 23, 2019, Hollywood, CA TOPIC: “How Recent ‘AI’ Breakthroughs Are Transforming Moving Image Restoration”
Presented by Inna Kozlov, Alexander Petukhov and Michael Inchalik:
http://vimeo.com/359693988
We are very grateful to Bart Santello (Psychotropic Films) for recording and providing the presentation for the website.