Examples of Continuity Errors in Movies
Lyndsey C. Pickup & Andrew Zisserman
Overview
A continuity error is a lapse in the self-consistency of the scene or story being portrayed. The two movie frames shown above, from the movie Love Actually, come from two shots a few seconds apart in the movie, but display a number of discrepancies; the biggest visual discrepancies between the two are shown by the boxes.
Because such errors exist in films and television, humans become interested in tracking them down, putting more pressure onto the film makers to achieve higher standards of continuity throughout their productions. Some websites exist solely to comment on errors in movies, such as moviemistakes.com. Others like the Internet Movie Database (IMDb) cater for the wider interests of the movie-going public, but also contain user-contributed lists of continuity and other errors; such lists appear on IMDb in the "goofs" section.
The goal of this work is to detect visual continuity errors automatically, so that given a DVD of a movie, our software produces a ranked list of pairs of shots in the movie between which things change in an unexplained way. Such continuity errors include props that move inconsistently, shadows that lengthen or shorten, or drink in a glass that spontaneously empties and replenishes itself over the course of a scene.
Spot-the-Difference
In a standard spot-the-difference problem, every variation between a pair of images should be reported as a difference.
In movies, there exist many allowable differences, due primarily to the movement of people and their influence over items they interact with. There is no guarantee that the two movie frames to be compared are shot from exaclty the same camera angle, so image alignment forms part of the problem.
The goal is to return unexplained differences while supressing differences that have a plausible explanation. The examples below show eight of the positive results that our system has returned from several movies. Mousing over the images should show the second frame of the pair, warped so that it aligns with the first image. The image title (visible as mouse-over text) in each case shows the name of the movie and explains why it represents a continuity error.
Using the Movie Structure
There are typically a possible 100K-200K frames in a movie, but using the movie structure, this can be reduce to a list of around 200-550 pairs of frames which should be tested for errors. To achieve this, the frames' RGB histograms are compared to partition the movie in to shots, then shots which follow on from one another in threads are found, such that each shot in the thread is filmed from a similar camera angle in a sequence. There should not be major discrepancies between consecutive shots in the same thread, so the two frames on either side of the gap between shots are taken and treated as a candidate pair for further checking.
Locating Differences
The frames are first registered accurately, then small image patches are densely sampled from corresponding locations in both of them and compared. Regions where the patches do not match are flagged as potential discrepancies. Some of these mismatches are easily explained by the presence of people, which are found using upper-body detection, pedestrian detection and face detection. Others are due to other motion (e.g. vehicles, or people in unusual poses) which are identified by examining nearby frames in the movie.
Areas which are not close enough to people or moving objects in the scene remain unexplained, and are flagged as probable erros. See the step-by-step example page for a few more details on how this difference-finding processing is carried out. The CIVR paper (linked below) also contains more information.
Outputs
Clicking on each of the examples above will reveal the system's output for that pair of frames: red for an unexplained discrepancy, green for a flagged region that can be explained by a person-detection, and blue for a region that can be explained by motion.
More Fun
More examples of visual continuity errors similar to those above are shown on the More Mistakes page. These are retrieved from a wide variety of other movies.
Useful External Components
- Upper Body Detection
- Face Detection with OpenCV
- Kovesi's Matlab Libs
Related publication
Proceedings of the ACM International Conference on Image and Video Retrieval (2009)
Acknowledgements
This work is supported by an EPSRC Platform grant.
Source: https://www.robots.ox.ac.uk/~vgg/research/continuity/index.html#:~:text=Such%20continuity%20errors%20include%20props,the%20course%20of%20a%20scene.
0 Response to "Examples of Continuity Errors in Movies"
Post a Comment