Candidate

Péter Vajda

Supervisor

Prof. Touradj Ebrahimi

Introduction

Replica detection aims at discovering the replicas, or copies, of any originals. By replica, we refer not only to a bit exact copy of a given original, but also to modified versions of the original after minor manipulations, malicious or not, as long as these manipulations do not change its perceptual content. In particular cases of images, replicas could include all variants of the original image obtained after common image processing manipulations such as compression, blurring, contrast adjustment, change in colour space, etc.

The problem of image replica detection can be seen as a particular subset of the more general problem of content-based search and retrieval of multimedia content. For instance, in an image search and retrieval system based on query by example, the system returns images similar to the example, where the exact definition of 'similar' is often semantic. A query using the picture of a flower would return pictures of all sorts of flowers. Although, in general, a generic search and retrieval by example can be used also for replica detection, the problem can be resolved more efficiently by optimizing the solution to specific query examples. We have been working on such an approach in the framework of a previous Ph.D. thesis with successful results in the context of still image replica detection. The goal of this Ph.D. thesis is to extend further the work previously carried out towards detection of object replicas instead of entire images, in databases of stored images and video.

Systems for object replica detection can be used for a wide variety of tasks, including:

• Detect copyright infringement by identifying variations of a given copyrighted object.
• Cluster retrieved images based on visible objects to reduce reduce redundant information for the user.
• Propagate tags to new images based on the detection of the same object in previously annotated images.
• Detect occurrence of objects of interest in large surveillance databases.
• Provide information about an object captured by a mobile device.

Research activities

• Video duplicate detection using small video signature.
• Object replica detection based on local features and graph matching.
• Detection of specific object(s) that appear within video sequences.
• Specific object detection, such as flowers, shoes, clothes, etc.

Object duplicate detection

In this section, we present an efficient solution for 3D object duplicate detection in static images. The goal is to detect the presence of a target object and to predict its bounding box, based on a set of images containing that object. A small number of training images, typically one to four, containing different views of the target object, are sufficient enough to achieve good performance.

The block diagram of the developed object duplicate detection algorithm is depicted in the following figure:

During the training phase, a set of images of the target object (from different views, and with eventual deformations) is processed. First, features are extracted from the training images. To resolve the localization problem efficiently, we use accurate sparse features detector and descriptor. Hessian affine detector and SIFT affine invariant image descriptors are extracted. Hierarchical clustering is then applied in the feature space to increase the efficiency of the matching process.

In the testing phase, we retrieve images according to a query by applying two-direction nearest neighbor feature matching is applied. K-nearest neighbors in the query image are calculated and combined into the spatial neighborhood graph, considering the spatial distance of the features and similarity with the features in the model image. This graph contains spatial information from the potentially matching objects, hence making our algorithm more robust. The position is considered by applying General Hough Transform on the nodes of the graph.  This method results in several bounding boxes with overlapping and duplicated bounding boxes. These are then rejected by applying Non-maximum suppression algorithm.

Results can be seen in the following figure:

Related publications

• P. Vajda, F. Dufaux, T. H. Minh, T. Ebrahimi, “Graph-Based Approach for 3D Object Duplicate Detection”, 10th International Workshop on Image Analysis for Multimedia Interactive Services, 2009.
• L. Goldmann, T. Adamek, Peter Vajda, M. Karaman, R. Mörzinger, E. Galmar, T. Sikora, N. O'Connor, T. Ha-Minh, T. Ebrahimi, P. Schallauer, and B. Huet. "Towards, Fully Automatic Image Segmentation Evaluation", In Advanced Concepts for Intelligent Vision Systems (ACIVS), Lecture Notes in Computer Science, Berlin, 2008. Springer