[Reading] Image Retrieval: Ideas, Influences, and Trends of the New Age

I choose to summarize from 3. IMAGE RETRIEVAL TECHNIQUES: ADDRESSING THE CORE PROBLEM (p.14) to 3.2. Image Similarity Using Visual Signature (p.30).

CBIR technology amounts to 2 problems: (a) the design of image description (signature), and (b) the similarity measure between two image descriptions. In the recent years, the design of features and the signatures constructed by these features have much progress. Besides, using machine learning techniques in CBIR has become more popular and also important.

Signature Extraction

Feature extraction is the first step, after we extract features from an image, we need to do signature construction using these features. There are 2 ways to do signature construction: (a) using segmentation as first step, and (b) segmentation-free approach.

To acquire a region-based signature, image segmentation is needed. Several methods proposed to do segmentation on medical images. Segmentation-based approach may have the problem that result is too sensitive to segmentation quality, so several methods tried to solve these problems.

Computing global feature is efficient, but it is insensitive to location. So a better way (also a trend) is to compute local features then summarize them. Several types of local features are discussed such as color, texture, shape, spatial modeling and interesting points.

When the # of features are very large for us to choose, we can use machine learning techniques to do feature selection.

For constructing region-based signature, several methods proposed to do signature construction. Lots of them have a connection with histograms.

Similarity

There are 3 types of signatures: (a) region-based signature, (b) feature vector, and (c) summary of local feature vectors. Different types of signatures have different similarity measures. For (a), the definition of distance between “set of vectors” is crucial. For (b), several recent efforts have been made to measure the distance on a manifold, because using geodesic as distance measure is more reasonable. For (c), codebooks and probability density functions have been used as signatures.

For region-based signature, there are basically 2 formulations to compute similarity, one is using the sum of weighted pair-wise distance as formulation, different constraints lead to different design of weights. Another approach is using Hausdorff distance. Recently, several improvements have been made including feature tuning, weight computation, robustness against inaccurate segmentation and speeding-up retrieval.

For feature vector, computation of similarity is performed nonlinearly along the manifold, typical methods are locally-linear embedding (LLE), isomapping, and multidimensional scaling.