## Difference between Z-test, F-test, and T-test

A z-test is used for testing the mean of a population versus a standard, or comparing the means of two populations, with large (n ≥ 30) samples whether you know the population standard deviation or not. It is also used for testing the proportion of some characteristic versus a standard proportion, or comparing the proportions of two populations.

Example:Comparing the average engineering salaries of men versus women.

Example: Comparing the fraction defectives from 2 production lines.

A t-test is used for testing the mean of one population against a standard or comparing the means of two populations if you do not know the populations’ standard deviation and when you have a limited sample (n < 30). If you know the populations’ standard deviation, you may use a z-test.

Example:Measuring the average diameter of shafts from a certain machine when you have a small sample.

An F-test is used to compare…

View original post 90 more words

## Understanding Harris corner detector

Let’s first go over Harris detector a little bit. For a basic idea about Harris detector, check textbooks or opencv or blogs. For its intuition, check its precursor Movarec operator, which explains why we want to maximize the variation within a window to find a corner. The key to Harris detector is the variation of intensity within a sliding window (with displacement in the x direction and in the y direction)

where:

- is the window weighting function at position
- is the intensity at (x,y)
- is the intensity at the moved window

The aim is to maximize . Applying Taylor expansion and some arithmetic operations, we can get

where is the gradient at . Matrix is known as structure tensor of a pixel. It actually a characterization of information of all pixels within the window. Then we calculate the eigenvalues and of matrix to determine if window corresponds to a corner. (I)

- If and then this pixel has no features of interest.
- If and has some large positive value, then an edge is found.
- If and have large positive values, then a corner is found.

[Note: to avoid computing the eigenvalues which is computationally expensive, in original Harris detector, the authors suggested using the following score:

to determine if a point is a corner.]

The question is WHY? Why does the matrix have the capacity to determine the cornerness? Here, we look around to the covariance matrix of gradient at a pixel. Let . By definition of covariance matrix, we have

The last step is derived based on the the assumption that the mean of derivatives and is zeros, i.e. .

On the other hand, the structure tensor is as follows if we apply an averaging filter over the time dimension (= we give uniform weight over the pixels within the pixels):

where is total number of pixels within the window, and is the average of .

It can be seen that ** structure tensor is actually an unbiased estimate of the covariance matrix of the gradients for the pixels within the window**. Therefore, any findings/conclusions about covariance matrix could be used to analyze structure tensors.

Let’s look at an example showing how the derivatives are distributed for different types of image patches (i.e. the window). The following figures are adapted from Robert Collins‘s slides. In this example, the image patches are from three different types: linear edge, flat area and corner. You can image that we are testing if the pixel at the center is a corner (feature point).

The first step is calculating the derivatives and for each pixel in the image, then the distributions of (, ) are shown in the figure below. Obviously, the distributions are discriminative fro these three different type of pixels (on the edge, on flat area and a corner).(II)

- For corners, the ranges in both and are large.
- For points on an edge, one derivative has wide distribution while the other is almost all near zeros.
- For points of a flat area, both derivatives are around zeros.

This observation is indeed the reason behind (I) we discussed above.

In Relationship between eigenvectors and directions of data distribution, we have shown that the eigenvectors of a covariance matrix are the same as the directions of principle component (i.e. the axes of a ellipsoid enclosing the data points), and the eigenvalues measure the variances of date points along eigenvectors. (Click Read more to see the relationship between eigenvalues and the semi-axes) Therefore, if the smallest eigenvalue is big enough, then it means the data have large variance along all different directions–it’s a corner!

In a word, the structure tensor is actually the covariance matrix of gradients for pixels around the pixel investigated, the the reason for current eigenvalue-based analysis is based on the property of covariance matrices.

## [OPENCV] Feature detectors and descriptors

For a simple example of how to setup and use them, see [OPENCV] OpenCV+Eclipse|Qt|VS2010

For many computer vision tasks, the first step is usually extracting features from images. It consists of two parts: feature detection and description. By feature detection, we mean to find the pixels (or regions, objects) of interest in an image, which are represented by ** cv::KeyPoint**. They are corresponding to the corners of objects in the image. OpenCV implements many popular feature points detection methods, such as SIFT, SURF, ORB, GoodFeatureToTrack and so on.

After these feature points are located, the next step is to describe them. An intuitive description of a feature point could be its intensity/color value. But a single intensity/color value does not provide sufficiently discriminative information so that this feature point can be uniquely matched to feature points in another image. Usually we use the information around a feature point to represent it, i.e. we use information of many pixels to represent one feature point. Based on Hubel & Wiesel’s research on information processing, orientation is the most sensitive factor to visual system. That’s why most feature techniques are based on orientation to describe feature points. For example, SIFT descriptor uses the normalized orientation histogram to describe a feature point. See figure below. Since this post is not about how the descriptors are designed, interested readers should read their papers.

To provide consistent programming interface, OpenCV designs the feature detectors and descriptor extractors following OO principle. Below is the class hierarchy of most detectors and descriptors for version 2.4.1.** FeatureDetector **and

**are the base class for detectors and descriptors, respectively.**

*DescriptorExtractor***simplifies detection and extraction by deriving from**

*Feature2d***and**

*FeatureDetector***. But**

*DescriptorExtractor***is still an abstract class because it doesn’t implement the virtual function**

*Feature2d***and**

*detectImpl()***.**

*computeImpl()*Classes * SURF*,

*,*

**SIFT***and*

**BRISK***(and others) are concrete classes which are both detectors and descriptor extractors. In order to explicitly use them as detectors or descriptor extractors, aliases are defined. For example,*

**ORB**typedef SURF SurfFeatureDetector; typedef SURF SurfDescriptorExtractor; typedef SIFT SiftFeatureDetector; typedef SIFT SiftDescriptorExtractor; typedef ORB OrbFeatureDetector; typedef ORB OrbDescriptorExtractor;

From the perspective of functionality, there’s no distinction between ** SurfFeatureDetector** and

**. They are exactly the same thing except they have different names. The same is true for**

*SurfDescriptorExtractor***and**

*SiftFeatureDetecor***, and others.**

*SiftDescriptorExtractor*From this class hierarchy diagram, it can also be seen that ** FAST**,

**and**

*GFTTDetector***are only feature point detectors. After obtaining the feature points (**

*DenseFeatureDetector***<**

*std::vector***>), we can use SURF, SIFT or any other descriptor extractors to get a description for these feature points.**

*cv::KeyPoint***<**

*std::vector***is the data structure for data passing.**

*cv::KeyPoint>*## Relationship between eigenvectors and directions of data distribution

For approaches based on dimension reduction, such as *Principle Component Analysis*, one critical question is “To which direction should we project the data?” Given observations of variable , we might compute the covariance matrix, and then use its eigenvectors as the direction of projection. Lots of approaches use this technique (e.g. Harris corner detector), but **WHY**? **What’s the reason behind**? This posts explains.

In the figure above, we illustrate a distribution of a 2D dataset. It’s already demeaned. The discussion below applies to any dimension size. Suppose each data point is -dimensional . Intuitively, we want to find linear functions of , which project into a new space. Since the data is already demeaned, which means the origin of the axes is now at the center of the data, and the linear function has no intercept part.

We start by defining the first linear function (or projection): , where is the coefficients of the linear function (see the figure above for 2D projection where . Looking at the data plot above, the line of should pass through the data points, and the projected points should have largest span. Mathematically speaking, we want to maximize the data variance after projecting them to a line , i.e. to find so that

**Example**: For 2D data, , . . It’s easy to show that the expectation . The variance

where is the covariance matrix of the original data. It is worth noticing that the finding can be extended to any higher dimensional variable . So right now our aim becomes

Without any constraint, we could always pick a larger so there’s no solution to . We choose normalized , which means .

**Solution**: Now our problem becomes a optimization problem subject to a constraint. We use Lagrange multipliers to maximize the function

with respect to . This results in

Suppose the solution to this equation is and .

It can be easily seen that the is the eigenvector of . **When the line (represented by ) is in the same direction as the eigenvector associated with largest eigenvalue, the variance will be the largest when we project the original data onto this line,** and **the variable is**:

Therefore, we should choose the eigenvector of the covariance matrix with largest eigenvalue as the basis of first dimension (first principle component), and the corresponding eigenvalue is indeed the variance of that projected data in that dimension.

__________________

Following the same deduction, we can conclude that the -th eigenvector is the $k$-th principle component, and . For example, for ,

(The second due to .)

Use Lagrange multiplier again, you will get

___________________

If you read here, I hopeit is clear now the reason why the eigenvector with largest eigenvalue coincide with the direction of biggest variance and why the eigenvectors of a covariance matrix are used as the directions of principle components.

Note: One more question in my mind is why we are so furtunate to have such a simple yet powerful solution, but I want to stop here today.

## Statistical Data Mining Basics

**Statistical Data Mining Tutorials**

**by Andrew Moore**

The following links point to a set of tutorials on many aspects of statistical data mining, including the foundations of probability, the foundations of statistical data analysis, and most of the classic machine learning and data mining algorithms.

## Articles on Doctoral Education

**A deeper look into the “80% of PhDs who do not become professors”**

**Doctorate recipients holding tenure and tenure-track appointments at academic institutions**

**Improving Graduate Education to Support a Branching Career Pipeline**

## [OPENGL]Some OpenGL Examples

Here are some OpenGL programs written for course assignments, from the very basic one to some advanced application. The codes are hosted on github：https://github.com/gccheng/openGL-tutorials

**1. Rotating pendulum
**

**2. Hand & fingers
**

**3. Solar System
**

基于线框模型的简单物体绘制与操作。

**4. Array operation (Rotation)
**绘制一个函数的图像，基于Array进行绘制。下面的图形是函数实时绘制的结果。

function from gc-cheng@hotmail.com on Vimeo.

**5. Quaternion algebra
**使用quaternion表示旋转、缩放等操作。由于使用了Quaternion，极大地简化了操作和代码：

planet-quaternion algebra from gc-cheng@hotmail.com on Vimeo.

**6. 分段贝塞尔(Bezier)曲线**

一段Bezier曲线是由四个点确定的，两段Bezier曲线是将7个点组成的一条连续可导的Bezier曲线，的原理可以查询Wikipedia中的文章。

实现代码：prog6_cheng.cpp

实现效果：