Unit–II: Correlation and Regression

1. Meaning, Definition, and Use of Correlation

Correlation is a statistical technique used to measure and analyze the degree and direction of the linear relationship between two or more variables. It determines whether a change in one variable is accompanied by a corresponding change in another.

CORRELATION ANALYSIS
|
+———————————–+———————————–+
| |
DIRECTION OF RELATIONSHIP DEGREE OF RELATIONSHIP
Positive (move in same direction) or Ranging from −1.00 (perfect negative)
Negative (move in opposite directions) to +1.00 (perfect positive)
 

Key Uses of Correlation

  • Business Forecasting: Helps managers estimate demand, sales, and inventory levels based on economic indicators.
  • Policy Formation: Assists in understanding macro-level economic relationships (e.g., price vs. demand, inflation vs. unemployment).
  • Scientific Research: Identifies associations between physical or social variables prior to establishing cause-and-effect models.

2. Covariance and Scatter Diagram

Covariance (Cov(X,Y))

Covariance measures the joint variability of two random variables. It indicates the directional relationship between variables, but its magnitude depends on the units of measurement.

Cov(X,Y) = Σ(X − X̄)(Y − Ȳ) / N
  • Positive Covariance: Both variables tend to increase or decrease together.
  • Negative Covariance: One variable tends to increase while the other decreases.
  • Zero Covariance: No linear relationship exists between the variables.

Scatter Diagram

A Scatter Diagram is a graphical method where paired observations (Xi, Yi) are plotted as points on a Cartesian plane to visually assess the nature of correlation.

TYPES OF SCATTER DIAGRAMS
|
+——————–+————–+————–+——————–+
| | | | |
PERFECT POSITIVE STRONG POSITIVE NO CORRELATION STRONG NEGATIVE PERFECT NEGATIVE
Points form line Points trend Points scattered Points trend Points form line
sloping UP right UP to right randomly DOWN to right sloping DOWN right
(r = +1) (0 < r < 1) (r = 0) (−1 < r < 0) (r = −1)
 

3. Types of Correlation

TYPES OF CORRELATION
|
+———————–+———–+———–+———————–+
| | | |
DIRECTION LINEAITY NUMBER OF VARIABLES STRENGTH
• Positive • Linear • Simple (2 variables) • Perfect (r = ±1)
• Negative • Non-Linear (Curvilinear) • Multiple / Partial • High / Moderate / Low
 
Type Operational Definition Real-World Example
Positive Correlation Variables move in the same direction. Advertising expenditure and sales revenue.
Negative Correlation Variables move in opposite directions. Price of a commodity and market demand.
Linear Correlation Ratio of change between variables remains constant. Constant marginal costs leading to direct linear price adjustments.
Non-Linear (Curvilinear) Ratio of change varies across data points. Yield response to varying fertilizer inputs.

4. Karl Pearson’s Correlation Coefficient (r)

Karl Pearson’s Coefficient of Correlation (also called product-moment correlation coefficient) measures the strength and direction of a linear relationship between two continuous variables.

r = Cov(X,Y) / (σx · σy) = Σ(X − X̄)(Y − Ȳ) / √[Σ(X − X̄)² · Σ(Y − Ȳ)²]

Alternative Shortcut Formula

r = [N ΣXY − (ΣX)(ΣY)] / √{[N ΣX² − (ΣX)²][N ΣY² − (ΣY)²]}

Key Properties

  • Range: −1 ≤ r ≤ +1.
  • Pure Number: Independent of scale and origin changes.
  • Symmetric: rxy = ryx.

5. Spearman’s Rank Correlation Coefficient (rs or ρ)

Used when data is qualitative or ranked (e.g., beauty contests, leadership ratings, honesty, intelligence).

rs = 1 − [6 Σd²] / [N(N² − 1)]
  • Where d = R₁ − R₂ (difference in ranks for each pair).
  • N = Number of paired observations.

Adjustment for Tied Ranks

When ranks are repeated, an adjustment factor m(m² − 1)/12 is added to Σd² for each repeated rank group of size m:

rs = 1 − 6[ Σd² + Σ m(m² − 1)/12 ] / [N(N² − 1)]

6. Probable Error (PE) & Standard Error (SE)

Probable Error (PE) measures the reliability and statistical significance of the calculated correlation coefficient (r).

Standard Error (SEr) = (1 − r²) / √N
PEr = 0.6745 × SEr = 0.6745 × (1 − r²) / √N

Decision Rules for Significance

  • If r < PEr: Correlation is insignificant (no evidence of linear relationship).
  • If r > 6 × PEr: Correlation is significant and reliable.

7. Regression: Meaning and Utility of Regression Analysis

Regression Analysis is a statistical method used to estimate or predict the unknown value of a dependent variable (Y) based on the known value of an independent variable (X).

REGRESSION ANALYSIS UTILITY
|
+——————-+—————-+—————-+——————-+
| | | | |
VALUE PREDICTION CAUSAL MODELING POLICY MAKING ERROR REDUCTION TREND ANALYSIS
Estimating Y from Establishing Setting targets Measuring standard Forecasting business
known X values direction of effect using estimates error of estimate growth trajectory
 

8. Comparison Between Correlation and Regression

Feature Correlation Analysis Regression Analysis
Core Purpose Measures degree and direction of linear relationship. Predicts the value of a dependent variable from an independent variable.
Variable Nature Both variables are treated symmetrically. Distinguishes between dependent (Y) and independent (X) variables.
Cause & Effect Does not imply cause and effect. Reflects functional cause-and-effect relationships.
Units of Measurement Unitless relative measure (−1 to +1). Expressed in physical units of the dependent variable.
Symmetry Symmetric (rxy = ryx). Asymmetric (byx ≠ bxy in general).

9. Regression Lines, Equations, and Coefficients

There are two regression lines, representing the line of best fit calculated using the Principle of Least Squares.

REGRESSION LINES
|
+———————————–+———————————–+
| |
LINE OF Y ON X LINE OF X ON Y
Predicts Y given X Predicts X given Y
Equation: (Y − Ȳ) = byx(X − X̄) Equation: (X − X̄) = bxy(Y − Ȳ)
 

Regression Equations & Coefficients

Regression Line of Y on X:

(Y − Ȳ) = byx(X − X̄)

Where byx = Cov(X,Y)/σx² = r · (σy/σx) = [N ΣXY − (ΣX)(ΣY)] / [N ΣX² − (ΣX)²]

Regression Line of X on Y:

(X − X̄) = bxy(Y − Ȳ)

Where bxy = Cov(X,Y)/σy² = r · (σx/σy) = [N ΣXY − (ΣX)(ΣY)] / [N ΣY² − (ΣY)²]

Key Properties of Regression Coefficients (byx and bxy)

  • Relationship to Correlation: Correlation coefficient is the geometric mean of two regression coefficients:
    r = ±√(byx · bxy)
  • Sign Consistency: Both regression coefficients and the correlation coefficient share the same sign (all positive or all negative).
  • Magnitude Rule: If one regression coefficient is greater than 1, the other must be less than 1 (since r² ≤ 1).
  • Intersection Point: The two regression lines always intersect at the mean point (X̄, Ȳ).

Key Learning Summary

Section Core Operational Takeaway
Correlation (r) Quantifies strength and direction of linear association (−1 to +1).
Spearman’s rs Applies to qualitative data or rank-ordered observations.
Probable Error (PE) Confirms statistical significance (r > 6 × PEr).
Regression (byx, bxy) Enables predictive modeling and estimation using least squares.
Geometric Property r = ±√(byx · bxy).

 

PreviousNext