EFFECT OF OUTLIERS ON CLASSICAL VS. ROBUST REGRESSION TECHNIQUES

Abstract

This study investigates the effect of outliers on the performance of Ordinary Least Squares (OLS) and Robust Regression techniques using simulated data. A series of simulations was conducted under varying levels of outlier contamination (0%, 5%, 10%, 20%, and 30%) to evaluate model performance in terms of bias, Root Mean Square Error (RMSE), and the coefficient of determination (R²). Results showed that both methods performed efficiently with clean data, producing nearly unbiased slope estimates (−0.000815 for OLS and −0.000727 for Robust) and very low RMSE values (0.0124 for OLS and 0.0125 for Robust) with high explanatory power (R² = 0.971 for both). However, as contamination increased, OLS became highly unstable, with RMSE rising sharply from 0.0554 at 5% contamination to 0.116 at 30%, while R² dropped from 0.611 to 0.266. Robust Regression demonstrated greater stability, maintaining lower RMSE values (0.0140 at 5% and 0.0910 at 30%) and smaller bias across all levels. Nonetheless, its explanatory power declined more rapidly, with R² falling from 0.595 at 5% contamination to 0.207 at 30%. Overall, the findings highlight that OLS is efficient with clean data but highly sensitive to outliers, whereas Robust Regression provides more reliable estimates and lower error rates in contaminated datasets, albeit with reduced explanatory power.