BİLDİRİ DETAY

Hasan YILDIRIM, M.Revan ÖZKALE
A COMPARATIVE STUDY ON THE EFFECT OF REGULARIZATION PARAMETERS ON EXTREME LEARNING MACHINE BASED ON RIDGE REGRESSION
 
Extreme learning machine (ELM) has been extensively used in many fields owing to some superior properties such as simplicity, extremely fast training speed and pretty good generalization performance. However, it has some drawbacks in the case of multicollinearity. In order to find solution to multicollinearity, ELM based on ridge regression (RR-ELM) and ELM based on almost unbiased ridge regression (AUR-ELM) have been proposed. Since ridge constant, i.e regularization parameter, affects the performance of RR-ELM and AUR-ELM, we have investigated the effects of different selection criteria of ridge constant on this performance. Three most well-known criteria including AIC, BIC and cross validation have been considered in this study. The relative performance changes between ELM, RR-ELM and AUR-ELM in the sense of testing error and standart deviation of testing error based on each selection criterion of ridge constant have been given and interpreted. In consequence of applications performing on eight popular benchmark datasets, it has been shown that the generalization and stability performance of RR-ELM and AUR-ELM have been affected depending on the selection methods of ridge constant. Additionally, it has been seen that the type of ELM algorithm based on ridge regression and selection method for ridge constant should be determined carefully to improve the basic ELM algorithm.

Anahtar Kelimeler: Extreme Learning Machine, Ridge Regression, Almost Unbiased ridge Regression, Regularized Extreme Learning Machine, Model Selection



 


Keywords: