Proposed Methods for Preventing Overfitting in Machine Learning and Deep Learning
Vladimir Diukarev
Anti-Fraud Department, Sberbank, Moscow, Russian Federation.
Yaroslav Starukhin
McKinsey & Company, Boston, USA.
*Author to whom correspondence should be addressed.
Abstract
The article discusses various approaches and methods to combat overfitting, which is a key problem in the field of artificial intelligence. Overfitting occurs when the model over-adapts to the training dataset, losing the ability to generalize to new data. The main causes and signs of overfitting are also discussed, including excessive complexity of models and limited data. The focus is on methods such as regularization, dropout, and the use of ensemble methods that can significantly reduce the risk of overfitting. These approaches are evaluated using examples from various fields of neural network applications, providing the reader with a comprehensive understanding of the problem and its solution methods.
Keywords: Deep learning, overfitting, dropout, regularization, AI, artificial intelligence