Model aggregation methods and applications
Keywords:
Model Aggregation, Boosting; Bagging, Random Forest, StackingAbstract
Aggregation methods in machine learning models combine several assumptions made on the same dataset in order to obtain a predictive model with higher accuracy. They have been extensively studied and have led to numerous experimental and theoretical works in various contexts: classification, regression, unsupervised learning, etc. The aim of this work is in a first time reviewing several known models of aggregation and then compares their performances over two applications. The first is for making predictions on different databases, particularly in multiclass problems, and the second to use them in the context of estimating the density of a random variable.
Downloads
Download data is not yet available.
References
[1] Hastie, T., Tibshirani, R. y Friedman, J; The Elements of Statistical Learning, Springer-Verlag 2001, New York.
[2] Vapnik, V; Statistical Learning Theory, Wiley 1998, New York.
[3] Breiman, L., Friedman, J.,Olshen, R., y Stone, C; Classification and Regression Trees. Belmont, CA 1984: Chapman & Hall.
[4] Breiman, L; Bagging predictors, Machine Learning, 1996. 24(2): p. 123–140.
[5] Bradley, E y Tibshirani, R; An Introduction to the Bootstrap, Chapman & Hall/CRC 1993.
[6] Breiman, L; Random forests, Machine Learning, 2001. 45(1): p. 5–32.
[7] Freund, Y. y Schapire, E; A decision-theoretic generalization of on-line learning and application to boosting, Journal of Computer and System Sciences, 1997. 55(1): p 119-13.
[8] Friedman, J., Hastie, T. y Tibshirani, R; Additive logistic regression: a statistical view of boosting, Annals of Statistics, 2000. 28: p. 337–407.
[9] Breiman, L; Arcing Classifiers, The Annals of Statistics, 1998. 26(3): p. 801 – 849.
[10] Zhu, J., Zou, H. y Rosset, S., Hastie, T; Multi-class Adaboost, Statistics and its Interface, 2009. 2: p. 349 – 360.
[11] Saberian, M. J y Vasconcelos, N; Multiclass Boosting: Theory and Algorithms, Neural information Processing Systems, 2011.
[12] Mukherjee, I. y Schapire, R.E; A theory of multiclass boosting, Neural information Processing Systems, 2010.
[13] Wolpert, D.H; Stacked Generalization, Neural Networks, 1992. 5: p. 241–259.
[14] Breiman, L; Stacked regression, Machine Learning, 1996. 24(1): p. 49–64.
[15] Ting, K.M y Witten, I.H; Issues in Stacked Generalization, Journal of Artificial Intelligence Research, 1999. 10: p. 271 – 289.
[16] Smyth,P. y Wolpert, D.H; Linearly combining density estimators via stacking, Machine Learning, 1999. 36(1-2): p. 59 – 83.
[17] Bourel, M. y Ghattas, B; Aggregating density estimators: an empirical study, Enviado a publicación. 2012.
[18] Rigollet, P. y Tsybakov, A.B; Linear and convex aggregation of density estimators, Math. Methods Statist., 2007. 16(3): p. 260 – 280.
[19] Bourel, M; Apprentissage statistique par agrégation de modèles, tesis doctoral, Universidad de la República, Uruguay-Université d’Aix-Marseille, France. En curso.
[2] Vapnik, V; Statistical Learning Theory, Wiley 1998, New York.
[3] Breiman, L., Friedman, J.,Olshen, R., y Stone, C; Classification and Regression Trees. Belmont, CA 1984: Chapman & Hall.
[4] Breiman, L; Bagging predictors, Machine Learning, 1996. 24(2): p. 123–140.
[5] Bradley, E y Tibshirani, R; An Introduction to the Bootstrap, Chapman & Hall/CRC 1993.
[6] Breiman, L; Random forests, Machine Learning, 2001. 45(1): p. 5–32.
[7] Freund, Y. y Schapire, E; A decision-theoretic generalization of on-line learning and application to boosting, Journal of Computer and System Sciences, 1997. 55(1): p 119-13.
[8] Friedman, J., Hastie, T. y Tibshirani, R; Additive logistic regression: a statistical view of boosting, Annals of Statistics, 2000. 28: p. 337–407.
[9] Breiman, L; Arcing Classifiers, The Annals of Statistics, 1998. 26(3): p. 801 – 849.
[10] Zhu, J., Zou, H. y Rosset, S., Hastie, T; Multi-class Adaboost, Statistics and its Interface, 2009. 2: p. 349 – 360.
[11] Saberian, M. J y Vasconcelos, N; Multiclass Boosting: Theory and Algorithms, Neural information Processing Systems, 2011.
[12] Mukherjee, I. y Schapire, R.E; A theory of multiclass boosting, Neural information Processing Systems, 2010.
[13] Wolpert, D.H; Stacked Generalization, Neural Networks, 1992. 5: p. 241–259.
[14] Breiman, L; Stacked regression, Machine Learning, 1996. 24(1): p. 49–64.
[15] Ting, K.M y Witten, I.H; Issues in Stacked Generalization, Journal of Artificial Intelligence Research, 1999. 10: p. 271 – 289.
[16] Smyth,P. y Wolpert, D.H; Linearly combining density estimators via stacking, Machine Learning, 1999. 36(1-2): p. 59 – 83.
[17] Bourel, M. y Ghattas, B; Aggregating density estimators: an empirical study, Enviado a publicación. 2012.
[18] Rigollet, P. y Tsybakov, A.B; Linear and convex aggregation of density estimators, Math. Methods Statist., 2007. 16(3): p. 260 – 280.
[19] Bourel, M; Apprentissage statistique par agrégation de modèles, tesis doctoral, Universidad de la República, Uruguay-Université d’Aix-Marseille, France. En curso.
Downloads
Published
2012-10-01
How to Cite
[1]
M. Bourel, “Model aggregation methods and applications”, Memoria investig. ing. (Facultad Ing., Univ. Montev.), no. 10, pp. 19–32, Oct. 2012.
Issue
Section
Articles
