Comment: Boosting Algorithms: Regularization, Prediction and Model Fitting (2008)
Comment on ``Boosting Algorithms: Regularization, Prediction and Model Fitting'' [arXiv:0804.2752]
Discussion: The Dantzig selector: Statistical estimation when $p$ is much larger than $n$ (2008)
Efron, Bradley, Hastie, Trevor, Tibshirani, Robert
Discussion of ``The Dantzig selector: Statistical estimation when $p$ is much larger than $n$'' [math/0506081]
On the "degrees of freedom" of the lasso (2007)
Zou, Hui, Hastie, Trevor, Tibshirani, Robert
We study the effective degrees of freedom of the lasso in the framework of Stein's unbiased risk estimation (SURE). We show that the number of nonzero coefficients is an unbiased estimate for the...
Buess, Martin, Nuyten, Dimitry SA, Hastie, Trevor, Nielsen, Torsten, Pesich, Robert, Brown, Patrick O
Abstract Background Perturbations in cell-cell interactions are a key feature of cancer. However, little is known about the systematic effects of cell-cell interaction on global gene expression in...
Sparse inverse covariance estimation with the lasso (2007)
Friedman, Jerome, Hastie, Trevor, Tibshirani, Robert
We consider the problem of estimating sparse graphs by a lasso penalty applied to the inverse covariance matrix. Using a coordinate descent procedure for the lasso, we develop a simple algorithm that...
Pathwise coordinate optimization (2007)
Friedman, Jerome, Hastie, Trevor, Höfling, Holger, Tibshirani, Robert
We consider ``one-at-a-time'' coordinate-wise descent algorithms for a class of convex optimization problems. An algorithm of this kind has been proposed for the $L_1$-penalized regression (lasso) in...
Forward stagewise regression and the monotone lasso (2007)
Hastie, Trevor, Taylor, Jonathan, Tibshirani, Robert, Walther, Guenther
We consider the least angle regression and forward stagewise algorithms for solving penalized least squares regression problems. In Efron, Hastie, Johnstone & Tibshirani (2004) it is proved that the...
"Pre-conditioning" for feature selection and regression in high-dimensional problems (2007)
Paul, Debashis, Bair, Eric, Hastie, Trevor, Tibshirani, Robert
We consider regression problems where the number of predictors greatly exceeds the number of observations. We propose a method for variable selection that first estimates the regression function,...
Comment on "Support Vector Machines with Applications" (2006)
Comment on [math.ST/0612817]
Jen-Tsan Chi, Zhen Wang, Dimitry S. A. Nuyten, Edwin H. Rodriguez, Marci E. Schaner, Ali Salim, ...
The transcriptional response to hypoxia varies between cell types. A gene-expression signature of the cellular response to hypoxia is associated with a significantly poorer prognosis in breast and...
Saharon Rosset, Ji Zhu, Trevor Hastie
In this paper we study boosting methods from a new perspective. We build on recent work by Efron et al. to show that boosting approximately (and in some cases exactly) minimizes its loss criterion...
Efron, Bradley, Hastie, Trevor, Johnstone, Iain, Tibshirani, Robert
The purpose of model selection algorithms such as All Subsets, Forward Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be...
Rejoinder to "Least angle regression" by Efron et al (2004)
Efron, Bradley, Hastie, Trevor, Johnstone, Iain, Tibshirani, Robert
Rejoinder to ``Least angle regression'' by Efron et al. [math.ST/0406456]
Automatic Bayes Carpentry Using Unlabeled Data In Semi-Supervised Classification (2004)
Hui Zou, Ji Zhu, Trevor Hastie
In semi-supervised classification one important question is how to use the enormous amount of unlabeled data to improve the classifier built on the labeled data. We point out that under certain...
A Method for Inferring Label Sampling Mechanisms In Semi-Supervised Learning (2004)
Saharon Rosset, Ji Zhu, Hui Zou, Trevor Hastie
We consider the situation in semi-supervised learning, where the "label sampling" mechanism stochastically depends on the true response (as well as potentially on the features). We suggest a method...
The Entire Regularization Path for the Support (2004)
Trevor Hastie, Saharon Rosset, Robert Tibshirani, Ji Zhu
In this paper we argue that the choice of the SVM cost parameter can be critical. We then derive an algorithm that can fit the entire path of SVM solutions for every value of the cost parameter, with...
Printed in Great Britain (2004)
Trevor Hastie, Robert Tibshirani
this article we expose a class of techniques based on quadratic regularization of linear models, including regularized (ridge) regression, logistic and multinomial regression, linear and mixture...
Sparse Principal Component Analysis (2004)
Hui Zou, Trevor Hastie, Robert Tibshirani
Principal component analysis (PCA) is widely used in data processing and dimensionality reduction. However, PCA su#ers from the fact that each principal component is a linear combination of all the...
Regression Shrinkage and Selection via the (2004)
We propose the elastic net, a new regression shrinkage and selection method. Real data and a simulation study show that the elastic net often outperforms the lasso, while it enjoys a similar sparsity...
Efron, Bradley, Hastie, Trevor, Johnstone, Iain, Tibshirani, Robert
The purpose of model selection algorithms such as All Subsets, Forward Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be...
The Entire Regularization Path for the Support Vector (2004)
Trevor Hastie, Saharon Rosset, Rob Tibshirani, Ji Zhu
The Support Vector Machine is a widely used tool for classification. Many e#cient implementations exist for fitting a two-class SVM model. The user has to supply values for the tuning parameters: the...
Margin Maximizing Loss Functions (2004)
Saharon Rosset, Ji Zhu, Trevor Hastie
Margin maximizing properties play an important role in the analysis of classi- cation models, such as boosting and support vector machines. Margin maximization is theoretically interesting because it...
norm Support Vector Machines (2004)
Ji Zhu, Saharon Rosset, Trevor Hastie, Rob Tibshirani
The standard 2-norm SVM is known for its good performance in twoclass classication. In this paper, we consider the 1-norm SVM. We argue that the 1-norm SVM may have some advantage over the standard...
Discussions of boosting papers, and rejoinders (2004)
Bartlett, Peter L., Bickel, Peter J., Bühlmann, Peter, Freund, Yoav, Friedman, Jerome, Hastie, Trevor, ...
Discussions of: "Process consistency for AdaBoost" [Ann. Statist. 32 (2004), no. 1, 13-29] by W. Jiang; "On the Bayes-risk consistency of regularized boosting methods" [ibid., 30-55] by G. Lugosi and...
Kernel Logistic Regression and the Import Vector Machine Ji Zhu (2004)
The support vector machine (SVM) is known for its good performance in binary classification, but its extension to multi-class classification is still an on-going research issue. In this paper, we...
Expression Arrays and the p (2003)
Trevor Hastie, Robert Tibshirani
Gene expression arrays typically have 50 to 100 samples and 5,000 to 20,000 variables (genes). There have been many attempts to adapt statistical models for regression and classification to these...
1-norm Support Vector Machines (2003)
Ji Zhu, Saharon Rosset, Trevor Hastie, Rob Tibshirani
The standard 2-norm SVM is known for its good performance in twoclass classification. In this paper, we consider the 1-norm SVM. We argue that the 1-norm SVM may have some advantage over the standard...
Margin Maximizing Loss Functions (2003)
Saharon Rosset, Ji Zhu, Trevor Hastie
Margin maximizing properties play an important role in the analysis of classification models, such as boosting and support vector machines. Margin maximization is theoretically interesting because it...
Boosting as a Regularized Path to a Maximum (2003)
Saharon Rosset, Ji Zhu, Trevor Hastie
In this paper we study boosting methods from a new perspective. We build on recent work by Efron et al. to show that boosting approximately (and in some cases exactly) minimizes its loss criterion...
Francesca Dominici, Aidan Mcdermott, Trevor Hastie
In 2002, methodological issues around time series analyses of air pollution and health attracted the attention of statisticians, epidemiologists, the press, industry, and policy makers. As the...
Class Prediction by Nearest Shrunken Centroids, with Applications to DNA Microarrays (2003)
Tibshirani, Robert, Hastie, Trevor, Narasimhan, Balasubramanian, Chu, Gilbert
We propose a new method for class prediction in DNA microarray studies based on an enhancement of the nearest prototype classifier. Our technique uses "shrunken" centroids as prototypes for each...
Zhao, Hongjuan, Hastie, Trevor, Whitfield, Michael L, Børresen-Dale, Anne-Lise, Jeffrey, Stefanie S
Abstract Background T7 based linear amplification of RNA is used to obtain sufficient antisense RNA for microarray expression profiling. We optimized and systematically evaluated the fidelity and...
Thomas D. Wu, Scott C. Schmidler, Trevor Hastie, Section On Medical Informatics
this paper, we take a di#erent approach to analyzing multiple protein structures. We assume that variations in protein structure can be represented by a statistical model. We solve that model to...
Functional Analysis of Human Motion Data (2002)
Dirk Ormoneit, Trevor Hastie, Michael Black
We present a method for the modeling of 3D human motion data using functional analysis. First, we estimate a statistical model of typical activities from a large set of 3D human motion data. For this...
Olga Troyanskaya, Gavin Sherlock, Trevor Hastie, Robert Tibshirani, David Botstein, Russ B. Altman
Motivation: Gene expression microarray experiments can generate data sets with multiple missing expression values. Unfortunately, many algorithms for gene expression analysis require a complete...
Kernel Logistic Regression and the Import Vector Machine (2002)
The support vector machine (SVM) is known for its good performance in binary classification, but its extension to multi-class classification is still an on-going research issue. In this paper, we...
Bradley Efron, Trevor Hastie, Lain Johnstone, Robert Tibshirani
The purpose of model selection algorithms such as All Subsets, Forward Selection, and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be...
Kernel Logistic Regression and the Import Vector Machine (2001)
The support vector machine (SVM) is known for its good performance in binary classification, but its extension to multi-class classification is still an on-going research issue. In this paper, we...
Imputing Missing Data for Gene Expression Arrays (2001)
Trevor Hastie, Robert Tibshirani, Gavin Sherlock, Michael Eisen, Patrick Brown, David Botstein
this report. 1.2 SVD imputation using all the data
Exploratory Screening of Genes and Clusters (2001)
Robert Tibshirani, Trevor Hastie, Balasubramanian Narasimhan, Michael Eisen, Gavin Sherlock, Pat Brownil
We discuss a method called "cluster scoring" for supervised learning from a set of gene expression experiments. Cluster scoring generalizes methods that rank individual genes based on their...
Missing value estimation methods for DNA microarrays (2001)
Olga Troyanskaya, Gavin Sherlock, Pat Brown, Trevor Hastie, Robert Tibshirani, David Botstein, ...
Motivation: Gene expression microarray experiments can generate data sets with multiple missing expression values. Unfortunately, many algorithms for gene expression analysis require a complete...
Supervised harvesting of expression trees (2001)
Hastie, Trevor, Tibshirani, Robert, Botstein, David, Brown, Patrick
Abstract Background We propose a new method for supervised learning from gene expression data. We call it 'tree harvesting'. This technique starts with a hierarchical clustering of genes, then models...
Degrees of Freedom Tests for Smoothing Splines (2000)
When using smoothing splines to estimate a function, the user faces the problem of choosing the smoothing parameter. Several techniques are available to select this parameter according to certain...
Handwritten Digit Recognition via Deformable Prototypes (2000)
Trevor Hastie, Robert Tibshirani
We present a new method for classifying handwritten characters, in particular digits. In our approach each character in the alphabet is represented by a prototype, in particular a piecewise-linear...
Supervised Harvesting of Expression Trees (2000)
Trevor Hastie, Robert Tibshirani, David Botstein, Patrick Brown
Background We propose a new method for supervising learning from gene expression data. We call it Tree Harvesting". This technique starts with a hierarchical clustering of genes, and models the...
Hastie, Trevor, Tibshirani, Robert, Eisen, Michael B, Alizadeh, Ash, Levy, Ronald, Staudt, Louis, ...
Abstract Background Large gene expression studies, such as those conducted using DNA arrays, often provide millions of different pieces of data. To address the problem of analyzing such data, we...
Bayesian backfitting (with comments and a rejoinder by the authors (2000)
Hastie, Trevor, Tibshirani, Robert
We propose general procedures for posterior sampling from additive and generalized additive models. The procedure is a stochastic generalization of the well-known backfitting algorithm for fitting...
Penalized Discriminant Analysis (2000)
Trevor Hastie, Andreas Buja, Robert Tibshirani
Fisher's linear discriminant analysis (LDA) is a popular data-analytic tool for studying the relationship between a set of predictors and a categorical response. In this paper we describe a penalized...
Flexible Discriminant Analysis by Optimal Scoring (2000)
Trevor Hastie, Robert Tibshirani, Andreas Buja
Fisher's linear discriminant analysis is a valuable tool for multigroup classication. With a large number of predictors, one can nd a reduced number of discriminant coordinate functions that are...
Flexible Discriminant Analysis by Optimal Scoring (2000)
Trevor Hastie, Robert Tibshirani, Andreas Buja
Fisher's linear discriminant analysis is a valuable tool for multigroup classi cation. With a large number of predictors, one can nd a reduced number of discriminant coordinate functions that are...
Penalized Discriminant Analysis (2000)
Trevor Hastie, Andreas Buja, Robert Tibshirani
Fisher's linear discriminant analysis (LDA) is a popular data-analytic tool for studying the relationship between a set of predictors and a categorical response. In this paper we describe a penalized...
Degrees of Freedom Tests for Smoothing Splines (2000)
When using smoothing splines to estimate a function, the user faces the problem of choosing the smoothing parameter. Several techniques are available to select this parameter according to certain...
Getting the Mean Right is a Good Thing: Generalized Additive Models (2000)
Nathaniel Beck, Simon Jackman, We Thank Michael Dimock, Gary Jacobson, John Oneal For, Their Data, ...
i 1 Introduction 1 2 Generalized Additive Models 2 2.1 Interpreting GAMs . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Contrasting other non-parametric approaches . . . . . . . . 4 3...
Metrics and Models for Handwritten Character Recognition (2000)
Trevor Hastie, Patrice Y. Simard
A digitized handwritten numeral can be represented as a binary or greyscale image. An important pattern recognition task that has received much attention lately is to automatically determine the...
Learning Prototype Models for Tangent Distance (2000)
Trevor Hastie, Patrice Simard, Eduard Sackinger
Simard, LeCun & Denker (1993) showed that the performance of near-neighbor classification schemes for handwritten character recognition can be improved by incorporating invariance to specific...
Optimal Kernel Shapes for Local Linear (2000)
Local linear regression performs very well in many low-dimensional forecasting problems. In high-dimensional spaces, its performance typically decays due to the well-known "curse-of-dimensionality".
Discriminant Adaptive Nearest Neighbor Classification (2000)
Trevor Hastie, Robert Tibshirani
Nearest neighbor classification expects the class conditional probabilities to be locally constant, and suffers from bias in high dimensions. We propose a locally adaptive form of nearest neighbor...
Friedman, Jerome, Hastie, Trevor, Tibshirani, Robert
Boosting is one of the most important recent developments in classification methodology. Boosting works by sequentially applying a classification algorithm to reweighted versions of the training data...
Tree-based models provide an alternative to linear models for classication and regression data. They are used primarily for exploratory analysis of complex data or as a diagnostic tool following a...
Principal Curves and Surfaces (2000)
Principal curves are smooth one dimensional curves that pass through the middle of a p dimensional data set. They minimize the distance from the points, and provide a non-linear summary of the data....
Clustering methods for the analysis of DNA microarray data (1999)
Robert Tibshirani, Trevor Hastie, Mike Eisen, Doug Ross, David Botstein, Pat Brown
It is now possible to simultaneously measure the expression of thousands of genes during cellular differentiation and response, through the use of DNA microarrays. A major statistical task is to...
Clustering methods for the analysis of DNA microarray data (1999)
Robert Tibshirani, Trevor Hastie, Mike Eisen, Doug Ross, David Botstein, Pat Brown
It is now possible to simultaneously measure the expression of thousands of genes during cellular differentiation and response, through the use of DNA microarrays. A major statistical task is to...
Robert Tibshirani, Trevor Hastie, Mike Eisen, Doug Ross, David Botstein, Pat Brown
It is now possible to simultaneously measure the expression of thousands of genes during cellular differentiation and response, through the use of DNA microarrays. A major statistical task is to...
Learning Prototype Models for Tangent Distance (1999)
Trevor Hastie, Patrice Simard, Eduard Sackinger
Simard, LeCun & Denker #1993# showed that the performance of near-neighbor classi#cation schemes for handwritten character recognition can be improved by incorporating invariance to speci #c...
Additive Logistic Regression: a Statistical View of Boosting (1999)
Jerome Friedman, Trevor Hastie, Robert Tibshirani
Boosting (Freund & Schapire 1996, Schapire & Singer 1998) is one of the most important recent developments in classification methodology. The performance of many classification algorithms can often...
Principal Component Models for Sparse Functional Data (1999)
Data often arrives as curves --- functions sampled at regular times or frequencies. Functional principal components (Ramsay and Silverman, 1997) can be used to describe the modes of variation of...
Principal Component Models for Sparse Functional Data (1999)
Data often arrives as curves --- functions sampled at regular times or frequencies. Functional principal components (Ramsay and Silverman, 1997) can be used to describe the modes of variation of...
Optimal Kernel Shapes for Local Linear (1999)
Local linear regression performs very well in many low-dimensional forecasting problems. In high-dimensional spaces, its performance typically decays due to the well-known "curse-of-dimensionality"....
Statistical Models for Image Sequences (1999)
Neil Crellin, Trevor Hastie, Iain Johnstone
Identification of brain activity using functional magnetic resonance imaging (fMRI) depends on blood flow replenishing activated neuronal sites. In this paper we describe how previous studies have...
Statistical Models for Image Sequences (1999)
Neil Crellin, Trevor Hastie, Iain Johnstone
Identification of brain activity using functional magnetic resonance imaging (fMRI) depends on blood flow replenishing activated neuronal sites. In this paper we describe how previous studies have...
The Global Pairwise Approach to Radiation Hybrid Mapping (1999)
Robert Tibshirani, Laura Lazzeroni, Trevor Hastie, Adam Olshen, David Cox
Introduction We propose a global pairwise method for constructing maps from radiation hybrid data. The method depends upon a novel statistical criterion for identifying the best map than it more...
The Global Pairwise Approach to Radiation Hybrid Mapping (1999)
Robert Tibshirani, Laura Lazzeroni, Trevor Hastie, Adam Olshen, David Cox
Introduction We propose a global pairwise method for constructing maps from radiation hybrid data. The method depends upon a novel statistical criterion for identifying the best map than it more...
Additive Logistic Regression: a Statistical View of Boosting (1998)
Jerome Friedman, Trevor Hastie, Robert Tibshirani
Boosting (Freund & Schapire 1995) is one of the most important recent developments in classification methodology. Boosting works by sequentially applying a classification algorithm to reweighted...
Generalized Additive Models, Cubic Splines and Penalized Likelihood. (1998)
Hastie,Trevor, Tibshirani,Robert
Generalized additive models extended the class of generalized linear models by allowing an arbitrary smooth function for any or all of the covariates. The functions are established by the local...
Trevor Hastie, Debra Ikeda, Robert Tibshirani
We propose a statistical method for finding masses on mammograms. The technique is based on fitting broken line regressions to local intensity plots of the images. The method is illustrated on a...
Additive Logistic Regression: a Statistical View of Boosting (1998)
Jerome Friedman, Trevor Hastie, Robert Tibshirani
Boosting (Freund & Schapire 1996, Schapire & Singer 1998) is one of the most important recent developments in classification methodology. The performance of many classification algorithms can often...