Linear models are generally considered the “old faithful” of the machine learning world.

Pros linear models:

Very resistant to over fitting
Simple
Very robust, and can handle messy outlier ridden data.

Cons linear models:

Can fail to understand more complicated relationships.

Regressors

Model Name	Summary	Link to sklearn-documentation
LinearRegression	The classic Ordinary Least Squares (OLS) model. It fits a linear model with coefficients to minimize the residual sum of squares between targets and predictions.	Documentation
Ridge	Linear least squares with L2 regularization. It imposes a penalty on the size of coefficients to prevent overfitting and handle multicollinearity.	Documentation
ARDRegression	Automatic Relevance Determination. A Bayesian approach that leads to sparse weights, effectively performing feature selection by driving irrelevant weights to zero.	Documentation
BayesianRidge	Estimates a probabilistic model of the regression problem. It is more robust to ill-posed problems by including regularization parameters directly in the estimation.	Documentation
ElasticNet	A linear regression with combined L1 and L2 priors as regularizers. Good for when multiple features are correlated with each other.	Documentation
GammaRegressor	A Generalized Linear Model (GLM) with a Gamma distribution. Ideal for modeling strictly positive, continuous targets with a right-skewed distribution.	Documentation
HuberRegressor	A robust regression model that is less sensitive to outliers than OLS. It uses the Huber loss, which is linear for high residual values and quadratic for small ones.	Documentation
Lars	Least Angle Regression. A model for high-dimensional data that works similarly to forward step-wise regression, providing the full path of coefficients.	Documentation
Lasso	Linear Model trained with L1 regularization. It encourages “sparsity,” meaning it can shrink some coefficients to exactly zero, performing feature selection.	Documentation
LassoLars	A Lasso model implemented using the LARS algorithm. Unlike the coordinate descent solver, it provides the exact entire path of the coefficients.	Documentation
LassoLarsIC	Lasso model fit with LARS using Information Criterion (AIC or BIC) for model selection, helping to find the optimal regularization parameter automatically.	Documentation
PassiveAggressiveRegressor	An incremental/online learning algorithm. It remains “passive” if the prediction is accurate but becomes “aggressive” to correct the model if the loss exceeds a threshold.	Documentation
PoissonRegressor	A GLM with a Poisson distribution. Used for modeling “count data” or frequency (non-negative integers).	Documentation
QuantileRegressor	Estimates the median or other quantiles of the target variable rather than the mean. Useful for understanding the distribution or when the error is not normal.	Documentation
RANSACRegressor	RANdom SAmple Consensus. An iterative method that fits a model to “inliers” while ignoring “outliers,” making it extremely robust to data noise.	Documentation
SGDRegressor	Linear model fitted by minimizing a loss function using Stochastic Gradient Descent. Best for large-scale datasets ( $n > 100, 000$ ) where OLS is too slow.	Documentation
TheilSenRegressor	A robust estimator that uses the generalization of the median of slopes. It is highly efficient on small outliers in both the target and the input space.	Documentation
TweedieRegressor	A GLM with a Tweedie distribution. It is a flexible model that can represent Poisson, Compound Poisson, Gamma, or Inverse Gaussian distributions.	Documentation

Classifiers

Model Name	Summary	Link to sklearn-documentation
LogisticRegression	A fundamental classification model that uses a logistic function to model a binary dependent variable. It can be extended to multiclass problems using the “one-vs-rest” or multinomial schemes.	Documentation
RidgeClassifier	Converts target values into ${- 1, 1}$ and treats the problem as a regression task with $L_{2}$ regularization. It is often faster than LogisticRegression for multiclass problems with many classes.	Documentation
PassiveAggressiveClassifier	Part of the family of online learning algorithms. It remains “passive” if a classification is correct and “aggressive” (updates weights) if it is incorrect or doesn’t meet a margin.	Documentation
Perceptron	One of the simplest online learning algorithms. It is a linear classifier that does not require a learning rate and is essentially an SGDClassifier with a `loss="perceptron"` and no regularization.	Documentation
SGDClassifier	An efficient estimator that implements regularized linear models (SVM, Logistic Regression) using Stochastic Gradient Descent. Highly recommended for large-scale datasets.	Documentation