Solutions and StatsAPI

coef(cv::CVMultiLassoPath; λ = :λ1se)
predict(cv::CVMultiLassoPath, X; λ = :λ1se)

The coefficients (with intercepts in the first row when fitted), and the matrix of predictions at the rows of X, at cv.λ1se, at cv.λmin with λ = :λmin, or at a level within the path.

coef(cv::CVMultitaskLassoPath; λ = :λ1se)
predict(cv::CVMultitaskLassoPath, Xs; λ = :λ1se)

The coefficients (with the intercepts in the first row when the path has them), and the fitted values at the rows of the designs Xs[t] of the tasks, a vector per task, of the path on all the data at cv.λ1se, at cv.λmin with λ = :λmin, or at a level λ within the path.

coef(sol::GLMLassoSolution)
predict(sol::GLMLassoSolution, X; type = :response, offset = nothing)
fitted(sol::GLMLassoSolution)

The coefficients [β₀; β] (β without an intercept), as a SparseVector. The predictions at the rows of X, on the scale of the response (type = :response, the probabilities of logistic regression and the means of Poisson regression) or of the linear predictor β₀ + Xβ + offset (type = :link). fitted is the response scale at the data of the fit. nobs, dof (nonzero coefficients, plus one for the intercept), deviance and nulldeviance apply too.

coef(sol::GroupLassoSolution)
predict(sol::GroupLassoSolution, X)

The coefficients [β₀; β] (β without an intercept) as a SparseVector, and the fitted values β₀ .+ X * β at the rows of X. nobs and dof (the nonzero coefficients, plus one for the intercept) apply too.

coef(sol::LassoSolution)

The coefficients as a SparseVector: [β₀; β], the intercept first, when the solver fitted an intercept, and β otherwise, so that they line up with a model matrix whose first column is the constant.

coef(sol::MultiLassoSolution)
predict(sol::MultiLassoSolution, X)

The p × q coefficients B as a sparse matrix, with the intercepts of the responses in a first row when the solver fitted them, and the n × q fitted values 1β₀' + XB at the rows of X. nobs and dof (the nonzero coefficients, plus one intercept per response) apply too.

coef(r::PostLassoResult)
stderror(r::PostLassoResult)
vcov(r::PostLassoResult)
confint(r::PostLassoResult; level = 0.95)

The least-squares coefficients [β₀; β] (β without an intercept) and their HC1 standard errors as SparseVectors, zero off the selected columns, and their covariance as a sparse matrix. confint gives normal intervals coef ± z stderror at level, as a matrix with a row per coefficient; those off the selected columns are [0, 0], since the coefficients were not estimated. predict, residuals, nobs, dof (the coefficients estimated) and deviance (the residual sum of squares) apply too.

coef(sol::RLassoSolution)
predict(sol::RLassoSolution, X)

The coefficients [β₀; β] (β without an intercept) as a SparseVector, and the fitted values β₀ .+ X * β at the rows of X. residuals, nobs, dof (the selected columns, plus one for the intercept) and deviance (the residual sum of squares) apply too.

coef(sol::SparseGroupLassoSolution)
predict(sol::SparseGroupLassoSolution, X)
coef(path::SparseGroupLassoPath)
coef(path::SparseGroupLassoPath, λ)
predict(path::SparseGroupLassoPath, X)

As for the group lasso: the coefficients [β₀; β] (β without an intercept) and the fitted values β₀ .+ X * β, along the path as for LassoPath. nobs and dof (the nonzero coefficients, plus one for the intercept) apply too.

coef(te::TreatmentEffect)
stderror(te::TreatmentEffect)
vcov(te::TreatmentEffect)
confint(te::TreatmentEffect; level = 0.95)

The estimate of the treatment effect as a model with one coefficient: a vector of length one, its standard error, a 1 × 1 covariance using te.vcov_type, and the normal confidence interval at level as a 1 × 2 matrix (te.ci holds the one at the level of double_selection). nobs is the number of observations.

coef(path::MultiLassoPath)
coef(path::MultiLassoPath, λ)
predict(path::MultiLassoPath, X)

The coefficient matrices along the path, one per level (with the intercepts in the first row when the path has them), or the matrix at λ, interpolated linearly in λ between neighboring levels; and the matrices of fitted values at the rows of X, one per level.

coef(path::MultitaskLassoPath)
coef(path::MultitaskLassoPath, λ)
predict(path::MultitaskLassoPath, Xs)

The coefficient matrices along the path, one per level (with the intercepts in the first row when the path has them), or the matrix at λ, interpolated linearly in λ between neighboring levels; and the fitted values at the rows of the designs Xs[t] of the tasks, an nₜ × L matrix per task with a column per level.

coef(sol::MultitaskLassoSolution)
predict(sol::MultitaskLassoSolution, Xs)

The p × q coefficients B as a sparse matrix, a column per task, with the intercepts of the tasks in a first row when the solver fitted them, and the fitted values β₀ₜ .+ X_t * b_t at the rows of the designs Xs[t] of the tasks, a vector per task. nobs (the observations of all the tasks) and dof (the nonzero coefficients, plus one intercept per task) apply too.

coef(path::GLMLassoPath)
coef(path::GLMLassoPath, λ)
predict(path::GLMLassoPath, X; type = :response, offset = nothing)

The coefficients along the path (a sparse matrix, a column per level, the intercepts in the first row) or at λ (interpolated linearly in λ), as for LassoPath, and the predictions at the rows of X at every level, on the scale of the response or of the linear predictor (see predict for a GLMLassoSolution).

coef(path::GroupLassoPath)
coef(path::GroupLassoPath, λ)
predict(path::GroupLassoPath, X)

As for LassoPath.

coef(path::LassoPath)
coef(path::LassoPath, λ)

The coefficients along the path, as a sparse matrix with one column per value of path.λpath, whose first row holds the intercepts when the path has an intercept. With λ, the coefficients at λ as a SparseVector, interpolated linearly in λ between the two neighboring values of λpath (as glmnet's coef(fit, s = λ)), and exact at the values of λpath; λ must be within the range of λpath.

deviance(sol::LassoSolution)

The residual sum of squares ‖y - β₀ - X * β‖², weighted, ∑ᵢ wᵢ rᵢ², for a fit with observation weights.

dof(sol::LassoSolution)

The number of nonzero coefficients, plus one for an intercept: an unbiased estimate of the degrees of freedom of the lasso (Zou, Hastie and Tibshirani, On the "degrees of freedom" of the lasso, 2007). For the elastic net, MCP and SCAD it is the count that glmnet and ncvreg report, not an unbiased estimate.

fit(CVLassoPath, X, y, args...; kwargs...)
fit(GLMLassoPath, X, y, args...; kwargs...)
fit(CVGLMLassoPath, X, y, args...; kwargs...)
fit(GroupLassoPath, X, y, args...; kwargs...)
fit(SparseGroupLassoPath, X, y, args...; kwargs...)
fit(MultiLassoPath, X, Y, args...; kwargs...)
fit(CVMultiLassoPath, X, Y, args...; kwargs...)
fit(MultitaskLassoPath, Xs, ys, args...; kwargs...)
fit(CVMultitaskLassoPath, Xs, ys, args...; kwargs...)

The same as calling the constructor, for example GLMLassoPath(X, y, args...; kwargs...).

fit(LassoPath, X, y, args...; kwargs...)

The same as LassoPath(X, y, args...; kwargs...).

nobs(sol::LassoSolution)
nobs(path::LassoPath)

The number of observations the solver was given.

predict(sol::LassoSolution, X)

The fitted values β₀ .+ X * β at the rows of X, which has the columns of the data the solver was given (without a column for the intercept); an offset of the fit is not included.

predict(path::LassoPath, X)

The fitted values β₀ .+ X * β along the path: a matrix with one column per value of path.λpath.

residuals(sol::LassoSolution)

The residuals y - β₀ - X * β of the data the solver was given. A solution computed from sufficient statistics (scaled_lasso! with a CDSqrtQuadraticLoss) has none, and an error is thrown; so it is for nobs and deviance.

FormulaFit

A model fitted from a formula and a table. Loading StatsModels.jl adds formula methods to the estimators, which return it:

using HighDimensionalModels, StatsModels
path = fit(LassoPath, @formula(y ~ x1 + x2 + log(x3) + region), data)
sol = lasso(@formula(y ~ x1 + x2 + region), data, 0.1; standardize_x = true)
grp = group_lasso(@formula(y ~ x1 + region + x2 & region), data, 0.05)   # a group per term

The formula methods are fit with LassoPath, CVLassoPath, GLMLassoPath, CVGLMLassoPath, GroupLassoPath and SparseGroupLassoPath, and lasso, sqrt_lasso, adaptive_lasso, glm_lasso, group_lasso, sparse_group_lasso and rlasso with the formula and the table in place of X and y. The other arguments and keywords are those of the estimator, with these differences:

• The formula sets the intercept: y ~ x1 + x2 fits one, y ~ 0 + x1 + x2 does not, and the keyword intercept is an error.
• Categorical variables (strings, or CategoricalArrays) become dummy columns, coded by contrasts, a Dict from variable names to StatsModels contrasts. The lasso penalizes each dummy column; the group lasso and the sparse group lasso take the columns of each term as a group unless groups is given.
• weights and offset, where the estimator takes them, may also be names of columns of the table.
• Rows with a missing value in a variable of the model are left out.

model is the fitted model, a LassoPath, LassoSolution and so on, and formula the formula with the schema of the data applied. coef, nobs, dof, deviance, fitted, residuals, stderror, vcov and confint forward to the model, with their arguments: coef(m, λ) for a path. coefnames(m) names the coefficients in the order of coef(m), the intercept first, and responsename(m) the response. predict(m, newdata; kwargs...) builds the columns of the table newdata by the formula and predicts with the model.

coefnames(m::FormulaFit)
responsename(m::FormulaFit)

The names of the coefficients of a model fitted from a formula, in the order of coef(m) (the intercept first), and the name of its response.

DirtyMultitaskSolution

The dirty multitask fit of dirty_multitask. B, A and S are sparse p × q matrices on the data scale, with B = A + S. intercept holds the task intercepts, zero without hasintercept. λ_shared, λ_specific, shared_norm, task_weights and penalty_factors record the objective. activerows are the nonzero rows of B; sharedrows are those of the canonical split A, not identified shared features.

objective is the two-component objective. gap is the relative duality gap, or NaN when unavailable. kkt is the largest normalized violation of dual feasibility, the two Fenchel equalities, intercept stationarity, B = A + S and zeros outside the fitted coordinates, recomputed at the returned point. Each residual is divided by the maximum of one and the magnitudes of its terms. iterations counts row passes; converged requires the change rule and any requested, available gap target. nobs = sum(task_nobs) counts all tasks' observations, unlike multi_lasso's observation count. dof counts nonzero entries of B plus intercepts; it is not effective degrees of freedom.

coef(sol::DirtyMultitaskSolution)
predict(sol::DirtyMultitaskSolution, Xs)
predict(sol::DirtyMultitaskSolution, X)

Total coefficients B, with intercepts in the first row when fitted, and predictions as a vector per task or an n × q matrix for a shared design. nobs counts all task observations. dof counts nonzero entries of B plus fitted intercepts, not effective degrees of freedom. Paths support coef and predict at every level; CV uses its selected full-data solution.

SparseNuclearSolution

A fit of ‖Y - 1β₀' - XB‖²_F/(2n) + λ_rank ‖B‖_* + λ_sparse P(B). B, intercept, residuals, objective, activerows and nnz describe the returned coefficients. weights are positive row weights; sparsity is :entry or :row. rank counts singular values above the reported rank_tol, which also accounts for disagreement with the nuclear proximal point. gap is the relative duality gap, NaN for unpenalized least squares. iterations and converged report solver termination; r_norm, s_norm, eps_pri and eps_dual retain its KKT residuals and tolerances. coef, predict, residuals, nobs and deviance apply. No degrees of freedom or information criterion is provided for this estimator.

LowRankPlusSparseSolution

A fit of ‖Y - 1β₀' - X(L + S)‖²_F/(2n) + λ_rank ‖L‖_* + λ_sparse P(S). L, S and B = L + S are independent matrices. intercept, residuals, objective, coef and predict use the total B. weights are positive predictor weights, and sparsity is :entry or :row.

rank counts singular values of B above rank_tol = √eps(T) σ₁(B). lowrank_rank is the rank kept by the last nuclear proximal step (the numerical rank of L on a zero-penalty axis). sparse_nnz and sparse_rows describe the exact support of S. gap is the relative intersection-dual gap, up to floating-point accuracy. lowrank_residual and sparse_residual are Frobenius norms of the component proximal-gradient mappings at the returned pair. iterations and converged describe termination. On either zero-penalty axis, a minimum-norm least-squares convention applies and gap = NaN.

Fitted values are unique; components need not be. A small gap bounds objective error, not component error. coef, predict, residuals, nobs and deviance apply. No degrees of freedom or information criterion is provided for this estimator.

TransferMultitaskSolution

The joint-sparse transfer fit of transfer_multitask, corrected by debias when correction is not nothing. Domain 0 is the target.

• W: the reported p × K target slopes, the fused Ŵᶠ = ∑ₗ (nₗ/N) B̂ˡ or the corrected Ŵᴰ = Ŵᶠ + D̂; intercept: the target prediction intercepts ȳ⁰ - x̄⁰'W, zero without hasintercept.
• fused: Ŵᶠ; correction: D̂, or nothing.
• B: the domain blocks B̂⁰, …, B̂ᴸ; contrasts: B̂ˡ - B̂⁰ for the sources; domain_intercepts: ȳˡ - x̄ˡ'B̂ˡ per domain.
• λ₀, λ̃ (NaN without a correction), contrast_weights (the values a₁, …, a_L used), nobs (the target size n₀), domain_nobs (n₀, …, n_L) and solver.
• The fused stage: objective (the objective of transfer_multitask at the returned blocks), gap (the global relative duality gap there, NaN when unavailable), iterations and converged. With solver = :admm, primal_residual and dual_residual are the largest constraint residuals of the last iteration and penalties the final ADMM penalties ρ₀₀, …, ρ₀ᴸ, ρ₁, …, ρ_L; they are NaN and empty for coordinate descent.
• The correction stage: correction_objective, correction_gap, correction_iterations and correction_converged, the same for the correction problem; NaN, NaN, 0 and false without a correction.

coef, predict and nobs apply to it.

coef(sol::TransferMultitaskSolution; domain = nothing)
predict(sol::TransferMultitaskSolution, X; domain = nothing)

The target coefficients W (fused or corrected), with the target intercepts ȳ⁰ - x̄⁰'W in a first row when fitted, and target predictions, an n × K matrix. X is one design or a vector of K designs with the same rows. With domain = ℓ, both use that domain's block B̂ˡ and intercepts instead (domain = 0 gives B̂⁰, not W). nobs is the target size n₀. Paths give a result per level, and CV uses its solution.