This function provids an illustration of the process of finding out the optimum number of variables using k-fold cross-validation in a linear discriminant analysis (LDA).
For a classification problem, usually we wish to use as less variables as possible because of difficulties brought by the high dimension.
The selection procedure is like this:
- Split the whole data randomly into
\(k\)folds: - For the number of features
\(g = 1, 2, \cdots, g_{max}\), choose\(g\)features that have the largest discriminatory power (measured by the F-statistic in ANOVA): - For the fold
\(i\)(\(i = 1, 2, \cdots, k\)): - Train a LDA model without the
\(i\)-th fold data, and predict with the\(i\)-th fold for a proportion of correct predictions\(p_{gi}\); - Average the
\(k\)proportions to get the correct rate\(p_g\); - Determine the optimum number of features with the largest
\(p\).
Note that \(g_{max}\) is set by ani.options('nmax') (i.e. the
maximum number of features we want to choose).
library(animation)
ani.options(nmax = 10)
par(mar = c(3, 3, 0.2, 0.7), mgp = c(1.5, 0.5, 0))
cv.nfeaturesLDA(pch = 19)
## Loading required namespace: MASS
This animation provides an illustration of the process of finding out the optimum number of variables using k-fold cross-validation in a linear discriminant analysis (LDA).
Donate
As a freelancer (currently working as a contractor) and a dad of three kids, I truly appreciate your donation to support my writing and open-source software development! Your contribution helps me cope with financial uncertainty better, so I can spend more time on producing high-quality content and software. You can make a donation through methods below.
-
Venmo:
@yihui_xie, or Zelle:xie@yihui.name -
Paypal
-
If you have a Paypal account, you can follow the link https://paypal.me/YihuiXie or find me on Paypal via my email
xie@yihui.name. Please choose the payment type as “Family and Friends” (instead of “Goods and Services”) to avoid extra fees. -
If you don’t have Paypal, you may donate through this link via your debit or credit card. Paypal will charge a fee on my side.
-
-
Other ways:
WeChat Pay (微信支付：谢益辉) Alipay (支付宝：谢益辉) 
When sending money, please be sure to add a note “gift” or “donation” if possible, so it won’t be treated as my taxable income but a genuine gift. Needless to say, donation is completely voluntary and I appreciate any amount you can give.
Please feel free to email me if you prefer a different way to give. Thank you very much!
I’ll give back a significant portion of the donations to the open-source community and charities. For the record, I received about $30,000 in total (before tax) in 2024-25, and gave back about $15,000 (after tax).