Prediction of reproducibility of effects for regressions based on top predictors

Cover Page

Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription or Fee Access

Abstract

Two simple methods for predicting the reproducibility of effects in test samples after multiple regression analysis of the discovery sample are proposed. In particular, the method allows us to assess the feasibility of constructing efficient polygenic risk indices (PRS, PGS). Using the theory of ordinal statistics, we obtained a simple analytical formula that estimates the coefficient of determination for the model constructed for the top neutral indices (). This is the coefficient of determination under the null hypothesis, which depends only on the sample size, the total number of indicators studied (e.g., snips or CpG methylation levels), and the number of top indicators chosen to construct the regression. Comparing the observed multiple correlation square for the discovery sample with . Allows a reasonably confident prediction of the reproducibility of effects in the test samples. If the observed correlation square for the discovery sample is 1.3 times , then at least half of the original correlation square can be expected in the test samples. The second method is based on a similar comparison of the maximum correlation coefficient for the discovery sample with the expected maximum correlation for neutral traits.

About the authors

A. V. Rubanovich

Vavilov Institute of General Genetics of the Russian Academy of Sciences

Author for correspondence.
Email: rubanovich@vigg.ru
Moscow, 119991 Russia

References

  1. Fan J., Lv J. A selective overview of variable selection in high dimensional feature // Space. Stat. Sin. 2010. V. 20(1). P. 101–148. PMID: 21572976; PMCID: PMC3092303
  2. Hastie T., Tibshirani Robert, Tibshirani Ryan. Best subset, forward stepwise or lasso? Analysis and recommendations based on extensive comparisons // Stat. Science. 2020. V. 35. № 4. Р. 579–592. https://doi.org/10.1214/19-STS733
  3. Christopher R., Genovese C.R., Jin J., Wasserman L. Revisiting marginal regression // arXiv:0911.4080v1 [math.ST]. 2009.
  4. Genovese C.R., Jin J., Wasserman L., Yao Z. A compa- rison of the lasso and marginal regression // J. Mach. Learn. Res. 2012. V. 13(1). P. 2107–2143.
  5. Leek J. Prediction: The lasso vs just using the top 10 predictors // Blog “Simply Statistics”. 2012. http://simplystatistics.tumblr.com/post/18132467723/ prediction-the-lasso-vs-just-using-the-top-10
  6. Freedman D.A. A note on screening regression equations // Am. Stat. 1983. V. 37. № 2. P. 152–155.
  7. Lukacs P.M., Burnham K.P., Anderson D.R. Model selection bias and Freedman’s paradox // Ann. Inst. Stat. Math. 2010. V. 62. P. 117–125. https://doi.org/10.1007/s10463-009-0234-4.
  8. Rubanovich A.V., Khromov-Borisov N.N. Genetic risk assessment of the joint effect of several genes: Cri- tical appraisal // Russ. J. Genet. 2016. V. 52. № 7. P. 757–769. https://doi.org/10.1134/S1022795416070073
  9. Alam К., Wallenius К. Distribution of a sum of order statistics // Scand. J. Stat. 1979. V. 6. № 3. P. 123–126.
  10. Rencher A.C., Pun F.C. Inflation of R2 in best subset regression // Technometrics. 1980. V. 22(1). P. 49–53. https://doi.org/10.2307/1268382
  11. Salt D.W., Ajmani S., Crichton R., Livingstone J.D. An improved approximation to the estimation of the cri- tical F values in best subset regression // J. Chem. Inf. Model. 2007. V. 47. № 1.
  12. Wray N.R., Yang J., Hayes B.J. et al. Pitfalls of predicting complex traits from SNPs // Nat. Rev. Genet. 2013. V. 14. № 7. P. 507–515. https://doi.org/10.1038/nrg3457.5
  13. Ahsanullah M., Nevzorov V.N., Shakil M. An Introduction to Order Statistics. Amsterdam–Paris–Beijing: Atlantis Press, 2013. 244 p.
  14. Nagaraja H.N. Contributions to the theory of the selection differential and to order statistics. 1980. Dissertation. Digital Repository @ Iowa State Univ. http://lib.dr.iastate.edu/rtd/6746/
  15. Cohen J.E. Statistical Power Analysis for the Behavio- ral Sciences. Hillsdale, N.J.: Lawrence Erlbaum Associates, Inc. 1988.
  16. Borinskaya S.A., Rubanovich A.V., Larin A.K. et al. Epigenome-wide association study of CpG methylation in aggressive behavior // Russ. J. Genet. 2021. V. 57. № 12. P. 1454–1460. https://doi.org/10.1134/S1022795421120048
  17. Koch S., Schmidtke J., Krawczak M., Caliebe A. Clinical utility of polygenic risk scores: A critical 2023 appraisal // J. Community Genet. 2023. V. 14(5). P. 471–487. https://doi.org/10.1007/s12687-023-00645-z
  18. Herzig A.F., Clerget-Darpoux F., Génin E. The false dawn of polygenic risk scores for human disease prediction // J. Pers Med. 2022. V. 12(8). https://doi.org/10.3390/jpm12081266.
  19. Benjamini Y., De Veaux R., Efron B. et al. ASA President’s task force statement on statistical significance and replicability // Harvard Data Sci. Review. 2021. V. 3(3). https://doi.org/10.1162/99608f92.f0ad0287
  20. McShane B.B., Gal D., Gelman A. et al. Abandon statistical significance // Am. Statistician. 2019. V. 73. Supl. 1. P. 235–245. https://doi.org/10.1080/00031305.2018.1527253
  21. Cheruiyot E.K., Yang T., McRae A.F. GWAS significance thresholds in large cohorts // BioRxiv preprint. 2024. https://doi.org/10.1101/2024.12.09.627629

Supplementary files

Supplementary Files
Action
1. JATS XML
Download

Copyright (c) 2025 Russian Academy of Sciences