Schedule for Spring 2023

1/23/23

Krishna Balasubramanium (UC Davis)

Title: Recent Advances in Non-log-concave and Heavy-tailed Sampling

Abstract: This talk will be about recent advances in the complexity of sampling from non-log-concave and heavy-tailed densities. Taking motivation from the theory of non-convex optimization, first, a framework for establishing the iteration complexity of sampling of the Langevin Monte Carlo (LMC) when the non-log-concave target density satisfies only the relatively milder Holder-smoothness assumption will be discussed. In particular, this approach yields a new state-of-the-art guarantee for sampling with LMC from distributions which satisfy a Poincare inequality. Next, the complexity of sampling from a class of heavy-tailed distributions by discretizing a natural class of Ito diffusions associated with weighted Poincare inequalities will be discussed. Based on a mean-square analysis, we obtain the iteration complexity in the Wasserstein-2 metric for sampling from a class of heavy-tailed target distributions. Our approach takes the mean-square analysis to its limits, i.e., we invariably only require that the target density has finite variance, the minimal requirement for a mean-square analysis.

Bio:  Krishna Balasubramanian is an assistant professor in the Department of Statistics, University of California, Davis. His research interests include stochastic optimization and sampling, geometric and topological statistics, and theoretical machine learning. His research was/is supported by a Facebook PhD fellowship, and CeDAR and NSF grants.

Johannes Wiesel (Columbia)

2/6/23

Yun Yang (UIUC)

2/13/23

Edward Kennedy (CMU)

2/20/23

Jiayang Sun (George Mason University)

Title: Semi-parametric Learning for Explainable Models -Triathlon

Abstract: Feature selection is critical for developing drug targets or understanding reproductive success at high altitudes. However, selected features depend on the model assumption used for feature selection. Determining variable transformations to make the model more realistic or interpretable is not trivial in the case of many features or variables. This talk presents our advance toward a semi-parametric learning pipeline to study feature, transformation, and model selection in a “triathlon.” We introduce a concept of transformation necessity-sufficiency guarantee,  open up dialogues for paradigm changes, provide our learning procedure for explainable models, illustrate its performance, and demonstrate its application in understanding social, physiological, and genetic contributions to reproductive success of Tibetan women. This is the joint work with Shenghao Ye, Cynthia Beall, and Mary Meyer.

Bio:

Jiayang Sun currently holds the positions of Professor, Chair, and Bernard Dunn Eminent Scholar at the Department of Statistics at George Mason University. Before joining GMU, she was Professor of the Department of Population and Quantitative Health Sciences and the Director of the Center for Statistical Research, Computing and Collaboration (SR2c), Case Western Reserve University.  She has published in top statistical and computational journals, including AOS, JASA, AOP, Biometrika, Statistica Sinica, Biometrical Journal, Statistics in Medicine, JCGS and SIAM J Sci. & Stat. Comp, and other statistical and scientific journals. Her statistical research has included simultaneous confidence bounds, multiple comparisons, biased sampling, measurement errors, mixtures, machine learning, causal Inference, crowdsourcing, EHR, text mining, bioinformatics, network analysis, imaging,  high-dimensional and big data. Her interdisciplinary work has included cancer, environmental science, neuroscience, wound care, dental, and biomaterial, in addition to astronomy, computer science, energy, law, and agriculture. She is an elected Fellow of ASA, IMS, and an elected member of ISI, and an active member of the profession, having been the 2016 President of CWS, and on various committees in the ASA, IMS, CWS, ICSA and ISI. Her work has been supported by awards from the NSF, NIH, NSA, DOD, DOE, VA, ASA, and INOVA.

3/6/23

Speaker: Steve Hanneke (Purdue University)

Title: A Theory of Universal Learning

Abstract: How quickly can functions in a given function class be learned from data? It is common to measure the performance of a supervised machine learning algorithm by plotting its "learning curve", that is, the decay of the classification risk (or "error rate") as a function of the number of training samples. However, the classical theoretical framework for understanding optimal rates of convergence of the risk in statistical learning, rooted in the works of Vapnik-Chervonenkis and Valiant (known as the PAC model), does not explain the behavior of learning curves: rather, it focuses on minimax analysis, which can only provide an upper envelope of the learning curves over a family of data distributions, and the "optimal" rate is the smallest such upper envelope achievable. This does not match the practice of machine learning, where in any given scenario, the data source is typically fixed, while the number of training samples may be chosen as needed.

In this talk, I will describe an alternative framework that better captures such practical aspects of machine learning, but still gives rise to a complete theory of optimal learning rates in the spirit of the PAC model. Namely, we consider the problem of universal learning, which aims to understand the convergence rates achievable on every data distribution, without requiring uniformity of the guarantee over distributions for each sample size. In regard to supervised learning, the main result of this work is a remarkable trichotomy: there are only three possible optimal rates of universal learning. More precisely, we show that the learning curves of any given function class decay either at exponential, linear, or arbitrarily slow rates, under the realizability assumption. Moreover, each of these cases is completely characterized by appropriate combinatorial dimensions, and we exhibit optimal learning algorithms that achieve the best possible rate in each case. Allowing for non-realizable (so-called "agnostic") distributions, essentially the same trichotomy remains, with the linear rate replaced by sub-square-root rates.

In recent extensions, we have also characterized the optimal universal rates for multiclass learning, general interactive learning, active learning with label queries, semi-supervised learning, and several other variations. In the course of these works, some general principles have emerged regarding the design of optimal learning algorithms based on winning strategies for certain infinite sequential games (Gale-Stewart games), which are used to define data-dependent partial function classes whose minimax rates match the optimal universal rate for the original function class. The corresponding combinatorial dimensions determine the existence of such winning strategies, and reflect a fascinating blending of familiar dimensions from the classical theories of statistical learning and adversarial online learning.

Based on joint work with Olivier Bousquet, Shay Moran, Ramon van Handel, and Amir Yehudayoff, which appeared at STOC 2021, and various follow-up works (in preparation) with the aforementioned authors, as well as Idan Attias, Ariel Avital, Klim Efremenko, Alkis Kalavasis, Amin Karbasi, Amirreza Shaeiri, Jonathan Shafer, Ilya Tolstikhin, Grigoris Velegkas, and Qian Zhang.

3/13/23

Spring Break

3/20/23

Mouli Banerjee (University of Michigan)

Abstract: I will speak on some of our recent work on transfer learning from a source to a target population in the presence of `posterior drift': i.e. the regression function/Bayes classifier in the target population is different from that in the source. In the situation where labeled samples from the target domain are available, by modeling the posterior drift through a linear adjustment (on an appropriately transformed scale), we are able to learn the nature of the posterior drift using relatively few samples from the target population as compared to the source population, which provides an abundance of samples. The other (semi-supervised) case, where labels from the target are unavailable, is addressed by connecting the probability distribution in the target domain to that in the source domain via an exponential family formulation, and learning the corresponding parameters. Both approaches are motivated by ideas originating in classical statistics.

3/27/23

Qingyuan Zhao (University of Cambridge)

Title: Simultaneous hypothesis testing using negative controls

Abstract: Negative control is a common technique in scientific investigations and broadly refers to the situation where a null effect (“negative result”) is expected. Motivated by a real proteomic dataset and an ad hoc procedure shared with us by collaborators, I will present three promising and closely connected ways of using negative controls to assist simultaneous hypothesis testing. The first perspective uses negative controls to construct a permutation p-value for every hypothesis under investigation, and we give several sufficient conditions for such p-values to be valid and positive regression dependent on the set (PRDS) of true nulls. The second perspective uses negative controls to construct an estimate of the false discovery rate (FDR). We give a sufficient condition under which the step-up procedure based on this estimate controls the FDR. The third perspective, derived from the original ad hoc procedure given by our collaborators, uses negative controls to construct a nonparametric estimator of the local false discovery rate. I will conclude the talk with a twist. This talk is based on joint work with Zijun Gao.

Short bio: Qingyuan Zhao is an Assistant Professor at the Statistical Laboratory, University of Cambridge. He is interested in improving the general quality and appraisal of statistical research, including new methodology and a better understanding of causal inference, novel study designs, sensitivity analysis, multiple testing, and selective inference.

4/3/23

4/10/23

4/24/23