Dmytro Perekrestenko
Personal homepage
Representational capabilities of generative networks
We show that every d-dimensional probability distribution with bounded support can be generated through deep ReLU networks out of a one-dimensional uniform input distribution. What is more, this is possible without incurring a cost - in terms of approximation error measured in Wasserstein-distance - relative to generating the d-dimensional target distribution from d independent random variables.
- D. Perekrestenko, S. Müller, and H. Bölcskei, "Constructive universal high-dimensional distribution generation through deep ReLU networks," Proc. of the 37th International Conference on Machine Learning (ICML 2020), Vienna, Austria, July 2020. Paper and presentation.
- D. Perekrestenko, L. Eberhard, and H. Bölcskei, "High-dimensional distribution generation through deep neural networks," Partial Differential Equations and Applications, Springer, Sept. 2021 (invited paper). Paper.
- D. Perekrestenko, "Deep Neural Network Approximation Theory," Doctoral thesis, ETH Zurich, 2021. Thesis and presentation.
Deep neural network approximation theory
Understanding fundamental limits of deep neural network learning is crucial for machine learning applications. We developed fundamental limits of deep neural network learning by characterizing what is possible if no constraints on the learning algorithm and on the amount of training data are imposed. Concretely, we consider Kolmogorov-optimal approximation through deep neural networks with the guiding theme being a relation between the complexity of the function (class) to be approximated and the complexity of the approximating network in terms of connectivity and memory requirements for storing the network topology and the associated quantized weights. The theory we developed educes remarkable universality properties of deep networks.
- Short course at National University of Singapore (NUS) covering our results, Singapore, Apr. 2019. Slides.
- D. Elbrächter, D. Perekrestenko, P. Grohs, and H. Bölcskei, "Deep neural network approximation theory," IEEE Transactions on Information Theory, Feb. 2021 (invited paper). Paper.
- D. Perekrestenko, "Deep Neural Network Approximation Theory," Doctoral thesis, ETH Zurich, 2021. Thesis and presentation.
Adaptive importance sampling
As a part of my diploma thesis I was working on improving stochastic coordinate descent algorithm by introducing new adaptive rules for the random selection of their updates.
- D. Perekrestenko, "Faster Optimization through Adaptive Importance Sampling," Master thesis, EPFL, 2016. Thesis and presentation.
- D. Perekrestenko, V. Cevher, and M. Jaggi, "Faster coordinate descent via adaptive importance sampling," Proc. of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, Florida, USA, Apr. 2017. Paper and poster.
Visualizing hidden structures in DNNs
In this project I applied the nonlinear dimensionality reduction technique t-SNE to data representations given by the hidden layers of trained deep neural networks on music datasets. Built a music recommendation system.
- D. Perekrestenko, "Visualizing hidden structures in datasets using deep learning," EPFL, 2015. Report and presentation.
ECG data analysis
We proposed two deep neural network architectures for classification of arbitrary-length electrocardiogram (ECG) recordings and achieved top accuracy on the atrial fibrillation (AF) classification data set provided by the PhysioNet/CinC Challenge 2017.
- M. Zihlmann, D. Perekrestenko, and M. Tschannen, "Convolutional recurrent neural networks for electrocardiogram classification," 2017 Computing in Cardiology (CinC), Rennes, France, pp. 1-4, Sept. 2017. Paper, presentation and code for Tensorflow.
Grounded language learning
Grounded language learning (GLL) is a technique for language acquisition that uses a multimodal set of inputs rather than just sets of words or symbols, e.g. it uses a combination of words and related sounds or visuals. Due to the similarity of GLL with the way humans are exposed to language, studying GLL can potentially yield insights on how language is comprehended by humans. I supervised multiple semester projects on this topic.
- S. Hamdan, "Grounded Language Learning of Visual-Lexical Color Descriptions". Code for PyTorch.
- G. C. Ornelas, "Multimodal Emotion Recognition Using Lexical-Acoustic Language Descriptions". Code for PyTorch.
- P. Schenkel, "Recurrent Neural Networks for Oenological Review Generation". Code for Tensorflow.
Other stuff
- As a part of my internship at ABB Research Center I developed a Markov chain based automatic tool for prediction of web visitor behavior and for assessment of web page usability on ABB website. Tech Stack: Spark, Python, SQL, and Google BigQuery. Data: 4TB of log data from Google Analytics.
- I was teaching Harmonic Analysis, Mathematics of Information, and Neural Network theory courses at ETH Zurich, and supervised multiple semester and master projects. See my ETH page.