NIPS workshop on the Learning of Invariant Representations


We are working on defining a schedule.


Much work in recent years has shown that the sensory coding strategies employed in the nervous systems of many animals is well matched to the statistics of the natural environment. For example, it has been shown that lateral inhibition occuring in the retina may be understood in terms of a decorrelation or `whitening' strategy (Srinivasan et al., 1982; Atick & Redlich, 1992), and that the receptive properties of cortical neurons may be understood in terms of sparse coding or ICA (Olshausen & Field, 1996; Bell & Sejnowski, 1997; van Hateren & van der Schaaf, 1998). However, most of these models do not address the question of which properties of the environment are interesting or relevant and which others are behaviourally insignificant. The purpose of this workshop is to focus on unsupervised learning models that attempt to represent features of the environment which are invariant or insensitive to variations such as position, size, or other factors.

There is considerable evidence from both psychophysics and neurophysiology that the brain uses invariant representations. For example, priming of object shape is invariant with respect to position and size (Biederman 2001), and neurons in the visual cortex, beginning with complex cells and progressing up to neurons in IT, code for shape features independent of position (Hubel & Wiesel 1962; Hasselmo & Rolls 1989). How the brain forms such representations, and especially how they could be learned from the structure of the environment, remains one of the biggest, challenging problems of computational neuroscience.

Within the past year or two, a number of investigators have begun tackling this difficult problem, and some interesting results have begun to emerge. Some of the approaches to learning invariant representations involve sparse coding as well as methods beyond this classical concept such as temporal coherence. This workshop will bring together these investigators in order to discuss these various approaches of training neurons to be invariant with respect to various transformations in natural scenes.

Preliminary list of speakers:


This one day workshop will provide a forum for discussing both results and problems that are related to the learning of invariant representations. We furthermore want to examine future directions and keep the discussion lively. To spark much discussion we will restrict the time for each participants talk to 20 minutes focusing on talks that offer new directions. The workshop will be wrapped up in a general discussion session.


We expect attendees to be familiar with the general approach of natural scenes statistics. Among these are the ideas of decorrelation, sparse coding and independent component analysis. A great site providing an overview and bibliography about Natural scene statistics can be found at


Konrad Paul Körding, Winterthurerstr. 190, 8057 Zürich, tel: +41-1-6353044, fax: +41-1-6353053,

Bruno A. Olshausen, Center for Neuroscience, UC Davis, and Redwood Neuroscience Institute


Unsupervised Learning of Invariances, Complex Cells, Natural Images, Hierarchical Models, Natural Scenes, Higher order statistics, Machine Learning, ICA, natural sounds, Auditory receptive fields, Neuroscience, Learning with context, Information Maximisation (Infomax), Generative Models

This site will look much better in a browser that supports web standards, but it is accessible to any browser or Internet device.