Meta Learning — A Overly Simple Overview
I wanted to tackle the bootstrapped meta learning paper published by DeepMind, and figured a more broad overview of the field of Meta Learning would serve as a good introductory note to the topic.
What is Meta-Learning?
This video is a very elegant explanation of the field. As with most concepts, the framing of the question and the question itself is what matters most, rather than answer itself, and so we can break the question down further to: How do we learn to learn? This is what meta-learning is. In the world of bits, this question can be asked in several domains such as Reinforcement Learning, and in the world of atoms it can be asked with regards to the learning process of a human.
When a person learns, say the model of a car, typically it is done in a few/one shot process. It is usually sufficient to look at one example of that car, and identify it later in different environments, and its multitude of colours and options (… as long as one’s memory stays intact :) … ). This can be said for many human learning processes, from identification, to motor skills, to the thinking process itself. One is usually able to transfer the learning process itself, to a new task.
We essentially have two learning algorithms running concurrently. An inner one and an outer one. The inner learning is just a conventional learning algorithm, optimizing and learning for its specific task. The outer one, is the meta learning portion. This is an overly simplified way of explaining it, but it should make some intuitive sense.
We can think about this double learning process in the human context, this time from a broader evolutionary perspective. The inner learning portion is what a single individual learns through a lifetime, the outer learning portion is evolution learning for itself. As a result, an individual can learn certain concepts fast (one/few shot) — he or she can utilize the outer evolutionary learning process for individual needs.
Problems
As pointed out in this paper: “[when a machine trying to implement the above process], it must be able to learn its update rule. Meta-learning is one approach that learns (parts of) an update rule by applying it for some number of steps and then evaluating the resulting performance.”
When learning an update rule, it must be applied, doing so incurs high computational costs. This is usually combated by using K uses of the update rule and then optimizing, and assuming that this does not hinder the performance for the remaining life of the learner.
The failure of this assumption leads to short horizon bias (myopia). In addition, optimizing after the K-th step means the machine stops learning how to learn after this step, which again biases the learning process. This can lead to all sorts of problems.
The DeepMind paper aims at combating this problem which they classify as twofold: curvature, and myopia. Curvature meaning the the outer learning is constrained to the structure/geometry of the inner learning (the learner). Myopia meaning the outer learning is short-horizon biased (within the K-step horizon), and ignores future learning dynamics/subtleties
To be continued …
