What is the problem

From the paper:

Countdown is a popular quiz programme on British television that includes a numbers game that we shall refer to as the countdown problem. The essence of the problem is as follows: given a sequence of source numbers and a single target number, attempt to construct an arithmetic expression using each of the source numbers at most once, and such that the result of evaluating the expression is the target number. The given numbers are restricted to being non-zero naturals, as are the intermediate results during evaluation of the expression.

For eg. Given the numbers [1, 3, 7, 10, 25, 50] and target 765, one solution would be (1 + 50) ∗ (25 − 10).

Implementation

In the paper, an initial solution is presented which is then optimized. In this post we will directly implement the final solution.

First we need a way to represent the supported operations. A sum type is ideal for this and in Rust we can use enum for it:

1
2
3
4
5

data Op
  = Add 
  | Sub 
  | Mul 
  | Div

1
2
3
4
5
6
7

#[derive(Copy, Clone)]
enum Op {
  Add,
  Sub,
  Mul,
  Div
}

Next, we need to define an expression - which is either a number, or two sub-expressions combined with an operator.

1
2
3

data Expr
  = Val Int
  | App Op Expr Expr

1
2
3
4
5
6
7

type Int = i64;

#[derive(Clone)]
enum Expr {
  Val(Int),
  App(Op, Rc<Expr>, Rc<Expr>)
}

Next, the paper defines some utility functions: to check if an expr is valid and also to apply an expression and get its result:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

valid :: Op -> Int -> Int -> Bool
valid Add x y = x <= y
valid Sub x y = x > y
valid Mul x y 
  = x /= 1 
  && y /= 1 
  && x <= y
valid Div x y 
  = y /= 1 
  && x `mod` y == 0

apply :: Op -> Int -> Int -> Int
apply Add x y = x + y
apply Sub x y = x - y
apply Mul x y = x * y
apply Div x y = x `div` y

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

fn valid(op: &Op, x: Int, y: Int) -> bool {
  match op {
    Add => x <= y,
    Sub => x > y,
    Mul => x != 1 && y != 1 && x <= y,
    Div => y > 1 && ((x % y) == 0),
  }
}

fn apply(op: &Op, x: Int, y: Int) -> Int {
  match op {
    Add => x + y,
    Sub => x - y,
    Mul => x * y,
    Div => x / y,
  }
}

Now we get into the core logic of implementation. We need a way to split a list at all possible points. For eg. [1, 2, 3] to ([1], [2, 3]) and ([1, 2], [3]).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

split        :: [a] -> [([a], [a])]
split []     = [([], [])]
split (x:xs) = ([], x:xs) : [(x:ls, rs)
               | (ls,rs) <- split xs]

ne         :: ([a], [b]) -> Bool
ne (xs,ys) = not (null xs || null ys)

nesplit :: [a] -> [([a], [a])]
nesplit = filter ne . split

1
2
3
4
5
6

fn split<T>(ns: &[T])
  -> Vec<(&[T], &[T])> {
  (1..ns.len())
    .map(|i| ns.split_at(i))
    .collect()
}

For eg, using the cute crate, we can rewrite the split function as follows:

1
2
3

fn split<T>(ns: &[T]) -> Vec<(&[T], &[T])> {
  c![ns.split_at(i), for i in 1..ns.len()]
}

While this is concise, I'll stick with what we can do with standard Rust in this post.

Next we need a function that returns the list of all permutations of all subsequences of a list. In the paper it is called subbags and uses predefined functions that returns permutations etc. We use the permutations function in the itertools crate in Rust for simplicity:

1
2
3
4
5

subbags :: [a] ->  [[a ]]
subbags xs = [zs | ys <- subs xs,
                   zs <- perms ys]

-- subs and perms defined elsewhere

1
2
3
4
5
6
7

fn sub_bags<T: Clone>(xs: Vec<T>)
  -> Vec<Vec<T>> {
  (0..xs.len() + 1)
    .flat_map(|i|
      xs.iter().cloned().permutations(i))
    .collect()
}

1
2
3
4
5

combine :: Result -> Result 
           -> [Result]
combine (l,x) (r,y)
  = [(App o l r, apply o x y)
      | o <- ops, valid o x y]

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

fn combine((l, x): &Result, (r, y): &Result)
  -> Vec<Result> {
  [Add, Sub, Mul, Div].iter()
    .filter(|op| valid(op, *x, *y))
    .map(|op|
      (App(*op,
           Rc::new(l.clone()),
           Rc::new(r.clone())),
      apply(op, *x, *y)))
    .collect()
}

Finally the function that ties everything together:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

results :: [Int] -> [Result]
results []  = []
results [n] = [(Val n,n) | n > 0]
results ns
  = [res | (ls,rs) <- nesplit ns
         , lx  <- results ls
         , ry  <- results rs
         , res <- combine lx ry]

solutions :: [Int] -> Int -> [Expr]
solutions ns n
  = [e | ns' <- subbags ns
       , (e,m) <- results ns'
       , m == n]

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

fn results(ns: &[Int]) -> Vec<Result> {
  match ns {
    [] => vec!(),
    [n] => vec!((Val(*n), *n)),
    _ => _results(ns),
  }
}

fn _results(ns: &[Int]) -> Vec<Result> {
  split(ns).iter()
    .flat_map(|(ls, rs)| results(ls).into_iter()
      .flat_map(move |lx| results(rs).into_iter()
        .flat_map(move |ry| combine(&lx, &ry))))
    .collect()
}

pub fn solutions(ns: Vec<Int>, n: Int)
  -> Vec<Expr> {
  sub_bags(ns).iter()
    .flat_map(|b| {
        results(&b).into_iter()
          .filter(|(_, m)| *m == n)
          .map(|(e, _)| e)
          .collect::<Vec<Expr>>()
    })
    .collect()
}

Sidenote - adding parallelism

Since our code is written in a functional way it is trivial to make parts of it parallel. By using the rayon crate and just making two minor changes:

• sub_bags(ns).iter() –> sub_bags(ns).par_iter()
• Rc –> Arc

this reduces the execution time from ~370ms to ~70ms for the above example.

Summary

Rust is a fairly expressive language and using the functional style is quite natural. Ownership and lifetimes do get in the way sometimes, but it is a price worth paying considering the runtime benefits. Also the compiler error messages are really helpful and guides you to the right solution.

• Performance (zero cost abstractions)
• Reliability (no data races)

Rust is strongly influenced by functional programming languages like ML, so it is possible to follow a functional coding style in it. However being functional is not one of Rust's explicit goals.

In this post (and possibly follow up ones) I'll explore functional programming in Rust. This is not an introduction to Rust or functional programming. Basic familiarity with them is assumed.

Functional Programming

The goal of programming is to manage complexity.

In functional programming we manage complexity by principled composition - large things are composed of small things.

The more things we can compose together the better abstractions we can build and reduce complexity. The building blocks are types and functions, and let us see how well we can compose them in Rust.

Algebraic data types

Lets start with types. One way to classify types is by looking at number of inhabitants, that is, how many values of a particular type exists. For eg, the unit type (() in Rust) has only one value, bool has two values etc. Apart from builtin types, Rust supports Sum and Product types, which means we can combine basic types using addition or multiplication (in terms of number of inhabitants). The key insight here is that we can use the familiar algebra from maths to reason about them, hence the name.

In Rust we could create a sum type by using enum. Think of it more like type constructor in Haskell, than enum in C/Java.

For eg. we can create a type with a two values as follows:

1
2
3
4

enum Wing { // name of type
    Left    // value
    Right   // value
}

A product type can be created using a tuple (A, B) or struct.

Here are examples of some types:

As you can see we can compose types using addition or multiplication. This is useful in modelling types.

For eg. since A + A = 2 * A, we could represent a type enum Score<A> { Left(A), Right(A) } as (bool, A). Intuitively, the boolean represents whether the value is left or right.

Pattern matching

A language feature that goes well with algebraic data types in pattern matching. It helps to deconstruct data out of types in a simple way. Rust offers powerful pattern matching support and almost anything can be deconsutructed using it. For example with the following types:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

enum Maybe<A> {
    Just(A),
    Nothing
}

struct Person {
    name: String,
    age: u8,
    email: Maybe<String>
}

we can pattern match things out of it using:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

let person = Person {
  name: "Alice".to_string(),
  age: 25, 
  email: Just("alice@example.com".to_string())
};
match person {
    Person { name : name, age: age, email : Just(email) }
      => println!("Person {}, aged {} with email {}", name, age, email),
    Person { name: name, age: age, email : Nothing }
      => println!("Person {}, aged {} with no email", name, age)
}

Pattern guards and capturing pattern in a single variable is also supported. For eg:

1
2
3
4
5
6

match someVar {
    Some(val @ 0 ... 9) => println!("within 0 to 9, {}", val),
    Some(val) if val > 20 => println!("greater than 20, {}", val),
    Some(val) => println!("Should be less than 20, {}", val),
    None => println!("None")
}

Patterns is commonly used in match expressions but they can be used in place of identifier in other places as well like variable declaration, function arguments etc.

Functions

Functions are first class in Rust and it supports closures as well.

However, one has to think about ownership and lifetime of variables when dealing with them. Basically Rust is not a language with garbage collection and the programmer has to give hints to the compiler about ownership of variables so they can be dropped after use. This is probably the hardest part to get while learning Rust and also what makes Rust special.

Read more about ownership here.

Lets start with a simple definition of factorial:

1
2
3
4
5
6

fn factorial(n: u64) -> u64 {
    match n {
        0 | 1 => 1,
        n     => n * factorial(n - 1),
    }
}

This is a straightforward translation of recursive definition of factorial; factorial(n) = n * factorial(n-1). While this is fairly clear, it is often better to use high level combinators than explicit recursion. So we can write above as:

1
2
3

fn factorial(n: u64) -> u64 {
    (1..n+1).fold(1, |acc, i| acc * i)
}

where |acc, i| acc * i is a closure (similar to \acc i -> acc * i in Haskell, or (acc, i) -> acc * i in Java). So the above code basically takes range of numbers from 1 to n inclusive and multiplies them.

Note: Most of the operators in Rust have traits associated with it. So the * operator above is defined in std::ops::Mul trait and acc * i is a short hand for acc.mul(i). So the above can also be written as:

(1..n+1).fold(1, Mul::mul)

This makes it clear that factorial(n) is the product of first n natural numbers. It can be made explicit by using the builtin product function, which folds over using multiplication.

1
2
3

fn factorial(n: u64) -> u64 {
    (1..n+1).product()
}

The key here is all of the above does not have any performance overhead, it performs similar to an imperative loop that multiplies the numbers.

Higher order functions

Now lets look at how to define and use higher order functions. Function types in Rust implement the Fn trait (think of traits as something similar to type classes). A function that takes in type X and return type Y can be represented as Fn(X) -> Y.

For example, consider a function that takes a function and calls it with given argument:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

fn apply<F, A, B>(f: F, arg: A) -> B 
where F: Fn(A) -> B {
    f(arg)
}

fn square(a: i32) -> i32 { a * a }

// apply can take another function as argument
let n: i32 = apply(square, 4);    // n = 16
// apply can also take a closure as argument
let n: i32 = apply(|x| x * x, 4); // n = 16

A more interesting example of higher order function is the fixed point function. A fixed point of a function f is a value a such that f(a) == a. One way to define the fixed point function is provided in sicp. We can translate that to Rust almost verbatim by adding in the right types.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

const TOLERANCE: f64 = 0.000001;

fn fixed_point<F>(f: F, first_guess: f64) -> f64
    where F: Fn(f64) -> f64 {

    // Check if v1 and v2 are close enough to be considered equal
    fn close_enough(v1: f64, v2: f64) -> bool {
        (v2 - v1).abs() < TOLERANCE
    }

    // Call f(guess) and see if it is close enough to guess. If so
    // we have found the fix point of f
    fn try_guess<F>(guess: f64, f: F) -> f64 
        where F: Fn(f64) -> f64 {
        let next = f(guess);
        if close_enough(guess, next) {
            guess
        } else {
            try_guess(next, f)
        }
    }

    try_guess(first_guess, f)
}

fn sqrt(x: f64) -> f64 {
    fixed_point(|y| (y + x/y) / 2.0, 1.0)
}

How about functions returning functions? One way is to return a closure, but it requires little more work that you would expect. For example, consider the compose function that composes two functions f and g:

1
2
3
4

fn compose<X, Y, Z, F, G>(f: F, g: G) -> impl Fn(X) -> Z
where F: Fn(X) -> Y, G: Fn(Y) -> Z {
    move |x| g(f(x))
}

This might look unfamiliar, so lets go through it:

• X, Y, Z, F and G are type parameters. F is type of a function that takes an argument of type X and returns Y etc.
• impl Fn(X) -> Z: The function returns a closure. Since the actual type that is returned might vary per call (each closure is of a different type since the capturing variables might differ), we need to tell compiler that what we really return is an implementation of the Fn trait.
• move This is related to ownership. Since we return a closure, we don't know when that is called and its lifetime may outlive the function. So the closure needs to own the variables instead of borrowing it. This might seem confusing at first, but rust compiler has excellent error messages to guide you here.

Note: Fn is not the only function type. There is also FnOnce which only can be called once, and FnMut which allows the closure to mutate state.

Conclusion

Hope you got a basic taste of functional programming in Rust. Rust has pretty good support for functional programming compared to languages like Java or C++, but not as much as a pure functional language like Haskell. Rust is a good choice if performance and reliability are what you are after with good support for building composable software. It has excellent documentation, good community and a friendly compiler.

A basic interpreter

Let's consider implementing a basic interpreter - one which can add numbers (the paper talks about a bigger one, but this simple example will serve our purpose). And we are concerned only with evaluation, we assume the AST is already available to us. So, we should be able to do things like:

1
2
3

Num(1) == 1
Add(Num(2), Num(3)) == 5
Add(Num(10), Var("v")) {"v" : 5} == 15

In the last example, a variable is defined which should be available via an environment. Let us consider an implementation in Python.

Note: Don't focus too much on the implementation - I've chosen the commonly used way in Python, it doesn't matter if you use objects or functions here.

Coming back to the code above, there are three objects: Num which evaluates to its value, Var which evaluates to value defined in env and Add which evaluates its left and right operands and adds them. Pretty simple.

Error Handling

Currently we don't have any error handling. What if Num contains something other than numbers? a variable is not defined? One option is to return something like None, but that means every caller should check that explicitly. We could raise exceptions, but they don't compose well. Besides, currently all expressions evaluate to same type. Consistency is nice. So what could we do?

All problems in computer science can be solved by another level of indirection - Butler Lampson

Ok. Let us try to abstract the return type. Currently it is returning the value (number). Instead let us return an object that contains the value. Maybe we can extend this object later. Following is the first iteration - without changing any behavior, but only wrapping the result in the new object.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

class M(object):
  def __init__(self, val):
    self.val = val

  def __repr__(self):
    return "%s(%r)" % (self.__class__.__name__, self.val)

  def __eq__(self, other):
    return self.val == other.val

def unit(v):
  return M(v)

def bind(m, f):
  return f(m.val) 

So M is our container object. We also added two utility functions: unit which just creates M, and bind which applies a given function to value in M. Only changes to our interpreter is in evaluate methods:

1
2
3
4
5
6
7
8

# In Num.evaluate:
  return unit(self.val) # Instead of just self.val

# In Var.evaluate:
  return unit(env[self.name]) # Instead of just env[self.name]

# In Add.evaluate:
  return bind(self.left.evaluate(env), lambda x: bind(self.right.evaluate(env), lambda y: unit(x+y)))

The change to Add might seem complicated, so lets go through it. In evaluate, previously we get two numbers and add them. But now we get two M objects. Say, we got M(2) and M(3). How will we make an M(5) out of it? That is where the bind function comes in. It helps us to peek inside M and access the value. So in the example, the above code is equivalent to:

1
2
3
4
5

# bind(M(2), lambda x: ..)
#	=> {x = 2} bind(M(3), lambda y: ...)
#	=> {x=2, y=3} unit(x+y) 
#	=> unit(2+3) 
#	=> M(5). 

I hope it is clear now, otherwise try to take it apart yourself. So our interpreter still has the same functionality, but returns a new type now:

1
2
3

Num(1) == M(1)
Add(Num(2), Num(3)) == M(5)
Add(Num(10), Var("v")) {"v" : 5} == M(15)

Now that we have a generic return type, we could add error as a first class type.

1
2

class Error(M):
  pass 

We can start handling errors like so:

1
2
3
4
5

# In Num.evaluate:
  if isinstance(self.val, Number):
    return unit(self.val)
  else:
    return Error("%s is not a number" % self.val) 

We need to do one last thing. bind on an Error should pass it on instead of applying the operation:

1
2
3
4

def bind(m, f):
  if isinstance(m, Error):
    return m
  return f(m.val) 

That's all. Now our interpreter can handle errors and still maintain the same interface. Let us call this design technique Monads. In theory there are laws governing Monads, but let's ignore them for now.

Monads are return types that guide you through the happy path - Erik Meijer

See the complete code here.

Adding State

What if we need to add some state to our interpreter? Say we need to keep track of number of times add was performed (it could be any state that we want to maintain). One option is to add global variable to keep track, but that is evil. Since we already have abstracted the return type, let us see if we can utilize that. One way to model it is to return a pair of values - the result and count. For eg:

1
2
3

Num(1) == State(1, 0)                           # 0 addition
Add(Num(2), Num(3)) == State(5, 1)              # 1 addition
Add(Num(2), Add(Num(3), Num(5))) == State(5, 2) # 2 addition

So we will define State which encapsulates the data we need to track. It will a lazy object that when invoked will return the actual state.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

class State(M):
  def __init__(self, val, count=0, old=None, f=None):
    super(State, self).__init__(val)
    self.count = count
    self.old = old
    self.f = f

  def __call__(self, count):
    if not self.old:
      return State(self.val, count)
    else:
      new_state = self.old(count)
      return self.f(new_state.val)(new_state.count)

  def __eq__(self, other):
    return self.val == other.val and self.count == other.count

class Counter(State):
  def __call__(self, count):
    return State(self.val, count+1)

The unit and bind now return State objects:

1
2
3
4
5
6
7

def unit(v):
  return State(v)

def bind(m, f):
  if isinstance(m, Error):
    return m
  return State(m.val, old=m, f=f)

The only change required in the interpreter is in Add:

1
2
3
4
5

# In Add.evaluate
  return bind(self.left.evaluate(env),
	      lambda x: bind(self.right.evaluate(env),
                             lambda y: bind(Counter(None), 
                                            lambda t: unit(x+y))))

An additional bind call is added to get the result (unit(x+y)) and apply Counter to maintain count. The complete code is here.

Conclusion

You probably are aware of higher order functions like map which takes a function that acts on values inside a container and produce new values. But the shape of the container itself does not change. With Monads, the function (in bind) acts on value inside the container but returns a new container. This is where Monads get the power from. Languages like Haskell, Scala etc provide rich set of generic operations and syntactic sugar around it.

In any case, considering it purely as a design approach, this is simple and something anyone could've invented.

Starting Point

What you pick first is often important since unlearning is more difficult than learning. So start with the best - SICP. The course videos as well as the book are freely available online.

Other excellent courses

This can be followed up with some other good courses available online (at the time of writing):

• The Paradigms of computer programming has a section on FP. The course is also good overall, if you want to learn different paradigms of programming. Like SICP, the accompanying text is also a masterpiece.
• The functional programming course offers a modern perspective based on Scala. Again highly recommended along with the book.

Short bytes

If you are looking for short insights into why FP is important or some key aspects of it, following videos are recommended:

• Simple Made Easy
• Introduction to Haskel has couple of FP talks.
• Reactive Programming talk explains why FP is one of the best choices for reactive programming.

Thinking functionally

After you learn the basics, the most important skill needed is to think functionally. The best way to grok that is to get your hands dirty - do the exercises, apply it on the next programming task you do, etc.

Following are some good books which will deepen your knowledge on FP:

• Introduction to Functional Programming.
• Learn you a haskell.

Developing purely functional data structures is another art to master. Okasaki's Purely Functional Data Structures is still the best resource in this area. This post highlights some of the other functional data structures.

Interesting Papers

• Why FP matters
• Essence of Monads. I found this to be the best introduction to Monads.
• Original paper on Lisp
• All papers by Graham Hutton are excellent. Its hard to pick and choose from them.
• Somewhat advanced papers on functional pearls.

Summing up

These are just some pointers that came to my mind. If I missed anything important please let me know.

I hope this helps you in your functional programming journey!

• Functional Stacks
• Functional Sets

In this part we will implement a basic Priority Queue in Scala. It will support the following operations efficiently:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

abstract class PriorityQueue[A](implicit val ord: Ordering[A]) {
  // Add an item
  def +(x: A) : PriorityQueue[A]
  // Find min item
  def findMin: A
  // New Priority queue with min item deleted
  def deleteMin(): PriorityQueue[A]
  // Merges two PriorityQueue together
  def meld(that: PriorityQueue[A]) : PriorityQueue[A]
}

Basics

There are different ways to implement a Priority Queue. We will use Binomial Heap. As usual we follow the approach from Okasaki. See this paper for details about implementing efficient and purely functional priority queues. We start with the basic implementation which is simple to understand. If you know how Binomial heaps work, skip to the next section. Else read on.

A Binomial heap is a forest of Binomial trees. A binomial tree of rank 0 has just a single node and of rank k (k > 0) has binomial trees of rank (0, 1, .. k-1) as its children. So it looks like (image from wikipedia):

Another way to think about binomial tree is: to get binomial tree of rank k, take binomial tree of rank k-1 and insert another tree of rank k-1 as a child to the first tree's root.

In any case, a Binomial heap of n elements would contain only trees of the above form and it will correspond to (k₁, k₂, ..k_j) where k₁k₂..k_j is the binary representation of n. So a heap of 5 (101) elements would have trees of rank 0 and 2. This means merging two binomial heap is similar to adding two binary numbers etc.

Ok, but how can we use a Binomial heap as a Priority Queue? Let's walk through an example and add the following items: (3, 5, 1, 2, 13, 15, 11, 12) in this order and see what we get:

So you get the picture. Now to find the min element, we just need to walk through the roots of trees (log(n) items) and return the min. Adding an element is similar to adding 1 to a binary number and deletion is similar. Merging (aka melding) is like adding two binary numbers. In all operations all we need to make sure is that the root is properly selected (since that should always have the minimum).

Implementation

Now that the idea is clear, let's start writing some code. We have a concrete class BinomialQueue to represent the queue and a Node class to represent individual trees. Let's look at the Node class first:

1
2
3
4
5
6
7
8
9

case class Node[A](data: A, rank: Int = 0, children: List[Node[A]] = Nil)
                          (implicit val ord: Ordering[A]) extends Ordered[Node[A]] {

  def link(other: Node[A]) =
    if (ord.compare(data, other.data) < 0) Node(data, rank+1, other :: children)
    else Node(other.data, other.rank+1, this :: other.children)

  override def compare(that: Node[A]): Int = ord.compare(data, that.data)
}

For simplicity, every Node stores its rank and have data and list of children. It is ordered based on the data (which needs to have an implicit Ordering). Other than this we have a link method which links two Node together keeping the min element as new root. For example, in the above figure, when we insert 2, we link [1,2] and [3,5] together and 1 gets to be the root.

Now we can define the BinomialQueue class:

1
2
3
4

private final case class BinomialQueue[A] (nodes: List[Node[A]])
                        (implicit override val ord: Ordering[A])
                        extends PriorityQueue[A] {
}

Operations

Insert

1
2
3
4
5
6
7
8

def +(x: A) : BinomialQueue[A] = BinomialQueue(insertNode(Node(x), nodes))

private def insertNode[T](n: Node[T], lst: List[Node[T]]) : List[Node[T]] = lst match {
  case Nil => List(n)
  case x :: xs =>
    if (n.rank < x.rank) n :: x :: xs
    else insertNode(x.link(n), xs)
}

We have 3 case to handle. Inserting to:

• Empty heap : Create a new List having a rank-0 Node (Eg. inserting 3 above)
• Heap without a rank-0 node: Just add a rank-0 Node to our list (Eg. inserting 1, 13 etc above)
• Heap with a rank-0 Node: Now we need to merge and recursively insert merged Node (Eg. inserting 5, 2 etc)

Find Minimum

Just returns the Node with min element.

1

def findMin: A = nodes.min.data

Merge

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

def meld(that: PriorityQueue[A]) : PriorityQueue[A] =
  BinomialQueue(meldLists(this.nodes, that.nodes))

private def meldLists[T](q1: List[Node[T]], q2: List[Node[T]]) : List[Node[T]] = (q1, q2) match {
  case (Nil, q) => q
  case (q, Nil) => q
  case (x :: xs, y :: ys) => if (x.rank < y.rank) x :: meldLists(xs, y :: ys)
  else if (x.rank > y.rank) y :: meldLists(x :: xs, ys)
  else insertNode(x.link(y), meldLists(xs, ys))
}

Logic is similar to insert: If a particular ranked node does not exist on one list we can just copy it to output. Same ranked Nodes are merged via linking them together.

Delete Minimum

1
2
3
4

def deleteMin(): PriorityQueue[A] = {
  val minNode = nodes.min
  BinomialQueue(meldLists(nodes.filter(_ != minNode), minNode.children.reverse))
}

Finds the min Node and removes it. The children of a Binary tree conveniently forms a Binary heap in reversed order, so we just merge that to other Nodes.

Adapting to Scala Collections

We are done and have a working Priority Queue implementation (we could make some operations more efficient by using Skew binary heaps, but thats for a later time). However, the other common operations on collections like iterating, map, filter etc are missing. Scala has a rich collections library and it is not difficult to reuse most of it for a new collection object. See this excellent guide to understand the design of Scala collections and how to adapt your collection to use it.

In a nutshell we need to do two things: Provide a way to iterate over the items (Traversable or Iterable) and a way to build a new collection (Builder). There are also higher level of abstractions that make things easier. So we change the PriorityQueue definition as follows:

1
2
3
4

abstract class PriorityQueue[A](implicit val ord: Ordering[A])
  extends Iterable[A]
  with GenericOrderedTraversableTemplate[A, PriorityQueue]
  with IterableLike[A, PriorityQueue[A]] {

That's quite a mouthfull, so let's take it one by one. Iterable just defines an iterator method which yields elements of the collection in some order and an isEmpty method. We can implement it as follows:

1
2
3
4
5
6
7

override def isEmpty: Boolean = nodes.isEmpty

//Inside BinomialHeap
override def iterator: Iterator[A] = nodes.flatMap(_.toList).iterator

//Inside Node:
def toList : List[A] = data :: children.flatMap(_.toList)

IterableLike is for ensuring the same-result-type principle. GenericOrderedTraversableTemplate is an easy way to define the builder using a companion object for ordered collections. It has following methods to be implemented:

1
2
3

override def orderedCompanion: GenericOrderedCompanion[PriorityQueue] = PriorityQueue
override protected[this] def newBuilder: mutable.Builder[A, PriorityQueue[A]] =
                                                             PriorityQueue.newBuilder

Basically we just need to tell which is the companion object where the builder is specified. Finally we need to define the companion object which defines how to build a BinomialQueue from other collections:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

object PriorityQueue extends OrderedTraversableFactory[PriorityQueue] {
  override def newBuilder[A](implicit ord: Ordering[A]):
                                 mutable.Builder[A, PriorityQueue[A]] =
    new ArrayBuffer[A] mapResult { xs =>
      xs.foldLeft(BinomialQueue[A](Nil))((t, x) => t + x)
    }

  implicit def canBuildFrom[A](implicit ord: Ordering[A]):
   CanBuildFrom[Coll, A, PriorityQueue[A]] = new GenericCanBuildFrom[A]
}

Now we can use all the power of Scala collections.

1
2
3
4
5

val pq = PriorityQueue(1,2,3,4,5,6)        // BinomialQueue(5, 6, 1, 3, 4, 2)
pq map (_ * 2) filter (_ < 6)              // BinomialQueue(2, 4)
val sq = PriorityQueue("a", "aa", "aaaaa") // BinomialQueue(aaaaa, a, aa)
sq map (_.length) reduce (_*_)             // 10

Isn't that cool?

You can find the complete code here. The Binomial heap images were generated using this code. That's all for now, see you later!

How to Augment the Data

Recall that a Node in our RBT looked like the following:

1

case class Node[+T : Orderable](color: Color, v: T, left: RBT[T], right: RBT[T])

One way to augment the data is to subclass Node and add relevant data to that class. But there is a simpler approach. Note that in the above declaration, the data stored has an abstract type T whose only constraint is that it should be Orderable. So we could package all the data we need in T itself and provide appropriate ordering logic. For example, in case of a TreeMap, T can be a pair of (K, V) so that ordering is only done on K. A generic AugmentedData class can be written as follows:

1
2
3
4

case class AugmentedData[T : Orderable, U](v: T, data: U)
                              extends Ordered[AugmentedData[T, U]] {
  def compare(that: AugmentedData[T, U]): Int = this.v.compare(that.v)
}

So an AugmentedData would have a value v which is orderable and additional data of some type U.

Example - Drawing Trees

Let's look at an example where we augment data. Let's say we need to draw a given RBT (or BST). There are two logical steps to it:

• Identify position of each Node in the picture
• Draw the picture

We separate these two steps because drawing a picture can be done in different ways - as an image, in a terminal etc.

Step 1 - Finding positions

First we need to Augment the data stored in a Node to include its position (x,y) as well. Then we need to find and populate the positions.

We could just use the depth of a node in the tree as its vertical position (y-coordinate). Finding a horizontal position (x-coordinate) is slightly tricky and there are many different ways to do it. We will use the simple approach of taking a node's position in the inorder traversal of the tree. By definition, node on the left comes first and will have a lower x coordinate value than its root which will be lesser than its right child etc. This is not the best approach but suffices for our discussion.

We are going to define a function which takes a RBT node and gives back an RBT node which is the root of tree with augmented data. So it will look like:

1

def toPos[T : Orderable](node: RBT[T], d: Int = 80, start: Int = 30) : RBT[AugmentedData[T, (Int, Int)]] = ???

It has two additional parameters; to adjust the space between nodes: d and start position: start. To simplify things, we can use a helper function calcPos which finds the position and propogates it back.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

def toPos[T : Orderable](node: RBT[T], d: Int = 80, start: Int = 30) : RBT[AugmentedData[T, (Int, Int)]] = {
  def calcPos(n: RBT[T], x: Int, y: Int) : (RBT[AugmentedData[T, (Int, Int)]], Int) = {
    n match {
      case Leaf => (Leaf, x)
      case Node(c, v, left, right) =>
        val (leftTree, myX) = calcPos(left, x, y+d)
        val (rightTree, nextX) = calcPos(right, myX+d, y+d)
        (Node(c, AugmentedData(v, (myX, y)), leftTree, rightTree), nextX)
    }
  }
  calcPos(node, start, start)._1
}

Step 2 - Drawing the Tree

Once we have the positions, drawing a tree is trivial. A natural way to represent the tree is using SVG format. Since it is xml the tree structure can be directly represented. Scala also support XML literals which makes things easier.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

def toSvg[T : Orderable](node: RBT[AugmentedData[T, (Int, Int)]],
                         width : Int = 640, height : Int = 480): NodeSeq = {

  val rgb = Map[Color, String](Black -> "#000000", Red -> "#7f0000")

  def genNode(n: RBT[AugmentedData[T, (Int, Int)]], x: Int, y: Int): NodeSeq =
    n match {
      case Node(_, AugmentedData(_, (lx, ly)), _, _) =>
        <line id="svg_1" y2={y.toString} x2={x.toString} y1={ly.toString}
              x1={lx.toString} stroke-width="3" stroke="#000000" fill="none"/> ++
        genSvg(n)
      case Leaf => <!-- leaf -->
  }

  def genSvg(n: RBT[AugmentedData[T, (Int, Int)]]) : NodeSeq = n match {
    case Node(c, AugmentedData(t, (x, y)), left, right) =>
      genNode(left, x, y) ++
      genNode(right, x, y) ++
      <ellipse stroke={rgb(c)} fill={rgb(c)} cy={y.toString} cx={x.toString}
               ry="20" rx="20" stroke-width="5"/> ++
      <text xml:space="preserve" text-anchor="middle" font-family="serif"
            font-size="24" y={(y+7).toString} x={x.toString} stroke="#ffffff"
            fill="#ffffff">{t}</text>
    case Leaf => <!-- leaf -->
  }
  <svg width={width.toString} height={height.toString} xmlns="http://www.w3.org/2000/svg">
    <g>
      <title>RBT</title>
      {genSvg(node)}
    </g>
  </svg>
}

This produces svg images like the following:

See the complete code here.

• insert: adds a new item to the Set
• delete: removes given item from the Set
• member: check if a given item is in the Set

As we did before with the Stack, we can define this behavior in a Trait like the following:

1
2
3
4
5

trait Set[+T] {
  def insert[U >: T](item: U) : Set[U]
  def delete[U >: T](item: U) : Set[U]
  def member[U >: T](item: U) : Boolean
}

Ordering

We are planning to implement the Set using Binary Search Trees (BST) which means elements in the Set needs to be comparable (or can be ordered). One way to do it is by constraining the type T to be a subclass of Ordered[T]. But a more lenient approach is to allow the caller to use any type as long as it can be implicitly converted to an Ordered[T]. Scala provides syntactic sugar for doing this easily called view bounds using which we can define the type as +T <% Ordered[T]. Since an implicit parameter needs to be passed under the hood, we define Set as an abstract class. A partial definition will look like the following:

1
2
3

sealed abstract class Set[+T <% Ordered[T]] {
  def insert[U >: T <% Ordered[U]](x: U) : Set[U]
}

There are plans to deprecate view bounds from Scala. We could use the equivalent:

1
2
3

sealed abstract class Set[+T](implicit t2ord: T => Ordered[T]) {
  def insert[U >: T](x: U)(implicit u2ord : U => Ordered[U]) : Set[U]
}

This is a bit verbose since we have to repeat the implicit parameter at multiple places. A better option is to use context bounds. We can also define a type alias for T => Ordered[T] to simplify things:

1
2
3
4
5
6
7

type Orderable[T] = T => Ordered[T]

sealed abstract class Set[+T : Orderable] {
  def insert[U >: T : Orderable](x: U) : Set[U]
  def delete[U >: T : Orderable](x: U) : Set[U]
  def member[U >: T : Orderable](x: U) : Boolean
}

Unbalanced BST

Now that we are done with defining proper types, let's start with the actual implementation. We will first implement an unbalanced version. A BST node can either be a Leaf which is a sentinel for empty, or a Node which has a value, left tree and right tree. Once we have this, implementing the operations recursively is trivial since the structure of the tree is also recursive. For instance, to insert a value we need to compare with the root and if the value is less insert on left tree otherwise on right tree etc.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

abstract class BST[+T : Orderable] extends Set[T] {
  def insert[U >: T : Orderable](x: U) : BST[U]
}

object BST {
  class Leaf extends BST[Nothing] {
    def member[U : Orderable](x: U) = false
    def insert[U : Orderable](x: U) : BST[U] = Node(x, Leaf, Leaf)
  }

  class Node[+T : Orderable](y: T, left: BST[T], right: BST[T]) extends BST[T] {
    def member[U >: T : Orderable](x: U) =
                               if (x < y) left.member(x)
                               else if (x > y) right.member(x)
                               else true

    def insert[U >: T : Orderable](x: U) : BST[U] =
                               if (x < y) Node(y, left.insert(x), right)
                               else if (x > y) Node(y, left, right.insert(x))
                               else this
  }
}

As is the case with persistent data structures, the operations will result in new copies with potential sharing of structure. For instance, if we insert a value which is less than that in root node, the complete right subtree can be shared etc.

Deletion

Deleting a node from BST is slightly more complex. One way is to replace the deleted node with its inorder predecessor. Additionally we need to reconstruct the subtree containing the deleted and predecessor nodes since our data structure is immutable. There are special cases to consider like one or both children being empty which can be handled by the Leaf object. In the following implementation we use delete method to locate the node to be deleted and deleteCurrent to actually replace stuff. It uses a recursive fixNodes function which finds the predecessor and propagates it back to deleted node rebuilding the tree.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

class Leaf extends BST[Nothing] {
  def delete[U : Orderable](x: U) : BST[U] = this
  protected def deleteCurrent[U : Orderable](t: BST[U]) = t
  protected def fixNodes[U : Orderable] = null
}

class Node[+T : Orderable](y: T, left: BST[T], right: BST[T]) extends BST[T] {
  def delete[U >: T : Orderable](x: U) : BST[U] =
                    if (x < y) Node(y, left.delete(x), right)
                    else if (x > y) Node(y, left, right.delete(x))
                    else left.deleteCurrent(right)

  protected def fixNodes[U >: T : Orderable] : (U, BST[U]) = {
    val r = right.fixNodes
    if (r != null) {
      val (ny, nr) = r
      (ny, Node(y, left, nr))
    } else (y, left)
  }

  protected def deleteCurrent[U >: T : Orderable](t: BST[U]): BST[U] = {
    val (ny, nl) = fixNodes
    Node(ny, nl, t)
  }
}

Balanced BST

As you would know, we need a better solution to work efficiently on all inputs. The tree operations take O(d) time where d is the depth of the tree. If we insert n items which are in sorted order, we create a tree with depth n which makes the operations linear time. To avoid this, we need to keep the tree balanced so that the depth is roughly log(n). There are multiple ways to do this like AVL trees, Red Black trees(RBT) etc. We will consider a Red black tree implementation here.

Okasaki gives an elegant implementation of RBT in the book (If you don't have a copy see this paper). As usual, we will adopt the solution to Scala and it is natural to extend from the BST, Leaf and Node classes which we already have. We need an additional method balancedInsert to take care of balancing, so let's create a Trait to capture RBT specific stuff:

1
2
3
4

trait RBT[+T] extends BST[T] {
  def balancedInsert[U >: T : Orderable](x: U) : RBT[U]
  def insert[U >: T : Orderable](x: U) : RBT[U]
}

RBT is a BST with the following additional properties which guarantees that the tree remains balanced:

1. Every node is either Red or Black. The Leaf nodes are black by convention.
2. A red node cannot have a red parent
3. Every path from root to leaf should have same no.of black nodes.

balancedInsert makes sure that (2) and (3) above are not violated when inserting an item. We start defining the Node for RBT as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

sealed abstract class Color
case object Red extends Color
case object Black extends Color

case class Node[+T : Orderable](color: Color, v: T, left: RBT[T], right: RBT[T])
                                extends BST.Node[T](v, left, right) with RBT[T] {

  override def insert[U >: T : Orderable](e: U) : RBT[U] = {
    val Node(_, a1,a2,a3) = balancedInsert(e)
    Node(Black, a1, a2, a3)
  }

}

Note that we don't need to define the member function here. The behavior is same as that of a BST and since we inherit from BST.Node we get that for free. We override insert and make sure we balance things when something is inserted if needed. Finally we maintain the invariant that the root node is always Black.

Let's define the Leaf for RBT which is simple enough:

1
2
3
4
5

case object Leaf extends BST.Leaf with RBT[Nothing] {
  override def insert[U : Orderable](x: U) = Node(Red, x, Leaf, Leaf)
  def balancedInsert[U : Orderable](x: U)  = insert(x)
}

So far we haven't done much in RBT and once we define balancedInsert we are done! Of course balancing the tree is the heart of insert operation. As discussed in the paper, we just need to handle balancing (by rotation) in four cases:

This can be declaratively expressed in code, thanks to pattern matching as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

def balancedInsert[U >: T : Orderable](e: U): RBT[U] = {
  def balance(cn: RBT[U]): RBT[U] = cn match {
    case Node(Black, z, Node(Red, y, Node(Red, x, a, b), c), d) =>
      Node(Red, y, Node(Black, x, a, b), Node(Black, z, c, d))
    case Node(Black, z, Node(Red, x, a, Node(Red, y, b, c)), d) =>
      Node(Red, y, Node(Black, x, a, b), Node(Black, z, c, d))
    case Node(Black, x, a, Node(Red, y, b, Node(Red, z, c, d))) =>
      Node(Red, y, Node(Black, x, a, b), Node(Black, z, c, d))
    case Node(Black, x, a, Node(Red, z, Node(Red, y, b, c), d)) =>
      Node(Red, y, Node(Black, x, a, b), Node(Black, z, c, d))
    case c => c
  }

  if (e < v) balance(Node(color, v, left.balancedInsert(e), right))
  else if (e > v) balance(Node(color, v, left, right.balancedInsert(e)))
  else this
}

balance does the actual rotation and it corresponds as is to the above diagram. This, in fact, is the heart of the algorithm. Most of the other things we talked about are about how we organize code cleanly, adding flexible type support etc.

Side Note: We didn't tackle deletion of a node here. Implementing the delete operation is slightly involved for RBT. See this post to see how it can be done.

Example

Let's see a sample tree construction to see how it works. Assume we insert 1,2,3,4 in this order, we get something like the following:

We are only discussing the very basic operations of the data structure here. Using this other convenience functions can be easily added. For example, we can add an apply function to help creating a Set from its arguments as follows:

1

def apply[T : Orderable](xs: T*) = xs.foldLeft(Leaf.asInstanceOf[RBT[T]])((t, x) => t.insert(x))

Testing

As we did before, tests using Scalacheck can be written once we have a generator of random trees.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

implicit def arbBST[T : Orderable](implicit a: Arbitrary[T]) = Arbitrary {
  for {
    v <- Arbitrary.arbitrary[List[T]]
  } yield BST(v:_*)
}

test("Set Insert") {
  check((s: BST[Int], x: Int) => s.insert(x).member(x))
}

test("Set delete") {
  check((s: BST[Int], x: Int) => !s.delete(x).member(x))
}

That's all for now. See the complete code for Set. Scala standard library also provides an implementation of RBT in a different style.

Stack

We start with implementation of a really simple data structure. It is a container which supports the following operation:

• cons: adds a new item to the container
• head : get last inserted item
• tail : get rest of the items
• isEmpty: the container is empty or not

You can think of it as a cons list or Stack as Okasaki calls it. Before thinking about implementation, we can capture the behavior in a Trait like so:

1
2
3
4
5
6

trait Stack[T] {
  def isEmpty: Boolean
  def cons(t: T): Stack[T]
  def head: T
  def tail: Stack[T]
}

Digression: Variance

In Scala, the type parameters are invariant by default, which means if you have a Stack[Shape], you cannot add a Circle to it though Circle is a subclass of Shape. For immutable collections it is better to have this behavior, so we need to make it covariant which can be easily done by adding a + to the type parameter. When you do that, Stack[Shape] will behave like a superclass of Stack[Circle]. But this also means that the following is possible:

1
2
3

val onlyCircles: Stack[Circle] = ...
val shapes: Stack[Shape] = onlyCircles
shapes.cons(new Square)

This is clearly a problem so Scala compiler will prevent you from creating such an cons function. In the example above, cons should ensure that it only accepts a Circle or its subclass. This can be specified by constraining its type to [U >: T] where T is the type parameter for Stack. This is the intuition behind variance (we skipped contravariance) and if you are interested, read more here.

Armed with the idea of variance, we can make our Stack trait more general:

1
2
3
4
5
6

trait Stack[+T] {
  def isEmpty: Boolean
  def cons[U >: T](t: U): Stack[U]
  def head: T
  def tail: Stack[T]
}

Implementation

One way to implement Stack is to consider two cases. A Stack could be empty or it could have a head and tail. Further, empty can be a singleton and we introduce an abstract class CList to represent this particular implementation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

sealed abstract class CList[+T] extends Stack[T] {
  def cons[U >: T](t: U): CList[U] = new Cons(t, this)
}

case object Empty extends CList[Nothing] {
  def isEmpty: Boolean = true
  def head: Nothing = throw new NoSuchElementException()
  def tail: Nothing = throw new NoSuchElementException()
}

case class Cons[+T](hd: T, tl: CList[T]) extends CList[T] {
  def isEmpty: Boolean = false
  def head: T = hd
  def tail: CList[T] = tl
}