Mathias Jean Johansen

Type-Level Programming in TypeScript

Mon, 29 Mar 2021 00:00:00 +0000

The TypeScript type system is immensely powerful, and while the language itself is, of course, Turing complete, so is the type system itself. This allows us to implement types that are far more sophisticated than what is possible in most popular programming languages, as we can do computations on a type-level which is, essentially, what characterizes type-level programming.

In the following, I will demonstrate how we can utilize features of the TypeScript type system such as conditional types, indexed array types, and more to implement a type that computes a given Fibonacci number.

It should be pointed out that while it is possible to implement this, you should consider it merely an academic exercise and not a recommendation for how to use the type system in your production applications.

Before we can implement anything, we need to have a notion of what a number is. In type-level programming, this is most commonly modeled using Peano numbers. We can define our Peano numbers in the following manner:

type Zero = "zero"

type Succ<N> = { n: N }

First, we define the number 0 with our type Zero. Here, it is a type alias for the string "zero", but it could be anything. Next, we define Succ which is a parameterized type (or generic type) that we will use for any number above 0. With these two types, we can now define 1 as Succ<Zero>, 2 as Succ<Succ<Zero>>, and so forth.

For convenience, we will create the following types for the numbers 1 through 10 which we will use later:

type One = Succ<Zero>
type Two = Succ<One>
type Three = Succ<Two>
type Four = Succ<Three>
type Five = Succ<Four>
type Six = Succ<Five>
type Seven = Succ<Six>
type Eight = Succ<Seven>
type Nine = Succ<Eight>
type Ten = Succ<Nine>

Correspondingly, we will define variables that match our types above. We will use the variables later when we are going to test that our Fibonacci type works.

const zero: Zero = "zero"
const one: One = { n: zero }
const two: Two = { n: { ...one } }
const three: Three = { n: { ...two } }
const four: Four = { n: { ...three } }
const five: Five = { n: { ...four } }
// ...
const thirteen: Succ<Succ<Succ<Ten>>> = { n: { ...twelve } }

We now have a notion of what a number is which allows us to move on to the next part of our implementation.

To implement our Fibonacci type, we need a few essential building blocks. We will start by defining the simplest types first.

For addition to work, we need to be able to decrement numbers. It will shortly become clearer why decrementing a number is important when we want to sum two numbers. We can define a Decrement type in this way:

type Decrement<N> = N extends Succ<infer R> ? R : Zero

TypeScript supports inferring within conditional types which means that we can extract types from parameterized types. If N extends Succ, we extract the type R and return it, and, otherwise, we return Zero. Decrement<Succ<Succ<Zero>>> would thus be equivalent to Succ<Zero>.

Similarly, to check if a type is Zero, we will need a type IsZero that we define as such:

type IsZero<N> = N extends Zero ? true : false

With this type, const typeChecks: IsZero<Zero> = true would type check while const doesNotTypeCheck: IsZero<Succ<Zero>> = true would not.

Next, we need to have a way to do if/else expressions so we can return types based on a given condition:

type IfElse<C extends boolean, T, F> = C extends true ? T : F

Here, C can be either true or false, and if it is true, we return the type T, and, otherwise, the type F. So a variable with the type IfElse<true, "foo", "bar"> would only type check if it was assigned to the value "foo".

Three types are missing at this point: Equals, Add, and the Fibonacci type. We define Equals this way:

type Equals<A, B> =
  A extends Succ<infer SA>
  ? B extends Succ<infer SB>
    ? Equals<SA, SB>
    : false
  : A extends Zero
    ? B extends Zero
      ? true
      : false
    : false

Checking for equality on a type-level in TypeScript is difficult to do elegantly, so the type definition above may be less clear than the types defined earlier in the blog post.

With this definition, two types are only equal if they are both Zero. If A and B extends Succ, then we recursively use our Equals type with the extracted types SA and SB until both of the extracted types are Zero, or one of them does not extend Succ meaning that the two types are not equal.

Lastly, we need the Add type to make computing Fibonacci numbers possible. The definition for this type is:

type Add<A, B> = {
  acc: B
  n: A extends Succ<infer _> ? Add<Decrement<A>, Succ<B>> : never
}[IfElse<IsZero<A>, "acc", "n">]

Add works by recursively decrementing the first number, A, and incrementing the second number, B, until the A is Zero. So to add 2 and 3, the steps would be:

Step 1. Add<Succ<Succ<Zero>>, Succ<Succ<Succ<Zero>>>> // 2 + 3
Step 2. Add<Succ<Zero>, Succ<Succ<Succ<Succ<Zero>>>>> // 1 + 4
Step 3. Add<Zero, Succ<Succ<Succ<Succ<Succ<Zero>>>>>> // 0 + 5

Once we reach Zero, we return the type B by indexing the property "acc" which is our result. In this case, Succ<Succ<Succ<Succ<Succ<Zero>>>>> or simply 5.

All that is left now is to implement the Fibonacci type which is defined as follows:

type Fibonacci<N, F0 = Zero, F1 = One> = {
  acc: F0
  n: N extends Succ<infer _> ? Fibonacci<Decrement<N>, F1, Add<F0, F1>> : never
}[IfElse<Equals<Zero, N>, "acc", "n">]

In our type parameter list, we first have N which represents the nth number in the Fibonacci sequence we want to find. This type parameter is then followed by F0 and F1 which have the default types Zero and One, respectively. We use these two types to match the formal definition of the Fibonacci sequence which states that the first two numbers in the sequence should be 0 and 1.

The Fibonacci type is similar to the Add type in that it returns the result stored in our accumulator "acc" once a type is Zero (in this case, N). Until we reach our base case, we keep recursing by indexing the property "n". For every step, we decrement N, replace F0 with F1, and calculate a new type for F1 by adding F0 and F1 together. Eventually, N will become Zero, and we have calculated our nth Fibonacci number.

Now we can finally put our newly defined Fibonacci type to use and check that it works:

const fib0: Fibonacci<Zero> = zero
const fib1: Fibonacci<One> = one
const fib2: Fibonacci<Two> = one
const fib3: Fibonacci<Three> = two
const fib4: Fibonacci<Four> = three
const fib5: Fibonacci<Five> = five
const fib6: Fibonacci<Six> = eight
const fib7: Fibonacci<Seven> = thirteen

If you are testing this out yourself, you will see that the code only type checks if the value assigned to the variable matches to corresponding Fibonacci number. So if we change the value of fib4 to, say, eight, our code will no longer type check.

Two things are worth pointing out here, though: a) TypeScript gives us an error stating that the “type instantiation is excessively deep and possibly infinite” once we try to compute any Fibonacci number larger than eight, and, b) any number from Fibonacci<Five> and above will type check as long it is above five. Admittedly, it is unclear to me whether this is a limitation of the TypeScript type checker or an issue with my implementation, but it clearly shows that while it is possible to implement this, issues will likely occur sooner or later.

I hope you have found this blog post insightful. If you are curious and want to play around with the Fibonacci type, you can find the code in a TypeScript playground here and a repository with more examples here.

Building a Passphrase Generator in Haskell

Tue, 26 Jan 2021 00:00:00 +0000

During the recent holiday season, I decided to put my admittedly limited Haskell skills to use. While the CIS 194 course exercises are excellent for learning and practicing Haskell, and I enjoy working through them when time permits, I wanted to work on a more structured “real world” project this time.

I had a few goals in mind for this particular project. First and foremost, it had to be a well-defined and relatively simple problem that I needed to solve. Secondly, I wanted it to require me to handle IO and side effects, and, lastly, I wanted it to offer me an opportunity to integrate third-party libraries in my app.

Only a couple of weeks prior to starting this project, I had been reading the guide by the Electronic Frontier Foundation on how to generate strong six-word Diceware passphrases. Turning their suggested method into a small command-line utility seemed to suit my three goals nicely, so I decided to give it the old college try. In this blog post, I will elaborate on how this method of generating passphrases works and how it can be implemented in Haskell. The entire project can be found on GitHub here.

The steps needed to generate a passphrase using the directions provided by EFF are fairly simple, and you only need five dice and a wordlist:

Roll five dice at the same time and write down the five numbers. The numbers might be something like 4, 3, 4, 6, 3.
In your wordlist, find the word that matches the number 43463 (in this case, panoramic), and write it down.
Repeat these two steps until you have (at least) six words. Once you have repeated the steps, you might have a passphrase such as “whinny daffodil aerobics upheld bankable niece.”

Before detailing the implementation, it should be noted that the project uses rio as a prelude replacement by setting {-# LANGUAGE NoImplicitPrelude #-} and importing RIO in all files. With our project description in place, we are now ready to implement it.

In order to solve the first step, we need to be able to roll five dice and join the numbers into a single digit. To do so, we define a rollDice function:

rollDice :: Int -> Integer -> Integer -> IO [Integer]
rollDice rolls start end = replicateM rolls $ generateBetween start end

rollDice takes three arguments: rolls which describes how many dice we want to roll, and start and end which respectively describe the start and end of the range of numbers, we will be working with. Here, we rely on generateBetween from the Cryptonite library to generate a random number within our range. Since our result is wrapped in a monad, we need to use replicateM to repeat this process N times.

Next, we need to be able to join these digits into one. To do this, we will define a function joinDigits in the following manner:

joinDigits :: [Integer] -> Integer
joinDigits = L.foldl' ((+) . (* 10)) 0

This function takes a list of Integers and uses L.foldl' to combine them into one.

For every element in the list, we multiply our accumulator with 10 and then add the first element in our list to the result. This process is repeated until our list is empty. For instance, given the numbers 5, 2, 3, 1, 6, and 0 as our initial accumulator value, this is how our result is calculated:

0                  # Initial accumulator value
0*10+5    = 5      # The list is now [2, 3, 1, 6]
5*10+2    = 52     # The list is now [3, 1, 6]
52*10+3   = 523    # The list is now [1, 6]
523*10+1  = 5231   # The list is now [6]
5231*10+6 = 52316  # The list is now []

With these two functions defined, we now have the necessary parts for generating the indices which we will need later when pairing dice rolls with words. To pair indices with words, we will need a function toTuple:

toTuple :: [String] -> (Integer, String)
toTuple [] = (0, "")
toTuple (index:word) = (fromMaybe 0 index', fromMaybe "" word')
  where
    index' = readMaybe index :: Maybe Integer
    word' = L.headMaybe word

toTuple takes a list of Strings and returns a tuple of (Integer, String) where the Integer represents an index, and the String represents the corresponding word from the wordlist. We will call this function later when we map over each line in our wordlist.

Since a line in our wordlist follows the format “11111 abacus”, we want to read the first string as an Integer and take the first word of the remaining words in our line. As read and head can throw exceptions, we use the safe alternatives readMaybe and L.headMaybe instead. To unwrap our Maybe values, we call fromMaybe and use 0 and "" as the default values for the index and word, respectively. Lastly, to avoid non-exhaustive patterns, we pattern match on the empty list and return (0, "") for good measure.

Now we can finally move on to writing the logic for generating passphrases which is handled by the passphrase function:

passphrase :: String -> [[Integer]] -> String
passphrase wordlist dice =
  unwords $ map (flip (Map.findWithDefault "") wordMap) indices
    where
      indices = map joinDigits dice
      wordMap = Map.fromList $ map (toTuple . words) $ lines wordlist

To generate a passphrase, we need to pass our wordlist as the first argument and then a list containing a list of our dice rolls as the second argument. The first argument will look like this:

abacus
abdomen
abdominal
abide
abiding
ability
ablaze
able
abnormal
abrasion
abrasive
abreast
abridge
...

While the second argument, i.e., our dice rolls, might look like this:

[ [5, 3, 2, 1, 6]
, [3, 4, 1, 6, 6]
, [2, 4, 4, 1, 3]
, [1, 3, 3, 5, 4]
, [2, 3, 2, 1, 5]
, [6, 4, 3, 2, 5]
]

For all the lists in our list of dice rolls, we use our previously defined joinDigits function to join the numbers in the list into single digits. We then store these digits, or indices, in the indices variable, so we can use them later when we need to look up words in our wordlist.

As shown above, the wordlist follows the format INDEX WORD, so we split our wordlist on line breaks using lines which gives us a list of Strings containing elements such as "11111 abacus" and "11112 abdomen". We want to further break these elements down so we will end up with a list of tuples. To do so, we map over all the lines in our wordlist and call (toTuple . words) on them. Now that we have a list of tuples, we can convert it to a Map using Map.fromList so we can perform efficient lookups later.

For all our indices, we want to find the corresponding word in our wordlist which we do with map (flip (Map.findWithDefault "") wordMap) indices. Since Map.findWithDefault expects the index of the map to be the second argument and the map to be the last, we use two useful techniques here to make our code less convoluted. First, we leverage currying by applying "" as the first argument to Map.findWithDefault, and then we use flip to reverse the order of the last two arguments.

By flipping the order, we can now do a simple map over our indices variable and perform a lookup in our wordMap in a slightly more elegant fashion. Once we have all our words, we can join them into a single String using unwords.

With the fundamental logic now defined, we only need to read our wordlist from a file, throw five dice six times, and pass these two values to our passphrase function. We do so by defining the entry point of our app in Main.hs as such:

main :: IO ()
main = do
  wordlist <- T.unpack <$> readFileUtf8 "data/eff-large-wordlist.txt"
  dice <- replicateM 6 $ rollDice 5 1 6
  runSimpleApp $ do
    logInfo . display . T.pack $ passphrase wordlist dice

This largely resembles the steps listed earlier. To retrieve our wordlist, we call readFileUtf8 with the path to our wordlist, and then T.unpack to convert the Text value into a String.

Next, we roll our five dice six times by calling replicateM 6 $ rollDice 5 1 6, and pass our wordlist and dice to the passphrase function.

After calling our passphrase function, we now have a passphrase that we need to output in our terminal. Following the conventional structure for how to build apps using rio, we run our app as a SimpleApp which provides a convenient default configuration. We convert the result from our passphrase function into a Text value, then a Utf8Builder, and, lastly, print it in our terminal with logInfo.

With everything in place, we have now reached our goal and can finally build our project by running stack build and generate a passphrase:

% stack exec -- passphrase
polar geek nimble okay appealing trim

In my experience, the Haskell community is great at sharing their knowledge on both Twitter and in various blog posts. With this blog post, I hope to contribute back to the community and help especially the people newer to the language who might want to move beyond contrived exercises and learn how to build smaller apps. If you are interested, you can find the project on GitHub here, and if you have any feedback, I am all ears.

Common Pitfalls in Ruby

Mon, 02 Jan 2017 00:00:00 +0000

Last week, I stumbled upon a document covering a list of common pitfalls in Ruby that I assembled in the beginning of 2015. Browsing through the various gotchas, I decided to rework the list, and publish it here for future reference. Hopefully, others will find it useful too.

`and`, `or`, and `not`

Since Ruby is often considered a readable language, many novices tend to believe that and, or, and not are more appealing alternatives to &&, ||, and !, respectively. They differ, however, in their behavior. Consider the following example:

foo = true and false
=> false
foo
=> true
foo = true && false
=> false
foo
=> false
foo = false || true
=> true
foo = false or true
=> true
foo
=> false

Similarly, this example below reveals the issue with not versus !.

not true
=> false
!true
=> false
not 3 == 4
=> true
!3 == 4
=> false

In the first example, foo = true and false will be interpreted as (foo = true) and false where foo = true && false will be interpreted as foo = (true && false), as and and or have lower precedence than && and ||. Thus, and and or are meant for flow control, while && and || are for boolean logic.

Correspondingly, not and ! also behave differently due to their precedence. As expected, not has lower precedence than !.

As shown in the second example, not 3 == 4 will be interpreted as not (3 == 4) which evaluates to true, while !3 == 4 will be interpreted as (!3) == 4, i.e. false == 4.

`equal?`, `eql?`, `===`, and `==`

These four methods of determining equality all behave widely different. The first method, equal?, is an identity comparison, so it will only return true in situations where a is the same object as b. One can think of equal? as a pointer comparison.

1.equal?(1)
=> true
"a".equal?("a")
=> false

It is worth nothing that if we enable immutable strings in the example above with # frozen_string_literal: true, or the --enable-frozen-string-literal flag, the last example will also evaluate to true.

eql? is essentially for hash comparisons. It returns true when a, and b refer to the same hash key. Hash uses this to test for equality. It is common to alias eql? to ==.

Finally, ===, and == are for case equality, and generic equality, respectively. === is typically overridden to provide meaningful semantics in case statements for Range, Regex, and Proc. == is the most common comparison operator, and therefore this is usually overridden to provide class-specific meaning.

`to_s`, `to_str`, and `String`

to_s, and String are more or less equivalent. String will check the class of its parameter, and if it is not already a string, it will call to_s on it. Calling to_s obviously means it will be called regardless.

to_str is different from the two, however. It should only be implemented in situations where your object acts like a string as opposed to being representable by a string meaning you should only implement to_str in your classes for objects that are interchangeable with String objects.

`any?`

The Enumerable module in Ruby defines an any? method. When I initially learned Ruby, I expected that it would return true if the collection was non-empty (as a negated empty?). Nevertheless, any? (without a provided block) returns true if at least one of the collection members is not false or nil. The following example demonstrates this behavior:

[false, nil].any?
=> false
[true, false].any?
=> true
[:truthy, nil].any?
=> true

`super`, and `super()`

In Ruby, we learn that we can omit parentheses in method calls without any arguments, as foo, and foo() returns the same result, and abandoning unnecessary parentheses is normally what most style guides advocate for.

Consequently, it might be rather tempting to leave out parentheses when calling super() but calling super, and super() is not entirely the same in Ruby. super (without parentheses) will call the parent method with exactly the same arguments that were passed to the original method, while the latter will call the parent method without any arguments at all.

`size`, `count`, and `length`

Similar to other methods in this blog post, people may be tempted to think that size, count, and length are simply aliases for the same operation but this is yet another quirk of Ruby.

length, and size are identical, and they usually run in constant time, so they are faster than count. Unlike count, they are not a part of Enumerable but rather a part of a concrete class (such as Array, or String). Normally, I tend to use length for strings, and size for collections.

As mentioned, count is a part of Enumerable, and it is usually meant to be used with a block, although this is not mandatory.

[1, 2, 3, 4, 5, 6].count(&:even?)
=> 3
[1, 2, 3, 4, 5, 6].count
=> 6

`Hash.new([])` vs. `Hash.new {|h, k| h[k] = [] }`

Hash.new([]), and Hash.new {|h,k| h[k] = [] } may look similar but they behave slightly different. When accessing an unknown element, Hash.new([]) will always return the same array where Hash.new {|h, k| h[k] = [] } creates a new array. A quick benchmark reveals that accessing an unknown element from a hash initialized with Hash.new([]) is approximately twice as fast as accessing an unknown element from a hash initialized with Hash.new {|h,k| h[k] = [] }

This behavior can also be seen in arrays where Array.new(42) { Foo.new } will initialize a new Foo every time, while Array.new(42, Foo.new) will refer to the same Foo object for each element.

The flip-flop operator (`..`)

In Ruby, .., and ... are most often used for ranges. It allows us to succinctly express ranges from A to Z as such 'a'..'z'. The .. operator always includes the last element where ... will skip the last element in the range.

('a'..'z').to_a.size
=> 26
('a'...'z').to_a.size
=> 25

We can also conveniently express a date range as such:

require 'date'
=> true
now = DateTime.now
=> #<DateTime: 2016-12-15T22:36:38+01:00 ((2457738j,77798s,446146000n), +3600s, 2299161j)>
last_month = now - 30
=> #<DateTime: 2016-11-15T22:36:38+01:00 ((2457708j,77798s,446146000n),+3600s,2299161j)>
(last_month..now).to_a.size
=> 31

The .. operator can, however, lead to a bit of confusion since it has a different behavior in other situations.

(1..20).each do |x|
  puts x if (x == 5) .. (x == 10)
end

The condition in the loop above evaluates to false every time it is evaluated until the first part, i.e. x == 5, evaluates to true. Then it evaluates to true until the second part evaluates to true. In the above example, the flip-flop is turned on when x == 5 and stays on until x == 10, so the numbers from 5 to 10 are printed.

The flip-flop operator only works inside ifs and ternary conditions. Everywhere else, Ruby considers it to be the range operator. With the flip-flop operator, we now conclude our whirlwind tour of the various pitfalls in the Ruby programming language.

As demonstrated in this blog post, Ruby has quite a few quirks. In order to avoid many of these pitfalls, I usually advocate for using Rubocop either locally on your own machine, or ideally as a part of the CI pipeline. While it may not detect all the issues at hand, it is at most times extremely good at reporting problems in your code.

Curry On: A Retrospective

Wed, 20 Jul 2016 00:00:00 +0000

This year’s Curry On conference in Rome ended last night, and it was certainly worth the trip. While I saw a fair amount of presentations throughout the conference, I’d like to briefly touch upon a few of my favorite ones in this blog post.

The Racket Manifesto by Matthias Felleisen was a good reminder that I need to spend more time with Racket. It is a deeply flexible language, and it is clear that countless hours have been spent refining it. Both Peter O’Hearn’s talk Move Fast to Fix More Things, and Dave Herman’s talk thoroughly examined their transitions from academia to industry, and how Facebook, and Mozilla have benefited from research with Infer, and Rust, respectively. It is projects like these that demonstrate how cooperation between the industry, and academia can truly lead to advancements in our field.

I also found Crista Lopes’s presentation on Exercises in Programming Style extremely entertaining. In the presentation, she essentially went through a large range of different programming styles from imperative to continuation-passing style, and it seemed somewhat reminiscent of The Evolution of a Haskell Programmer by Fritz Ruehr from Willamette University which was also a joy to read.

Rascal: the Swiss Army Knife of Meta Programming by Tijs van der Storm was an interesting presentation too. Although, I found the syntax to be a bit peculiar, it was obvious that Rascal truly enables non-trivial metaprogramming tasks to be solved almost effortlessly. The source-to-source Java transformation in the presentation impressed me, and certainly made me interested in the language. Finally, Cristina Cifuentes’s presentation Are We Ready for Secure Languages? greatly illustrated how we should start thinking security into our languages rather than adding security measures as an afterthought.

In addition to the numerous marvelous talks, I also found the conference to be extremely sympathetic. It was quite clear that the organizers had put an effort into encouraging diversity which I greatly appreciated, and I think the conference succeeded tremendously in enabling a conversation between academia, and the industry. Finally, it did not hurt that this was the venue:

Next year, Curry On will be held in Barcelona, and I hope I’ll be able to attend again.

2015 in Review

Fri, 01 Jan 2016 00:00:00 +0000

This post is simply a reminder to myself of all the great things that have happened during 2015.

Highlights

In many ways, 2015 was a year of JavaScript to me. I visited my first conference, ReactEurope, and went to a React hackathon at the Mozilla office in Paris. In addition, I spent countless hours learning Flow, and implementing it into our existing code base at work. I also familiarized myself with the wonders of Babel, and ECMAScript 2015, and I learned to apply, and appreciate the ideas behind JSON schemas.
I submitted, and successfully defended my bachelor’s thesis. This is possibly the most challenging work I’ve accomplished so far. It is an implementation of a lazy, functional programming language which can be executed on two, separate stack machines that perform lazy evaluation using two different strategies. It consists of a compiler written in OCaml, and the aforementioned stack machines written in C. The source code is available on GitHub, although sufficient documentation is still severely lacking.

Books read

Compared to 2014, I’ve read far fewer books in 2015 than I wanted to both in terms of fiction, and technical books. In 2016, I hope to delve into Tropic of Capricorn, Flaubert, and ideally also SICP which has been on my to-read list since the beginning of 2014. These are the books I completed in 2015 in chronological order:

Favorite records of the year

I’ve listed my favorite records of the year for quite some years now. This year is no different.

I have a slight feeling that 2016 will include working with Relay, and GraphQL. If time permits it, it would be interesting to dabble in Rust, Idris, Racket, and Elixir, or even notably older programming languages as Prolog, and Forth.

Hopefully, I’ll have more time to practice music too, and perhaps travel a bit as well. I’d also be fun to revisit my bachelor’s thesis, and refine the language. I might give an update on the state of affairs next year. We’ll see.

Here’s to 2016!

Ruby Extensions in C

Sat, 07 Feb 2015 00:00:00 +0000

Occasionally, we come across particular sections in our programs that need to be exceptionally fast. Ruby allows us to write extensions in C, so that we can delegate the heavy lifting. In this post, I’ll show you how easy it is to extend Ruby with C by writing a trivial factorial function in C which we’ll be able to call from Ruby.

We start out by creating the relevant directories and files.

$ mkdir fact
$ cd fact
$ mkdir ext
$ touch ext/extconf.rb ext/fact.c

In the ext/extconf.rb file, we require the mkmf module which allows us to generate an applicable Makefile that compiles our C code.

require "mkmf"

create_makefile("fact")

Afterwards, we write the actual C program in the ext/fact.c file. It should look like this:

#include "ruby.h"

void Init_fact();
VALUE fact(VALUE self, VALUE n);

void Init_fact()
{
  VALUE Fact = rb_define_module("Fact");
  rb_define_method(Fact, "fact", fact, 1);
}

VALUE fact(VALUE self, VALUE n)
{
  int x = NUM2INT(n);
  int factorial = 1;

  for (int i = 1; i <= x; i++) {
    factorial *= i;
  }

  return INT2NUM(factorial);
}

First, we include the ruby.h header file, so that we can access the necessary macros and functions. Ruby will by default execute our initializing function Init_fact, so we define our Fact module in here with the function rb_define_module. Additionally, we define a method fact in the Fact module with the rb_define_method function. It takes a class, a method name, a function and the number of arguments.

In the function VALUE fact(VALUE self, VALUE n), we pass both VALUE self and the number we want to take the factorial of. We convert the VALUE n to an int with the NUM2INT macro, and convert the integer back to Fixnum when returning the result.

We can then compile our program, and open irb to verify that it works as expected:

$ ruby ext/extconf.rb
creating Makefile
$ make
compiling ext/fact.c
linking shared-object fact.bundle
$ irb
irb(main):001:0> require_relative "fact"
=> true
irb(main):002:0> include Fact
=> Object
irb(main):003:0> fact(5)
=> 120

Utilizing C in our Ruby programs can be incredibly useful both in order to achieve higher performance for specific parts of our application, but also if we want interfacing with other C code. If you want to go further, I highly recommend that you take a look at the README.EXT documentation.

Encrypting Files With GPG and Vim

Tue, 27 Jan 2015 00:00:00 +0000

I do most of my writing in Vim whether it is programming or editing ordinary documents. The only two exceptions are journals and emails. For journals, I’ve been using Day One for quite an extensive amount of time now, but I’ve been considering to replace it with Vim. In order to replace Day One, I need a way to effortlessly encrypt and decrypt text files. In this post, I’ll show you how I’ve set up an environment that enables me to do so.

By default, Vim provides you with the ability to encrypt and decrypt files in a quite simple manner. You open a file with vim -x my-top-secret-document.md or the :X command, and then Vim prompts you for an encryption key. When you’ve entered the encryption key twice, you are able to edit the document. If you try to print the document with cat afterwards, you’ll see gibberish like VimCrypt~01!gd)�/�:�-(%), but if you open the file with Vim, the editor will prompt you for the key phrase, and after entering the correct encryption key you can read and edit the file. There is a caveat to this approach, however. According to the :X help page, Vim has not been tested for robustness, and we do not want swap files, the viminfo file or any other files for that matter to expose our file contents, so instead we are going to rely on GPG and autocommands.

What we need to include in our .vimrc is the following autocommand group:

augroup encrypted
  autocmd!
  autocmd BufReadPre,FileReadPre *.gpg set viminfo=
  autocmd BufReadPre,FileReadPre *.gpg set noswapfile noundofile nobackup
  autocmd BufReadPost *.gpg :%!gpg --decrypt 2> /dev/null
  autocmd BufWritePre *.gpg :%!gpg -ae --default-recipient-self
  autocmd BufWritePost *.gpg u
augroup END

Essentially, we disable auto-saving the .viminfo file, and then we disable swap files, undo files and backup files. After the buffer is read, we decrypt it with GPG, so that we are able to read the content in Vim. Before we eventually save our file, we encrypt the entire file with the user ID of the default key as the recipient of our message, and finally after writing the file we undo the last action, so that the file is still readable to us.

Curry Your Procs

Tue, 04 Nov 2014 00:00:00 +0000

Recently, I discovered that Ruby provides a rather esoteric #curry method for Procs that I’d like to examine in this post.

Currying basically means taking one function with multiple arguments and converting it into a function that takes only one argument and returns another function. The concept was originally coined by Moses Schönfinkel, and later developed by Haskell Curry.

In Ruby, we might have a Proc taking multiple arguments:

f = Proc.new { |a, b, c| a + b + c }

We can call f with all of its arguments by saying f[1,2,3] which would evaluate to 6 in our case.

Currying f by hand would look like this:

curried_f = Proc.new do |a|
  Proc.new do |b|
    Proc.new do |c|
      a + b + c
    end
  end
end

We can now evaluate our Proc by running curried_f[1][2][3] which would evaluate to 6 exactly as in our previous example.

The ingenious reader have probably already guessed that #curry will take our original f and turn it into curried_f. We can curry f in the following way instead: curried_f = f.curry, and then finally call curried_f[1][2][3] which will unsurprisingly return 6 as before.

What We Talk About When We Talk About Rack

Sat, 27 Sep 2014 00:00:00 +0000

I was encouraged by Jamie Hodge to give a talk at this week’s Copenhagen Ruby Brigade meetup about Rack, and in this post, I’d like to give a recap on the subject.

Rack is essentially a minimal web server interface that Rails, Sinatra, Lotus, Cuba, Camping and friends all heavily rely on. They do so in order not to interact directly with the lower levels of the socket communication, and instead they distribute this particular work for Rack, so they can focus on other parts of the architecture. The main benefit of Rack is that you can write your applications once, and run them everywhere. Almost all Ruby servers support Rack, so you can easily power up your application without having to tailor it to a specific platform.

A Rack application is basically an object that responds to #call, accepts env as its only argument and returns an array containing the HTTP status code, the headers and an object that responds to #each. Using a stabby lambda, a simple Rack application printing “Hello, world” to its users could look like this:

require 'rack'
app = ->(env) do
  [200, {"Content-type" => "text/html"}, ["<h1>Hello, world!</h1>"]]
end
Rack::Server.start(app: app)

At some point, we probably want to extract our logic into its own class. This is quite manageable to do, since we simply need an object that responds to #call and takes env as its only argument. We could for instance end up with the following:

require 'rack'
class SuperAdvancedWebApp
  def call(env)
    [200, {"Content-type" => "text/html"}, [env]]
  end
end
Rack::Server.start(app: SuperAdvancedWebApp.new)

Except from being extracted into a class, the aforementioned application is slightly different in that it actually returns the result of the env to its visitors, so that we can investigate the contents of the current environment from the browser.

For learning purposes, I’ve built a small web framework utilizing Rack that discovers how we can effortlessly build a framework similar to Rails, Sinatra and so forth which I’ve named Dolphy, and you can study the source code on GitHub.

It is possible to avoid the dependency of Rack by using TCP sockets as I presented in my previous post, “Investigating sockets”. The incredibly small framework busker is an attempt at building a web framework without the dependency of Rack. I keep a branch called majjoha/rackless in the Dolphy repository where I try to remove Rack from the dependencies of the project in a similar manner.

Furthermore, Rack itself is also in an exciting state at this moment, as Aaron Patterson who is also a Rails core contributor recently took over the development of the project. This is what he said about the env hash back in August:

@jcoglan not just that, the mutable env hash passed everywhere is the bane of my existence.
— Aaron Patterson (@tenderlove) August 21, 2014

I am thrilled to see that we’ll finally get rid of the env hash from our Rack applications. In the repository, the_metal, he keeps a spike for thoughts about Rack 2.0, and according to the examples in the project, it’ll slightly change how we use Rack to build our web applications:

class Application
  def call req, res
    res.write_head 200, 'Content-Type' => 'text/plain'
    res.write "Hello World\n"
    res.finish
  end
end
require 'the_metal/puma'
server = TheMetal.create_server Application.new
server.listen 9292, '0.0.0.0'

I highly welcome the change, and I find this new way of interacting with the request and response directly much more elegant than fiddling with the env hash as we’re used to. It is going to be interesting to see how these changes will affect all the existing frameworks that we use today.

Investigating Sockets

Wed, 17 Sep 2014 00:00:00 +0000

During this summer, I’ve revisited an old side project of mine which is a Ruby-based micro framework for web development. Like most web frameworks in Ruby, this heavily depends on Rack which can be extremely helpful in a lot of ways, but you do not learn all the lower levels of how the server handles requests from clients and so forth that I find somewhat intriguing. This ultimately led me to investigate the different sockets that the Ruby standard library provides and in general improve my understanding on sockets. This post covers some of my findings from the process.

Sockets are the endpoints of bidirectional communication channels. The Ruby standard library has six different socket classes: UNIXSocket, UDPSocket, TCPSocket, Socket, IPSocket, and they all inherit from the sixth socket class BasicSocket. The class hierarchy for the sockets in Ruby looks like this:

Starting from the bottom, TCP is the most commonly used protocol on the Internet as it offers error correction, and like UDP it belongs to the transport layer. Furthermore, TCP sockets guarantee delivery which means that it will resend packets if needed and stop the data flow until packets are successfully transferred, so TCP sockets are considered very reliable.

Implementing a small single-threaded web server that prints the current time to the user could be done this way utilizing the TCPServer.

require 'socket'

server = TCPServer.new('localhost', 2345)

loop do
  client = server.accept
  client.gets
  client.puts "Time is #{Time.now}"
  client.close
end

Accessing http://localhost:2345 in the browser would then return the current time to the user. This is actually what I started out with when I was replacing Rack with sockets, and from here you need to figure out how to match URLs from the user, handle multiple users, send/receive POST and GET parameters, send the necessary headers to the browser and so on. This process reveals how much work Rack actually does for you, but I enjoy seeing how much I can do without it as well.

Moving on to UDP, this protocol, on the other hand, is considered unreliable, but it offers speed why it is often used for streaming purposes. Also, UDP packets are remarkably smaller than TCP packets. Unlike TCP, UDP does not provide any error correction or flow control, so errors will be present.

Since UDP does not do retransmissions, we might also lose content, or content might be ordered the wrong way as UDP does not guarantee in-order delivery. Thus, TCP is quite obviously the most appropriate protocol of the two to rely on when implementing a web framework without Rack.

Implementing a simple chat server and client using UDPSocket could be done the following way:

# udp-server.rb
require 'socket'

socket = UDPSocket.new
socket.bind('localhost', 33333)

loop do
  data, address = socket.recvfrom(1024)
  puts "From address: '#{address.join(',')}', message: #{data}"
end

socket.close

# udp-client.rb
require 'socket'

socket = UDPSocket.new

while message = gets.chomp
  socket.send(message, 0, '127.0.0.1', 33333)
end

socket.close

The server creates a new UDPSocket and binds it to localhost on port 3333. From here, it will simply run a loop listening for messages and print them. The client works in a similar fashion. It creates a new UDPSocket, runs a loop that receives messages from the standard input which it then sends to our server.

Moving up the class hierarchy, we have UNIXSocket, IPSocket and Socket where the latter will not be elaborated. A UNIXSocket basically represents a UNIX domain stream client, and what characterizes a UNIX domain stream is that it does not use any underlying network protocol for communication, so it has less overhead. It solely exists inside a single computer, and processes communicating with a UNIX domain stream needs to be on the same computer too. For these reasons, the UNIXSocket is obviously not suited for a web framework, but the UnixSocketForker by Ryan LeCompte is an interesting usage of the UNIXSocket in a Ruby context. Finally, the IPSocket which is also the superclass of TCPSocket and UDPSocket is not so exciting on its own and therefore a little difficult to leverage, but one common usage for the IPSocket is looking up the IP address of the host which we could do in the following way:

require 'socket'

ip = IPSocket.getaddress(Socket.gethostname)
puts ip #=> "192.168.0.100"

Investigating sockets have been an interesting journey so far, and I’ll certainly need to investigate them even more for this project in particular. I hope you’ve found this whirlwind tour of the Ruby sockets useful, and feel free to ping me if you’ve any questions.