Crystal Gotchas

User ID: mio
Subject: Crystal Gotchas

Recently I've been working on luce, a Markdown library for the Crystal Language. While working on this, I came across a few things which didn't quite make sense to me, so I though I'd write about them. Currently there are only two main… quirks that I've encountered.

The first is actually one that is already documented, so I'm not going to spend too much time on that. In short — an array of BaseClass can contain DerivedClass, but an array of DerivedClass won't always work where an array of BaseClass is accepted.

The other issue is one that I can't see mentioned in the above reference, so I'll write about it here. The below code is a simple version of a syntax parser, somewhat inspired by the one Luce uses.

class Syntax
  def can_parse?(value : Char, pos : Int32? = nil)
    puts "We are currently checking the Syntax class"
    value == 'S' || value == 's'
  end
end

class EmojiSyntax < Syntax
  def can_parse?(value : Char, pos : Int32?)
    puts "We are currently checking the EmojiSyntax class"
    value == 'C' || value == 'c'
  end
end

class Parser
  @@default_syntaxes = [
    Syntax.new,
  ]

  getter syntaxes = [] of Syntax

  def initialize(list : Array(Syntax))
    @syntaxes.concat list
    @syntaxes.concat @@default_syntaxes
  end

  def parse(str : String)
    str.each_char do |char|
      puts "Testing character: #{char}"
      # Enumerable#any? will return true if true
      # is returned from the passed block
      next if @syntaxes.any? { |syntax| syntax.can_parse? char }
      puts "Char '#{char}' won't work :<"
    end
  end
end

parser = Parser.new [EmojiSyntax.new]
parser.parse "sc sc"

Run this code

So what this should do, is compare each character of the string sc sc and see if any of the known syntaxes can parse it. There are a few calls to puts so we can determine the output. The expected output is:

Testing character: s
We are currently checking the EmojiSyntax class
We are currently checking the Syntax class
Testing character: c
We are currently checking the EmojiSyntax class
Testing character:
We are currently checking the EmojiSyntax class
We are currently checking the Syntax class
Char ' ' won't work :<
Testing character: s
We are currently checking the EmojiSyntax class
We are currently checking the Syntax class
Testing character: c
We are currently checking the EmojiSyntax class

Compare that with the actual output:

Testing character: s
We are currently checking the Syntax class
Testing character: c
We are currently checking the Syntax class
We are currently checking the Syntax class
Char 'c' won't work :<
Testing character:
We are currently checking the Syntax class
We are currently checking the Syntax class
Char ' ' won't work :<
Testing character: s
We are currently checking the Syntax class
Testing character: c
We are currently checking the Syntax class
We are currently checking the Syntax class
Char 'c' won't work :<

It's skipping the EmojiSyntax entirely! The reason may be obvious when looking at the above code since it only small. Either way, the issue is with how Crystal deals with overriding methods and method overloads.

Explanation

In Crystal, you can overload methods, so long as they meet the requirements. You can have two methods called foo where one takes a String parameter and the other takes an Int32 parameter, and they'd be different. No Questions asked. That makes sense. You can also have a version of foo which takes an Int32? (that is, an Int32 that could be nil) and another version that also takes an Int32?, but provides a default value of nil. Would these also be different methods? What happens if you call foo 13 or foo nil, or just plain old foo? Well in this case it depends on whether foo(Int32?) or foo(Int32? = nil) was defined last. (example)

You can also override methods when inheriting from another class. Nothing revolutionary, but suppose you have a method in a base class which provides a default value (i.e. Syntax#can_parse?(Char, Int32? = nil), and in a derived class a method with the same name that doesn't include the default value (i.e. EmojiSyntax#can_parse?(Char, Int32?))? Well judging from the example above in overloading, it should use whichever is defined last and — since the derived class is defined after the base class — it should call EmojiSyntax#can_parse?(Char, Int32?), right?

Sort of.

If you were to call #can_parse?('c') on an instance of EmojiSyntax, it would call the implementation of #can_parse? defined in the Syntax class — since we didn't provide a value for the Int32? paramter. If we were to call #can_parse?('c', nil) on an instance of EmojiSyntax it would call the implementation defined in EmojiSyntax — since both values are present.

The Solution

For such a lengthy description, the solution is incredibly simple. The easiest option would be to add the default value to the derived class:

class EmojiSyntax < Syntax
  def can_parse?(value : Char, pos : Int32?)
  def can_parse?(value : Char, pos : Int32? = nil)

Alternatively, when calling the method you could always provide each value — though this would defeat the point of providing a default value:

next if @syntaxes.any? { |syntax| syntax.can_parse? char }
next if @syntaxes.any? { |syntax| syntax.can_parse?(char, nil) }

You could also remove the default value from the base class, but that would break any previous code that didn't provide it's own value.

In the end, it would be nice if the Crystal compiler provided a warning in these cases, since other programming languages require you to specify that you are overriding the method. That said, there is news about Crystal 2.0 and it looks like the issue will be fixed (emphasis mine):

A few incompatible changes to language semantics were introduced between 0.36.1 and 1.0.0, and we do not want to repeat the same when 2.0 drops. For breaking changes initiated by Crystal code, the @[Suppress] annotation is one possible solution, but the same idea won't work for language-level changes such as redefining overload order.

The above eventually links to an issue describing as I have above! Yes, I found this link after I wrote all the above… so on that note, take care!

notes

Explanation

The Solution