Why (fork) Arel?

In a few recent posts, you may have noticed me mention my Arel fork on GitHub. I’ve never really devoted a post to discussing that fork, what’s different about it, and why it exists, so here’s some background.

Before we begin

I’m not advocating any kind of long-term fork of Arel. I’m just giving the reasoning for why I forked Arel for now — I fully anticipate that Bryan Helmkamp will continue his excellent stewardship of the Arel code.

The MetaSearch of yore (okay, a month ago)

The initial version of MetaSearch generated SQL fragments in the typical sanitize_sql fashion, assembling table and column names, and using ? for variable substitution.

Let me be clear: I hate SQL fragments in Rails code. Resorting to where('name LIKE ?', '%something%') is an admission of defeat. It says, “I concede to allow your rigid, 1970′s-era syntax into my elegant Ruby world of object oriented goodness.” While sometimes such concessions are necessary, they should always be a last resort, because once you move away from an abstract representation of your intended query, your query becomes more brittle. You’re now reduced to hacking about with regular expressions, string scans, and the occasional deferred variable interpolation trick (like ‘#{quoted_table_name}’) in order to maintain some semblance of flexibility.

It isn’t that I hate SQL (much). I’m perfectly capable of constructing complex queries from scratch, and did more than my fair share before coming to the Rails world. It’s that I hate the juxtaposition of SQL against Ruby. It’s like seeing your arthritic grandfather hand in hand with some hot, flexible, yoga instructor. Good for him, but sooner or later something’s going to get broken. It’s like a sentence which, tanpa alasan, perubahan ke bahasa lain, then back again.* It just feels wrong. It breaks the spell — the “magic” that adds to programmer joy, and for no good reason.

Anyway, as I was working on MetaSearch, I realized that everything would become a whole lot more easy and flexible if I just took advantage of Arel, which provided almost all of the SQL-generation functionality I needed… almost. It was (and is, as of this writing) missing a few complements.

Why Arel, you’re looking lovely today…

Not compliments. Complements.

com·ple·ment   [n. kom-pluh-muhnt; v. kom-pluh-ment]
–noun
1. something that completes or makes perfect: A good wine is a complement to a good meal.
2. the quantity or amount that completes anything: We now have a full complement of packers.
3. either of two parts or things needed to complete the whole; counterpart.

So the complement to lteq (<=) predicate is the gt (>) predicate, because it is true where the other is false, and vice versa. Arel had recently had a not (!=) predicate added, which was the complement of the eq (=) predicate. There was still no complement for matches (LIKE) and in though. So I added them. The notmatches and notin predication methods were born, and Arel was logically complete.

I modified MetaSearch to use these predicates, submitted a pull request on GitHub, and waited.

In the meantime, however, I couldn’t stop looking at Arel. I really enjoyed reading Nick Kallen’s writeup about why he wrote it, and once you sort of get a handle on the general philosophy underlying Arel’s code, there’s a sort of beautiful simplicity to it, and I found myself amazed at how easy its architecture made adding functionality. Arel’s a really awesome piece of code. I can’t say enough nice things about it.

Getting a little OCD about it

So as I was looking at Arel, I kept on tweaking it, drawing inspiration from the rest of its codebase. I wanted to add _any and _all versions of the predicates, to support multiple possible right-hand operands, and ended up refactoring the way that predication methods are defined as a result — all of these predication methods followed the same basic template, and defining them similarly to the way that Nick had implemented his code for deriving made a lot of sense to me.

As an added benefit, this would discourage making decisions that break this fundamental consistency in predication methods, pushing engine-specific details into the engine — which led to a refactor of support for ranges with excluded ends.

I started noticing other little details. For instance, the complement to Inequality was implemented as Not, and NOT has a special meaning in SQL-land. NOT is negation, not inequality. It seemed to make sense that Arel::Predicates::Not become Arel::Predicates::Inequality, and the predication method be changed from not to noteq, in keeping with notmatches and notin. So I made those changes as well.

Just for fun

If you’ve read my writeup on BasicObject#!, you’ll know what came next. All that staring at Arel code and thinking about complements led to Arel::Predicates::Predicate#complement, designed to be overridden by subclasses. This is in keeping with the original vision for Arel, I think, in that it allows for a restatement of problems. You can define an arbitrarily complex set of conditions describing what you don’t want in your returned set of data, then with a simple flip of a switch, turn that into a query that returns what you do want. For example:

Article.where(
  articles[:title].matches_any('hello%','%goodbye','%hi%').
  or(
    articles[:created_at].gteq(1.week.ago)
  )
).to_sql
 
=> "SELECT "articles".* FROM "articles"
   WHERE ((
     ("articles"."title" LIKE 'hello%'
       OR "articles"."title" LIKE '%goodbye'
       OR "articles"."title" LIKE '%hi%')
     OR "articles"."created_at" >= '2010-04-26 20:00:54.644537'
   ))
=> [#
]

This defines a query which matches any of three possible conditions on the article title, OR an article created_at timestamp greater than a week ago. To determine the complement, we can approach it as follows:

• The complement of a series of conditions joined by OR is the complement of each condition in turn, joined by AND. That is to say, where OR asks, “are any of these true?” its complements asks “are all of these false?”
• The complement of a LIKE condition is a NOT LIKE
• The complement of >= is <

So, Arel can do this for you:

Article.where(
  not(
    articles[:title].matches_any('hello%','%goodbye','%hi%').
    or(
      articles[:created_at].gteq(1.week.ago)
    )
  )
).to_sql
 
=> SELECT "articles".* FROM "articles"
   WHERE ((
     ("articles"."title" NOT LIKE 'hello%'
       AND "articles"."title" NOT LIKE '%goodbye'
       AND "articles"."title" NOT LIKE '%hi%')
     AND "articles"."created_at" < '2010-04-26 20:00:54.644537'
   ))
 
=> [#
, #
, #
,
   #
, #
]

Notice the absence of article 4, which was in the first query. Note: If you’re not running Ruby 1.9, you’d be unable to call not() in the manner I did above, but you could tack .complement or .not to the end of your method chain to access this functionality.

How it works

This is what is awesome about the way Arel was designed. The implementation of this was as simple as defining a complement method on each Predicate subclass in lib/arel/algebra/predicates.rb:

    class And < CompoundPredicate
      def complement
        Or.new(operand1.complement, operand2.complement)
      end
    end
 
    class Or < CompoundPredicate
      def complement
        And.new(operand1.complement, operand2.complement)
      end
    end
 
    class GreaterThanOrEqualTo < Binary
      def complement
        LessThan.new(operand1, operand2)
      end
    end

…and so on — you get the idea.

In closing

I truly hope that these additions get rolled into the upstream Arel repository soon. I only started this fork so that MetaSearch would work while I waited for the changes to make it upstream, and Arel’s already being maintained by some fantastic developers who are, frankly, better at this stuff than I am. In the meantime, I hope that this post gives you some insight into why the fork exists, currently, and why it’s necessary for some of the cooler features in MetaWhere/MetaSearch. Thanks for reading!

* “for no reason, changes to another language” — with thanks to , and apologies to native speakers of Indonesian.