alex kuang

professional yak shaver

The Compiler Is Your Friend

One sentiment I see from people coming from dynamically typed languages (python, ruby) or even Java and C is a general dismissiveness about static typing and compilers. At best, the sentiment is “Oh, well it makes sure my input is an Int, or a Float, or a Bool… That’s cool I guess, but I can do that with TDD”. At worst, static typing is seen as something to fight against – shackles that limit our creativity and bar us from what we really want to do and the beautiful programs that we could write if only the type-checker got out of our way.

Personally, I think of the type-checker (and by extension the compiler, really) not only as a free suite of rigorous tests built from first principles, but as a friend that is nice enough to correct me, the silly human, when I think I’m making sense but I’m really not. Sure, sometimes that friend is a bit dense (ahem Java, ahem) and can’t quite understand what I’m trying to say, but in the case of a language like Scala I find that the compiler is right more often than not. In fact, I had a whole giant wall of text geared up to talk about the value in using Option over null, in using Either instead of exceptions, in capturing values without using Stringly-Typed data… But then Li Haoyi beat me to it with another addition to his wonderful Strategic Scala Style series: Practical Type Safety. I highly recommend reading it (and the rest of the Strategic Scala Style series) before coming back.

Still here? That post covers a lot of what I wanted to say, but I wanted to put some extra emphasis on one particular topic:

Writing an API Client With Akka-http

Stockfighter is a CTF (short for Capture the Flag) game that I first heard about at Microconf 2015, but haven’t gotten a chance to play up until very recently. I plan on posting more about my impressions of the game later, but very shortly: it is a series of programming challenges based on the concept of stock exchanges and ways to manipulate stock exchanges. Along with the web UI, a public json API is exposed as a mechanism for interacting with the game. There did not seem to be any Scala clients floating around, so I took this as a chance to play around with akka-http.

Purity, Types, and Other Functional Nonsense

A while ago the topic of lenses came up in one of the chatrooms at work. This led to a discussion on the (ahem) suboptimal readability of advanced functional programming code examples, and how a lot of the fancier constructs are by-products of being super hardcore about purity. This led to me writing a great big email to the team that generated a whole bunch of thoughtful discussion on FP, complexity, and straw men with free lunches. I was also told I should transcribe to blog form, so here it is reproduced (mostly) faithfully:

I’ve been digging into Haskell/etc a bunch lately and have hit some similar pain points. Typeclasses and Haskell-specific syntax aside, I think one of the biggest mental shifts I have experienced is learning to reason based on types alone. Full disclaimer: I am by no means an expert on any of this. But since I am (at the original time of writing) sitting in an airport with a mildly disgruntled cat, I figure I may as well write up a blurb about my experiences on purity, types, and what can follow from it.

Hello, Blog!

Trying to take another shot at this whole writing-about-tech thing. :) Please excuse appearances while this blog gets revamped for the umpteenth time.

`gn` in Vim

Back to vim this week! This one might be a bit esoteric–apologies in advance–but it’s such a useful feature that I had to share.

gn is a kinda-sorta-but-not-really-motion that selects the matches of the current search pattern. It’s kind of like n (go to next match in search), except it also selects the contents of the match in visual mode. But where it really shines is when used with operators like c (change), etc to operate on the match. Quick demo to see it in action, before continuing:

So this is nice for a few reasons. First, it’s neat to be able to operate on just the match of a search. I remember a few times before I found gn where I was doing something similar to the demo, tried to use cw, and got mildly annoyed that I had to retype half the word because the w motion was too aggressive.

Another nice thing is that with gn, the dot command will both move to the next match and repeat whatever operation you did, instead of having to do the movement and operation separately using n.. It seems petty, but hey it’s a full 50% keystroke savings!!

Arguably, the demo above could have been done with an ex command like :%s/test/actual (universal search and replace), but I think gn still offers a lot. It works on all operators, so ygn would yank the match, gUgn would uppercase the match, and so on. Plus, it allows an easy on-the-fly workflow that’s still decently fast without the “crap I have to tune all the edge cases in my expression”: for example, if you want to only change some instances of test to actual, but others to sandbox.

As always, see :h gn for more details. Also note that gn has only been available since vim 7.4 or something like that, but there’s no reason anyone shouldn’t be using the latest on their personal boxes…

Bonus

This email also generated some additional discussion, which led to my discovery of the multi-select feature in Sublime, and the associated vim plugin. It seems like it’s in a different spirit than gn, but still a very cool way of thinking about operating on search results that I had not seen before.

Thread Macro

Let’s deviate from Scala for a bit and talk about clojure. Or lisps in general, I guess. A lot of the “kinda joking except not really” quips that commonly float around on the internet are about the parentheses, as in how there are so many of them. For example, if you want to take a number x and add one, multiply by two, then add 3, the code might naively look something like this:

1
(+ (* (+ x 1) 2) 3)

Or perhaps like this:

1
(+ 3 (* 2 (+ 1 x)))

Look at the parens! Especially the consecutive 3 closing ones in the second variation. For a sufficiently long chain of functions, it can get pretty unreadable—especially with multiple arguments and whatnot.

Enter clojure’s thread macro. The thread macro is a macro in the form of (-> x & forms), and it “threads” x through the forms as the first arg*. Which sounds terribly confusing explained, so an example is probably better here. Take this snippet using the thread macro:

1
2
3
4
5
;; add one, multiply by two, and add three
(-> x
  (+ 1)
  (* 2)
  (+ 3))

This desugars into (+ (* (+ x 1) 2) 3), i.e. the first variation of the initial example above. Personally, I find the macro version much more readable since each call is on its own line, and it seems more expressive of applying a series of functions to the initial x.

The thread macro is also useful for chaining together collection methods like map. Since clojure doesn’t have first-class OO support (instead favoring protocols and such), map exists as a regular function that takes the collection as an arg, instead of as a method on a collection class. So chaining together a bunch of ops on a vector might look something like…

1
2
3
4
5
6
7
;; add one to every number and filter for even numbers
(->> [1 2 3 4 5 6]
  (map #(+ 1 %))
  (filter even?))

;; Without the thread macro, would look like:
(filter even? (map #(+ 1 %) [1 2 3 4 5 6]))

The only difference is that in this case, ->> was used. ->> is “thread last”, which is like -> (“thread first”), except it inserts the expression at the end of the form.

This pattern also exists in other languages (especially those that don’t offer first-class OO, which allows fancy return self type stuff), like Elixir’s pipe |> (in the spirit of the unix pipe) which is what prompted me to spread the word about this:

1
2
3
4
# double and add one to each element
[1, 2, 3]
|> Enum.map(fn x -> x * 2)
|> Enum.map(fn x -> x + 1)

The thread macro pattern doesn’t have as much of a place in Scala, since Scala has mechanisms like the collection library and implicit conversions to help express similar logic in elegant ways. But when I first read up on macros in lisp, I spent some time scratching my head at the day-to-day practical uses until I found this and had my first “ohhhhhhhhhhhh” moment. In any case, hope this was mildly interesting!

* - Well technically, as the second item in the form, which is effectively the first arg for functions… But that might be a bit too lispy.

Bonus

When I published this email to the internal list, it generated some discussion wherein I learned that there are other neat features of the same sort like doto, and that they’re all just various derivations of the K combinator. Of course, googling k-combinators led to a pretty heavy looking wiki page, so I was referred to http://combinators.info/ , which I have been trying to get through since.

The Silver Searcher

grep

This week I’d like to talk about grep. Grep is a great general-purpose tool and works very well for filtering text in the middle of a long command chain, but I’ve found it a bit clunky as a codebase search tool.

For example, let’s say you’re sitting in some project and trying to grep for all the places where a function is being called. The naïve first attempt would be:

1
grep myFunc

Except that just hangs, since grep defaults to reading from stdin. A next attempt might be:

1
grep myFunc .

Except then grep would complain that . is a directory, which leads to:

1
grep –r myFunc .

Which finally works, but still leaves a bit to be desired. The biggest annoyance is that grep will get caught up in files that you don’t necessarily care about, e.g. Tags files, third-party dependency files, binary files… “Binary file ./lib/default/xxx.jar matches” anyone?

ag

Introducing ag, the silver searcher! Ag:

  • Fulfills the above “find this in cwd” use case via a simple, short ag myFunc
  • Is easy to install and super fast
  • Respects project ignore files: for example, it will ignore the patterns found in your .gitignore
  • In the case of files that you want in the repo but still don’t want to search, it also supports the use of a .agignore file
  • Integrates well into other tools: AFAIK there are ag plugins for vim, emacs, and text mate.

Fun fact: I was doing the whole grep –r thing for an embarrassingly long time before I bothered to search for a better workflow. My initial search turned up ack, which then led to ag. As far as I can tell, feature-wise they’re comparable; I eventually settled on ag just ‘cause the command requires less typing.

Vim % Expansion

% (Current File Name)

Another vim tip this week! This time, it’s about ‘%’, which expands to ‘current file name’. This is especially useful in projects with java/scala style directory setups, where your source is approximately 1.5 million folders away from the project root, but you kind of want to hang around project root for things like ant/sbt/etc to work. % makes this easier to work with files in the deeply nested folders while doing this.

Taking a contrived example, instead of doing something like this to git log the file you are currently editing:

1
:!git log src/main/scala/com/bizo/(…)/Foo.scala

You can just do:

1
:!git log %

This is extremely convenient and works everywhere in command line mode (basically, whenever ‘:’ is used), but is also useful to have if you’re ever writing vim script. See :h expand for the function to use in vim script, and some other special keywords.

But wait! There’s more!

Scala Enrichment

Douglas Crockford recently gave a tech talk at work, where he casually endorsed Scala during the Q&A at the end. Given that and the fact that I’ve been hearing increasing mentions of it in the company at large, I figured this week I’d plug a neat feature of Scala and a recent use case where I found it extremely handy.

The Feature

Anyone who’s worked with The AWS SDK (or just Java code in general) will be familiar with the builder pattern. Recently, I was writing some code to automate setup of CloudWatch alarms for a DynamoDB table. The alarm request started looking something like:

1
2
3
4
5
6
7
8
9
10
11
12
13
val req = new PutMetricsAlarmRequest()
  .withNamespace("DynamoDB")
  .withDimensions(dimensionsForTableName)
  .withStatistic("Sum")
  .withComparisonOperator(GreaterThanThreshold)
  .withThreshold(thresholdNumber)
  .withMetricName(throttledMetricName)
  .withEvaluationPeriods(periods)
  .withPeriod(periodDuration.toSeconds.toInt)
  .withAlarmActions(idMappingSNS)
  .withAlarmName(alarmNameFor(tableName, throttledMetric, throttledThreshold))

cloudwatchClient.putMetricAlarm(req)

Which is serviceable, but less than ideal given that I had multiple alarms of roughly the same nature. Plus, I felt like it left something to be desired in terms of readability and communicating the intent of the alarm in a succinct way. Enter implicit classs and the “enrichment”* pattern! Basically, it lets us turn the above into something more like this:

1
2
3
4
5
6
7
8
val req = new PutMetricAlarmRequest()
  .forTable(tableName)
  .triggerOnSumGreaterThan(throttledMetric, throttledThreshold)
  .afterEvaluationPeriods(evaluationPeriodDuration, evaluationPeriods)
  .withAlarmActions(idMappingSNS)
  .withAlarmName(alarmNameFor(tableName, throttledMetric, throttledThreshold))

cloudwatchClient.putMetricAlarm(req) /** Can still pass req back into AWS API! */

With the addition of this:

1
2
3
4
5
6
7
8
9
10
11
12
13
implicit class MetricAlarmRequestHelper(req: PutMetricAlarmRequest) {
  def forTable(tableName: String) = {
    req.withNamespace("DynamoDB").withDimensions(tableMetricDimensions(tableName).asJava)
  }

  def triggerOnSumGreaterThan(metricName: String, threshold: Int) = {
    req.withStatistic("Sum").withComparisonOperator(GreaterThanThreshold).withThreshold(threshold).withMetricName(metricName)
  }

  def afterEvaluationPeriods(periodDuration: Duration, periods: Int) = {
    req.withEvaluationPeriods(periods).withPeriod(periodDuration.toSeconds.toInt)
  }
}

Bash Bangs

This week I wanted to do a quick tip on some neat functionality in bash (and other bash-like shells), the bang commands.

Let’s say you have some long important command that you want to run. You run it, only to discover that you need sudo privileges. For situations like this, !! (entire last command) can be a great time-saver.

1
2
3
4
5
6
7
$ echo 1 2 3 4 5
1 2 3 4 5 # let's pretend echo throws an error too and wants sudo for some reason

$ sudo !!
sudo echo 1 2 3 4 5
Password:
1 2 3 4 5

Now, let’s say you don’t want to blindly re-run the last command. :p can be used to print it without overwriting the “last command” history.

1
2
3
4
5
6
7
8
9
10
$ echo 1 2 3 4 5
1 2 3 4 5

$ !!:p
echo 1 2 3 4 5

$ sudo !!
sudo echo 1 2 3 4 5
Password:
1 2 3 4 5

Bang commands also extend to the individual parts of the last command you ran. The most basic form of this is echo !:[n], where [n] is the nth word in the command, indexed from 0. There are also shortcuts: !^ gives the first arg (like !:1) and !$ gives the last arg.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
$ echo a b c d e
a b c d e

$ echo !:1
echo a
a

$ echo a b c d e
a b c d e

$ !:5:p # :p works with any bangs!
e

$ echo !:5
echo e
e

$ echo a b c d e
a b c d e

$ echo !$
echo e
e

Personally, I use !$ the most, since very often I’ll only want the last arg (e.g., ls [some tab-completed dir] -> rm -r !$). Plus, it’s the easiest sequence to hit, finger-wise.

One final tip: awesome shells like, say, zsh, will actually tab-complete bangs and get rid of the need for :p. e.g. sudo !!<TAB> gets replaced with sudo echo 1 2 3 4 5 in place.