Marc Schwieterman

Technical Ramblings

Modelines in Git Commit Messages

I’ve been cleaning up my Vim config this week, and I noticed some strange errors while doing an interactive rebase in my dotfiles repository.

1
2
3
Error detected while processing modelines:
line    1:
E518: Unknown option: <some text from the first commit message>

Because of the way I format commit messages for my dotfiles, the first line of the file presented during an interactive rebase was being interpreted as a modeline. I was hoping there was a command line option or something that I could use to disable modelines, but as far as I can tell, you have to do it in your .vimrc.

This isn’t likely to be a problem for most people, as I don’t imagine there are many that have vim: at the beginning of their commit messages. However if you run into this problem, you can solve it by adding the following to an appropriate place in your .vimrc

1
autocmd FileType gitrebase setlocal nomodeline

Return of the Daemon

I’ve been enjoying having the new Apple Music service in iTunes quite a bit, but one thing has been driving me crazy: the keyboard controls don’t work anymore. Except they do.

Before 12.2, iTunes would infuriatingly launch every time you plugged a set of headphones into your laptop. I eventually found a post that identified the Remote Control Daemon as the culprit for this behavior. Simply unloading the daemon fixed the problem. No more iTunes launching when it wasn’t wanted, and the keyboard controls still worked. Until now that is.

Fortunately starting the service back up is as easy as:

1
launchctl load /System/Library/LaunchAgents/com.apple.rcd.plist

Note that if you disabled the service to make double-sure it’d never run again, you may have to re-enable it by editing the following line in the plist.

1
2
3
4
5
6
7
8
9
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Disabled</key>
    <false/> <!-- double negate here! -->
    ...
</dict>
</plist>

So far my keyboard controls are working again, and iTunes has yet to launch when plugging in headphones. Let’s hope it stays that way.

The Initiative

My friend Bill and I have been working on a podcast about learning iOS development, and it’s finally here. We recorded several episodes before we figured out enough of the recording and hosting process to release anything, so there’s already a decent backlog. New episodes should continue to come out every Friday.

Check it out at The Initiative.

Bypass VPN by Hostname

If you run VPN on your router, you may have had cases where you want Internet traffic to bypass the VPN. Most solutions I’ve found use some form of a script originally found on LinksysInfo. However these approaches are IP based, which can be a problem if those IPs change, or if the service you’re accessing uses a CDN. If your router is running firmware that supports ipset, you can use dnsmasq and iptables to solve this problem.

This example uses release 112 of Tomato by Shibby. Some versions of utilities are not the latest available, so syntax may be slightly different than readily available man pages. The code snippets below include a commented reference to where they go in the admin GUI, but this is just what worked for me.

The core of the solution is to create an ipset for hosts of interest, configure Dnsmasq to populate that ipset and then use iptables rules to route outgoing traffic to those IPs over the WAN instead of over the VPN.

# Administration > Scripts > Init

ipset --create bypass_vpn iphash
modprobe ipt_set

The first thing you need to do is create an ipset and load the ipt_set kernel module. Here we’ve created an ipset named bypass_vpn of type iphash. Note that newer versions of ipset use a different syntax, hash:ip, for the set type. If you later get errors from iptables when trying to create rules with the ipset, you probably forgot to load the kernel module.

# Advanced > DHCP/DNS > Dnsmasq Custom configuration

server=/example.com/192.168.1.1
ipset=/example.com/bypass_vpn

#log-queries

Next you need to tell Dnsmasq to route DNS requests to a DNS server that is going to give you location appropriate results. This is particularly important if you’re trying to get traffic to use a local CDN. In the above example, example.com is our host of interest, 192.168.1.1 is the IP of a DNS server that will resolve that host to a desirable IP and bypass_vpn is the ipset to store the results in for later use by iptables. You will need a server/ipset pair for every hostname you wish to bypass the VPN.

When you’re first setting things up, you’ll probably want to also add log-queries to the Dnsmasq config, which will log DNS requests to the system log at /var/log/messages. In addition to hostnames you’re already aware of, be on the lookout for hosts referred to by CNAME records.

# Administration > Scripts > WAN Up

# The commands needed to create the routing table for packets marked with a
# 1 are not shown here. See the aforementioned LinksysInfo thread for more
# information.

# MARK = 0; all traffic goes through VPN by default
iptables -t mangle -A PREROUTING -i br0 -j MARK --set-mark 0

# MARK = 1; bypass VPN for bypass_vpn ipset
iptables -t mangle -A PREROUTING -i br0 -m set --set bypass_vpn dst -j MARK --set-mark 1

Finally you need to create rules to mark packets matching the ipset. In the above example, br0 is the interface the packets are entering through, bypass_vpn is the ipset containing the IPs to match and 1 is the mark value corresponding to a routing table that bypasses the VPN. This is not a complete iptables configuration. Only the packet marking rules are shown. Note that the --match-set option mentioned in the iptables man page did not work for me.

Testing in the OSX Sandbox

If you’re running Application Unit Tests, and have just enabled entitlements, you will immediately be greeted with an error similar to the following.

2012-11-26 20:16:56.352 SandboxFail[25356:303] Error loading /Users/marc/Library/Developer/Xcode/DerivedData/SandboxFail-arfkzlcptzkjqjcqelqwsfsxhasn/Build/Products/Debug/SandboxFailTests.octest/Contents/MacOS/SandboxFailTests:  dlopen(/Users/marc/Library/Developer/Xcode/DerivedData/SandboxFail-arfkzlcptzkjqjcqelqwsfsxhasn/Build/Products/Debug/SandboxFailTests.octest/Contents/MacOS/SandboxFailTests, 262): no suitable image found.  Did find:
    /Users/marc/Library/Developer/Xcode/DerivedData/SandboxFail-arfkzlcptzkjqjcqelqwsfsxhasn/Build/Products/Debug/SandboxFailTests.octest/Contents/MacOS/SandboxFailTests: open() failed with errno=1
IDEBundleInjection.c: Error loading bundle '/Users/marc/Library/Developer/Xcode/DerivedData/SandboxFail-arfkzlcptzkjqjcqelqwsfsxhasn/Build/Products/Debug/SandboxFailTests.octest'

It appears that test bundles cannot be injected into a signed application. After a couple failed attempts to sign the test bundle itself, I found a Stackoverflow post that recommended disabling code signing in a separate build configuration. It was not immediately obvious to me how to do that, so here’s what worked for me.

Create Build Configuration

The first thing to do is duplicate the Debug configuration to make one specifically for testing. This is done under the main project configuration, as shown below. I named my new configuration Test.

Duplicate Debug Configuration

Disable Code Signing

Code signing can be disabled in the build settings for the main target of the project. Make sure to expand the disclosure triangle and only clear the settings for the Test configuration. You don’t have to select or type anything specific in the values column. You can just click on Test and hit your keyboard’s equivalent of delete, which will restore the default values.

Disable Code Signing

Configure Schemes

After that you need to open the scheme editor (Command-Shift-<) and select the newly created Test configuration.

Update Test Scheme

Update Dependencies and Clean

If you use Cocoapods to manage your dependencies, you’ll want to run pod install to link the pod lib into the new Test configuration. After that just do a full clean (Command-Option-Shift-K), and you’re ready to run your tests. (Command-U)

RubyMotion Functional Tests with Storyboards

If you’ve been using RubyMotion, you probably noticed that some new testing features were released today. The video in the announcment post and the documentation are pretty good, so I’m just going to share a few things that I had to do to get things working.

main_spec.rb

Both the video and documentation have you add a test environment guard clause to your application:didFinishLaunchingWithOptions: method in the application delegate. Be aware that this will likely cause the window test in the default main_spec.rb to fail, so either delete that or anticipate some test failures due to the initialization changes.

Project Configuration

Update 2012/07/14: The specs_dir is searched recursively now, so the workaround that was previously in this section is no longer needed.

Storyboards

Update 2012/07/21: Storyboards are now supported by passing an :id option. The documentation has not yet been updated, but the pending updates can be found on the RubyMotionDocumentation github page.

If you’re using Storyboards, you’ll have to pass the Xcode identifier of the controller to the tests method in the :id option.

list_view_controller_spec.rb
1
2
3
4
5
6
7
8
9
10
describe 'list view' do

  # this call also extends the context with the test API methods
  tests ListViewController, :id => 'list-view-controller'

  it 'should have two cells by default' do
    views(UITableViewCell).count.should == 2
  end

end

Spurious Errors

You may occasionally see errors like the following.

1
2
3
4
5
6
7
8
...
3 specifications (3 requirements), 0 failures, 0 errors
*** simulator session ended with error: Error Domain=DTiPhoneSimulatorErrorDomain Code=1 "The simulated application quit." UserInfo=0x10014de60 {NSLocalizedDescription=The simulated application quit., DTiPhoneSimulatorUnderlyingErrorCodeKey=-1}
rake aborted!
Command failed with status (1): [DYLD_FRAMEWORK_PATH="/Applications/Xcode.a...]

Tasks: TOP => simulator
(See full trace by running task with --trace)

I suspect this is the 0.3 second delay condition mentioned in the comments for the functional test code. Clearly the universe is telling me to get the new retina MBP.

OSX Lion Key Repeat in IntelliJ

I’m mildly obsessed with Vim, so of course I prefer to use similar input methods whenever available. I’ve been working on a Javaish project lately, and I ran into the alternate character feature in OSX Lion, which does not play nice with the IdeaVim plugin for IntelliJ.

After getting tired of repeating keystrokes a ridiculous number of times, I did some searching and came across many explanations of how to globally disable the feature. While the lack of key repeat is a problem in IntelliJ, I could imagine alternate character input being useful in other applications. Fortunately I found a helpful blog post with a fix for specific applications in the comments.

Here’s the exact terminal command for IntelliJ.

1
defaults write com.jetbrains.intellij ApplePressAndHoldEnabled -bool false

Testing Warden-Based Sinatra Apps

I recently integrated Warden into a web service I’ve been building with Sinatra. Unfortunately doing so completely broke some of my tests. I spent a while trying to figure out how to stub out the Warden object before I discovered that Warden already provides support for testing. Awesome. While I could load my config.ru in the test, there are other things in there that I’d rather not deal with while testing. I came up with the following approach, which lets me more or less test the web service in isolation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
require 'rack/test'
require 'sinatra'
require 'rspec'
require 'warden'

# model
class User
  attr_reader :id
  attr_reader :name
  def initialize(name)
    @id = 1 # please don't really do this
    @name = name
  end
end

# modular sinatra app
class Greeter < Sinatra::Base
  get '/' do
    "Hello, #{request.env['warden'].user.name}"
  end
end

# tests
describe Greeter do
  include Rack::Test::Methods
  include Warden::Test::Helpers

  after(:each) do
    Warden.test_reset!
  end

  def app
    Rack::Builder.new do
      # these serialization methods don't do anything in this example,
      # but they could be necessary depending on the app you're testing
      Warden::Manager.serialize_into_session { |user| user.id }
      Warden::Manager.serialize_from_session { |id| User.get(id) }
      # your session middleware needs to come before warden
      use Rack::Session::Cookie
      use Warden::Manager
      run Greeter
    end
  end

  it 'says hi to me' do
    login_as User.new('Marc')
    get '/'
    last_response.body.should == 'Hello, Marc'
  end
end

This is basically just an inline rackup file. Where you would normally return just the Sinatra app, you instead put together the bits that you need to exercise the code under test.

Roo-based Jersey Apps on GAE

This past weekend I put together an example of a Spring Roo-based Jersey application that can run on Google App Engine. All code in this post is derived from the example project at https://github.com/marcisme/jersey-on-gae-example. You should be able to get a project up and running by copy and pasting the snippets in this entry, or you can clone the project and follow along. This example uses Spring Roo 1.1.1, Jersey 1.5 and App Engine SDK 1.4.0. A basic understanding of the involved technologies is assumed.

Project

The first step is to create your project. Create a directory for the project, fire up the Roo shell and then enter the commands below. This will both create a new project and configure it to use App Engine for persistence. Make sure to substitute an actual app id for your-appid if you plan to deploy this example.

1
2
project --topLevelPackage com.example.todo
persistence setup --provider DATANUCLEUS --database GOOGLE_APP_ENGINE --applicationId your-appid

Dependencies

Once you’ve set up your project for GAE, you’ll need to add the dependencies for Jersey and JAXB. If adding them manually, you can refer to the dependencies section of the Jersey documentation. In either case, you’ll need to add the Jersey repository to your pom.xml.

1
2
3
4
5
6
<repository>
    <id>maven2-repository.dev.java.net</id>
    <name>Java.net Repository for Maven</name>
    <url>http://download.java.net/maven/2/</url>
    <layout>default</layout>
</repository>

Jersey

Once you’ve added the Jersey repository, you can use these command to add the dependencies to your project.

1
2
3
4
dependency add --groupId com.sun.jersey --artifactId jersey-server --version 1.5
dependency add --groupId com.sun.jersey --artifactId jersey-json --version 1.5
dependency add --groupId com.sun.jersey --artifactId jersey-client --version 1.5
dependency add --groupId com.sun.jersey.contribs --artifactId jersey-spring --version 1.5

Unfortunately the jersey-spring artifact depends on Spring 2.5.x. Because Roo is based on Spring 3.0.x, you need to add some exclusions to prevent pulling in incompatible versions of Spring artifacts.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
<dependency>
    <groupId>com.sun.jersey.contribs</groupId>
    <artifactId>jersey-spring</artifactId>
    <version>1.5</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
    </exclusions>
</dependency>

JAXB

App Engine has issues with some versions of JAXB. I found 2.1.12 to work, while 2.1.13 and 2.2.x versions did not. This will hopefully change in the future. You can add a dependency on JAXB with the following command.

1
dependency add --groupId com.sun.xml.bind --artifactId jaxb-impl --version 2.1.12

Spring

If you’re adding jersey to a project that uses Roo’s web tier, you’ll already have the spring-web dependency in your project. If not, you’ll need to add that too.

1
dependency add --groupId org.springframework --artifactId spring-web --version ${spring.version}

You can specify an explicit version, but if you created your project with Roo, the spring.version build property should be set. Either way you’ll want to exclude commons-logging.

1
2
3
4
5
6
7
8
9
10
11
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-web</artifactId>
    <version>${spring.version}</version>
    <exclusions>
        <exclusion>
            <groupId>commons-logging</groupId>
            <artifactId>commons-logging</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Maven GAE Plugin

For some reason Roo uses a rather old version of the Maven GAE Plugin. The latest version at the time of this writing is 0.8.1. In addition to what Roo will have created for you, you’ll want to bind the start and stop goals if you plan on running integration tests. See the pom.xml in the example project for an example of how to do that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<plugin>
    <groupId>net.kindleit</groupId>
    <artifactId>maven-gae-plugin</artifactId>
    <version>0.8.1</version>
    <configuration>
        <unpackVersion>${gae.version}</unpackVersion>
    </configuration>
    <executions>
        <execution>
            <phase>validate</phase>
            <goals>
                <goal>unpack</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Entity

Next you’ll want to create an entity, which you can do with the commands below.

1
2
3
4
5
6
7
enum type --class ~.Status
enum constant --name CREATED
enum constant --name DONE

entity --class ~.Todo --testAutomatically
field string --fieldName description
field enum --fieldName status --type ~.Status

In order to leverage Jersey’s JAXB serialization features, you’ll need to annotate your entity with @XmlRootElement. Set a default value for status while you’re here.

1
2
3
4
5
6
7
8
9
10
11
12
import javax.xml.bind.annotation.XmlRootElement;

@RooJavaBean
@RooToString
@RooEntity
@XmlRootElement
public class Todo {

    @Enumerated
    private Status status = Status.CREATED;

...

Resource

Here’s a very simple resource for the entity we created earlier. In addition to the JAX-RS annotations, you also need to annotate the class with @Service, which makes the class eligible for dependency injection and other Spring services. This resource will support both XML and JSON.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
package com.example.todo;

import org.springframework.stereotype.Service;

import javax.ws.rs.*;
import java.util.List;

@Service
@Consumes({"application/xml", "application/json"})
@Produces({"application/xml", "application/json"})
@Path("todo")
public class TodoResource {

    @GET
    public List<Todo> list() {
        return Todo.findAllTodoes();
    }

    @GET
    @Path("{id}")
    public Todo show(@PathParam("id") Long id) {
        return Todo.findTodo(id);
    }

    @POST
    public Todo create(Todo todo) {
        todo.persist();
        return todo;
    }

    @PUT
    public Todo update(Todo todo) {
        return todo.merge();
    }

    @DELETE
    @Path("{id}")
    public void delete(@PathParam("id") Long id) {
        Todo.findTodo(id).remove();
    }

}

Final Configuration

At a minumum, you’ll need to have Spring’s OpenSessionInView and Jersey’s SpringServlet filters set up in your web.xml. As with the spring-web module dependency, you will have to create a web.xml in src/main/webapp/WEB-INF if you don’t already have one.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
<web-app>

    <!-- Creates the Spring Container shared by all Servlets and Filters -->
    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>classpath*:META-INF/spring/applicationContext*.xml</param-value>
    </context-param>

    <!-- Ensure a Hibernate Session is available to avoid lazy init issues -->
    <filter>
        <filter-name>Spring OpenEntityManagerInViewFilter</filter-name>
        <filter-class>org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>Spring OpenEntityManagerInViewFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

    <!-- Handles Jersey requests -->
    <servlet>
        <servlet-name>Jersey Spring Web Application</servlet-name>
        <servlet-class>com.sun.jersey.spi.spring.container.servlet.SpringServlet</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>Jersey Spring Web Application</servlet-name>
        <url-pattern>/*</url-pattern>
    </servlet-mapping>

</web-app>

You may also need to change the project’s packaging type to war.

1
2
3
4
<groupId>com.example.todo</groupId>
<artifactId>todo</artifactId>
<packaging>war</packaging>
<version>0.1.0.BUILD-SNAPSHOT</version>

Testing

You can now compile and run the application locally with mvn clean gae:run.

1
2
3
> mvn clean gae:run
...
INFO: The server is running at http://localhost:8080/

The following curl commands can be used to interact with the running application.

list
1
curl -i -HAccept:application/json http://localhost:8080/todo
show
1
curl -i -HAccept:application/json http://localhost:8080/todo/1
create
1
2
curl -i -HAccept:application/json -HContent-Type:application/json \
  http://localhost:8080/todo -d '{"description":"walk the dog"}'
update
1
2
3
4
curl -i -HAccept:application/json -HContent-Type:application/json \
  http://localhost:8080/todo \
  -d '{"description":"walk the dog","id":"1","status":"DONE","version":"1"}' \
  -X PUT
delete
1
curl -i http://localhost:8080/todo/1 -X DELETE

Once you’re happy with your application, you can upload it to App Engine. If you download the example project I created, you can build locally with mvn clean install -Dtodo-appid=<your-appid>. The local App Engine server will be used to run some basic integration tests during the build. After the app is built, you can deploy it with mvn gae:deploy. You will be prompted for your login information if you have not set up your your credentials in your settings.xml. Once deployed, you can run the included integration test against the live server with mvn failsafe:integration-test -Dgae.host=<your-appid>.appspot.com -Dgae.port=80. There is also a simple client you can use to write your own tests or experiment with.

Having some level of integration tests for any GAE apps you write is very important, as there are things that do not work consistently between the local development server and the real App Engine servers. Be aware that you can access the local development server console at http://localhost:8080/_ah/admin, which will let you browse the datastore amongst a few other things. Once deployed to the real App Engine, you’re best source of information is the application log. Be sure you’re looking at the correct version; there’s a drop-down menu in the upper left area of the screen that will let you choose the version of the logs you want to examine.

Beware the Empty String

Regardless of how you feel about empty strings, you should be aware of the implications of allowing them on objects that will be persisted. While it seems reasonable to expect a database to store the exact value that you give it, that is unfortunately not always the case. If you’re going to allow empty strings on persistent objects, you should take the time to determine the behavior of the database management system you’re using. You should also assess how likely it is that you are going to either have to support multiple databases, or that you will switch to another database management system in the future.

Different database management systems handle empty strings inconsistently. Some of them will persist and later return an empty string. Others convert the empty string to a null, and some will even turn it into a single space. If you’re not aware of this behavior, you may be in for some fun debugging if the object containing the empty string is used in a set or as a map key. Values that should be unique may appear to be duplicates or just disappear. Be particularly cautious of fields used to determine object equality. Below is a table of a few database management systems and the values that they will persist when presented with an empty string.

Database Persisted Value
MySQL 5.0 Empty String
Oracle 11g Null
Sybase 15.5 Single Space

If you must or want to allow empty strings on persisted objects, you need to determine how to handle them. The main decision to make is if you will support both null and empty strings for the same value. Due to the inconsistent handling across different database management systems, you’ll need to map empty strings into a value that can be persisted reliably. If you don’t need to support null, you can use that. Otherwise you’ll need to use some magic string or an extra field. The most important thing is to be aware that this issue exists, so you don’t waste your precious time figuring out why your objects aren’t given back to you the way you left them.