Sunday, August 30, 2009

Ruby Hoedown '09

This weekend I was in Nashville, TN for Ruby Hoedown 2009.  There were some great talks -- my favorites were Ben Mabey's Cucumber talk, Luigi Montanez's talk on using public data APIs for civic coding, and Jim Weirich's excellent (and impromtu) talk on Git's internals.

After my recent difficulties with getting Rhodes working for mobile development, I was really excited about Leon Gersing's talk about Appcelerator Titanium, but I missed all but the last five minutes of it because the schedule was changed for some reason (I was skipping what I *thought* was something else). =/  Hopefully he'll post some slides.

The resort where the event was held was horrid, but the conference was a lot of fun.  Hope they continue this next year.

Monday, August 24, 2009

Erlang & Ruby -- Ring Network

As part of my Erlang self-education, I'm doing a selection of sample problems (from the "Programming Erlang" book) in both Erlang and my language of choice: Ruby. The idea is to explore the differences between these very different languages.

Today's problem is to write a ring network benchmark. Given a network of N nodes, where each node is pointing to the next in line (or back to the first), each node should forward a received message along, until it has gone around the ring M times (so that a total of N * M messages are sent).

To start with, I came up with an Erlang module to spawn a set of processes, and pass along a message record. It was pretty ugly at first, but with a little refactoring it turned out alright:
-module(ring_network).
-export([start_ring/1, start_message/3]).

-record(message,
  {loops_remaining=300,
    times_sent = 0,
    runtime, wall_clock, owner, note}).

start_ring(N) ->
  spawn(fun() -> 
        FirstPid = start_ring(N-1, self()),
        io:format("Starting node ~p (~p) pointing to ~p~n",
          [N, self(), FirstPid]),
        loop(N, FirstPid) end).

start_ring(N, Next) ->
  Pid = start_node(N, Next),
  if
    N > 1 -> start_ring(N-1, Pid);
    true  -> Pid
  end.

start_node(Id, Next) ->
  spawn(fun() -> 
        io:format("Starting node ~p (~p) pointing to ~p~n",
          [Id, self(), Next]),
        loop(Id, Next)
    end).

start_message(Pid, Msg, N) ->
  {Runtime, _} = statistics(runtime),
  {WallClock, _} = statistics(wall_clock),
  Pid ! #message{
    loops_remaining = N,
    runtime         = Runtime,
    wall_clock      = WallClock,
    owner           = Pid,
    note            = Msg}.

loop(Id, Next) ->
  receive
    Message ->
      case Message#message.owner == self() of
        false ->
          Next ! Message#message{
            times_sent = Message#message.times_sent+1},
          loop(Id, Next);
        true ->
          case Message#message.loops_remaining == 0 of
            false ->
              Next ! Message#message{
                times_sent = Message#message.times_sent + 1,
                loops_remaining = Message#message.loops_remaining - 1},
              loop(Id, Next);
            true -> 
              print_totals(Message),
              loop(Id, Next)
          end
      end
  end.

print_totals(Message) ->
  {R2, _} = statistics(runtime),
  {W2, _} = statistics(wall_clock),
  U1 = (R2 - Message#message.runtime),
  U2 = (W2 - Message#message.wall_clock),
  io:format("~n"),
  io:format("Sent '~p' ~p times in..~n",
    [Message#message.note, Message#message.times_sent]),
  io:format("  Runtime: ~p milliseconds~n", [U1]),
  io:format("  WallClock: ~p milliseconds~n", [U2]).



Next I applied the same pattern in ruby. I created a Node class, told it where the next node was, and had it forward messages along the proper number of times:
class Node
  attr_accessor :next_node, :number

  def initialize(number, next_node = nil)
    @number = number
    @next_node = next_node
  end

  def self.ring_network(node_count)
    nodes = Array.new(node_count)
    nodes[0] = Node.new(1)
    (1..node_count-1).each do |i|
      nodes[i] = Node.new(i+1, nodes[i-1])
    end
    nodes[0].next_node = nodes[-1]
    nodes[-1]
  end

  def gets_message(msg)
    if msg.owner == self.object_id
      if (msg.passes_remaining-=1) == 0
        msg.print_totals
        return
      end
    end
    msg.owner = self.object_id if msg.owner.nil?

    msg.total_passes += 1
    next_node.gets_message(msg)
  end
end

class Message
  attr_accessor :runtime, :wallclock, :note,
    :passes_remaining, :owner, :start_time, :total_passes

  def initialize(note, passes_remaining = 300)
    @total_passes = 0
    @start_time = Time.now
    @note = note
    @passes_remaining = passes_remaining
  end

  def print_totals
    puts
    puts "Sent #{total_passes} messages in..."
    puts "  WallClock: #{(Time.now - @start_time) * 1000} milliseconds"
  end
end



The first impression is just what you'd expect: Ruby is easier to read, and erlang is *much* faster:
1> Pid = ring_network:start_ring(10).
Starting node 10 (<0.59.0>) pointing to <0.68.0>
Starting node 9 (<0.60.0>) pointing to <0.59.0>
Starting node 8 (<0.61.0>) pointing to <0.60.0>
Starting node 7 (<0.62.0>) pointing to <0.61.0>
Starting node 6 (<0.63.0>) pointing to <0.62.0>
Starting node 5 (<0.64.0>) pointing to <0.63.0>
Starting node 4 (<0.65.0>) pointing to <0.64.0>
Starting node 3 (<0.66.0>) pointing to <0.65.0>
Starting node 2 (<0.67.0>) pointing to <0.66.0>
Starting node 1 (<0.68.0>) pointing to <0.67.0>
<0.59.0>

2> ring_network:start_message(Pid, hello_world, 300).
{message,300,0,270,298496,<0.59.0>,hello_world}
  
Sent 'hello_world' 3000 times in..  
  Runtime: 0 milliseconds
  WallClock: 3 milliseconds


Versus...
irb(main):001:0> node = Node.ring_network(10); msg = Message.new('Hi!', 300); node.gets_message(msg)

Sent 3000 messages in...
  WallClock: 79.336 milliseconds
=> nil


The real difference comes out when you increase the numbers:
(Erlang):
1> Pid = ring_network:start_ring(10000).
<0.33.0>

2> ring_network:start_message(Pid, hello_world, 10000).
{message,10000,0,460,40488,<0.33.0>,hello_world}
  
Sent 'hello_world' 100000000 times in..
  Runtime: 95350 milliseconds
  WallClock: 157455 milliseconds



(Ruby):
irb(main):002:0> node = Node.ring_network(100); msg = Message.new('Hi!', 300); node.gets_message(msg)
SystemStackError: stack level too deep
  from ./ring_network.rb:29:in 'gets_message'
  from ./ring_network.rb:29:in 'gets_message'
  from (irb):2
  from :0



In both cases, we're using recursion to wait for another message after processing one. In Erlang's case, though, it compiles tail recursion as a JUMP statement, which doesn't eat up stack space. Ruby doesn't do this, which is why I get a "stack too deep" when I up the number of loops. To get around this in ruby, I would have to move towards the Erlang module of forking off separate processes (not just separate instances), each listening for some interprocess communication.

Erlang & Ruby

I've been playing around with Erlang lately, and it's been quite a departure from the language I'm most comfortable with (and use everyday): Ruby. As part of my exploration, I'm doing a variety of sample problems (from the "Programming Erlang" book, among other places) in both Ruby and Erlang, and comparing the different implementations I come up with. Erlang and Ruby are meant to solve different types of problems, and the apples-to-oranges nature of the comparison should be kept in mind. The goal is to improve both my Erlang and my Ruby.

I'll be posting on this periodically under the tag erlang_ruby.

Wednesday, August 19, 2009

ActiveRecord Callbacks Trigger on Clean Objects

I came across this issue the other day at work, and I'm not really sure how I feel about it. It doesn't necessarily seem right or wrong, but was definitely unintuitive for me...

Since Rails 2.1, we have had dirty object checking, which prevents ActiveRecord from saving an object if if hasn't changed:
>> person = Person.first
  Person Load (0.9ms)   SELECT * FROM "people" LIMIT 1
=> #<Person id: 1, first_name: "Patrick", last_name: "Schless", nickname: nil, created_at: "2009-08-20 02:21:13", updated_at: "2009-08-20 02:22:32">

>> person.save
=> true

>> person.nickname = "nick"
=> "nick"

>> person.save
  Person Update (20.6ms)   UPDATE "people" SET "updated_at" = '2009-08-20 02:31:17', "nickname" = 'nick' WHERE "id" = 1
=> true


That part is good, but what happens when you have a callback defined?
>> Person.after_save { puts 'After save...' }
=> [#<ActiveSupport::Callbacks::Callback:0xb7147898 @identifier=nil, @method=#<Proc:0xb71479d8@(irb):6>, @kind=:after_save, @options={}>]

>> person.nickname = nil
=> nil

>> person.save
  Person Update (2.4ms)   UPDATE "people" SET "updated_at" = '2009-08-20 02:33:21', "nickname" = NULL WHERE "id" = 1
After save...
=> true

>> person.save
After save...
=> true


As we see there, the callback gets triggered even if the save is short-circuited.

In my trivial example, the behavior isn't terribly important either way. However, if your callback is doing something relatively expensive it may be pointless (and wasteful) to do that on a clean object. There are certainly situations where you would want it to always trigger, so I guess it's just one of those things that a developer ought to keep in the back of his mind when coding.

Monday, August 17, 2009

Vim -- :TOhtml & Customization

When I first started this blog, I was surprised (and disappointed) that Google's Blogger platform didn't have some nice built-in feature for code formatting and syntax highlighting. I experimented with a number of third-party solutions, but didn't find anything that I particularly liked, so my posts have had some ugly-looking code in them. Until now.

...Well, until my previous post, anyway.

A then-coworker told me a while back about vim's :TOhtml feature, but it wasn't until last night that I actually checked it out. As it turns out, it was just what I was looking for. Out of the box, it's a little cumbersome for my purposes, but a little vim scripting helped a lot.

Default Behavior

To start, I set a few options in my .vimrc:
let html_use_css = 1 " Use stylesheet instead of inline style
let html_number_lines = 0 " don't show line numbers
let html_no_pre = 1 " don't wrap lines in <pre>


I also choose a vim color scheme I like (wombat). You can look through examples of tons of schemes here. (I spent a while trying to figure out why I couldn't set the color scheme (after I downloaded it to .vim/colors/), and the answer I found was that color schemes only work in gVim.)

So, having set some options in my .vimrc, and chosen a colorscheme in gVim, calling :TOhtml on
class Person
  attr_accessor :age

  def speak
    "Hi!"
  end

  def say_age
    "I am this many: #{'|' * @age}"
  end
end


gives:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
  <head>
    <title>~/demo.rb.html</title>
    <meta name="Generator" content="Vim/7.2">
    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
    <style type="text/css">
      <!--
      .Identifier { color: #cae682; }
      .Type { color: #cae682; }
      .Statement { color: #8ac6f2; }
      .Constant { color: #e5786d; }
      .Function { color: #cae682; }
      .Special { color: #e7f6da; }
      .String { color: #95e454; font-style: italic; }
      body { color: #f6f3e8; background-color: #242424; font-family: monospace; }
      .PreProc { color: #e5786d; }
      -->
    </style>
  </head>
  <body>
    <span class="PreProc">class</span>&nbsp;<span class="Type">Person</span><br>
    &nbsp;&nbsp;<span class="Statement">attr_accessor</span>&nbsp;<span class="Constant">:age</span><br>
    <br>
    &nbsp;&nbsp;<span class="PreProc">def</span>&nbsp;<span class="Function">speak</span><br>
    &nbsp;&nbsp;&nbsp;&nbsp;<span class="Special">&quot;</span><span class="String">Hi!</span><span class="Special">&quot;</span><br>
    &nbsp;&nbsp;<span class="PreProc">end</span><br>
    <br>
    &nbsp;&nbsp;<span class="PreProc">def</span>&nbsp;<span class="Function">say_age</span><br>
    &nbsp;&nbsp;&nbsp;&nbsp;<span class="Special">&quot;</span><span class="String">I am this many: </span><span class="Special">#{</span><span class="Special">'</span><span class="String">|</span><span class="Special">'</span>&nbsp;* <span class="Identifier">@age</span><span class="Special">}</span><span class="Special">&quot;</span><br>
    &nbsp;&nbsp;<span class="PreProc">end</span><br>
    <span class="PreProc">end</span><br>
  </body>
</html>



That's pretty good. If your goal was a stand-alone HTML file you'd be done, but for paste-ready output we need a little vim scripting help.

Altered Behavior

Instead of wrapping the output in a full html page, I really just want it wrapped in a div. I also want to strip out some "extra" (according to Blogger's HTML interpreter) line breaks, and pull out the CSS definitions (for later). To accomplish this, I've put together a simple vim script, which defines a new method :DivHtml
function! DivHtml(line1, line2)
  exec a:line1.','.a:line2.'TOhtml'
  %g/<style/normal $dgg
  %s/<\/style>\n<\/head>\n//
  %s/.vim_block {/.vim_block {/
  %s/<body\(.*\)>\n/<div class="vim_block"\1>/
  %s/<\/body>\n<\/html>/<\/div>
  %s/<br>//g

  set nonu
endfunction
command -range=% DivHtml :call DivHtml(<line1>,<line2>)


Now, calling :DivHtml on the ruby code above gives code I can just cut & paste into the 'Edit Html' Blogger window:
<!--
.Identifier { color: #cae682; }
.Type { color: #cae682; }
.Statement { color: #8ac6f2; }
.Constant { color: #e5786d; }
.Function { color: #cae682; }
.Special { color: #e7f6da; }
.String { color: #95e454; font-style: italic; }
.vim_block { color: #f6f3e8; background-color: #242424; font-family: monospace; }
.PreProc { color: #e5786d; }
-->
<div class="vim_block"><span class="PreProc">class</span>&nbsp;<span class="Type">Person</span>
  &nbsp;&nbsp;<span class="Statement">attr_accessor</span>&nbsp;<span class="Constant">:age</span>

  &nbsp;&nbsp;<span class="PreProc">def</span>&nbsp;<span class="Function">speak</span>
  &nbsp;&nbsp;&nbsp;&nbsp;<span class="Special">&quot;</span><span class="String">Hi!</span><span class="Special">&quot;</span>
  &nbsp;&nbsp;<span class="PreProc">end</span>

  &nbsp;&nbsp;<span class="PreProc">def</span>&nbsp;<span class="Function">say_age</span>
  &nbsp;&nbsp;&nbsp;&nbsp;<span class="Special">&quot;</span><span class="String">I am this many: </span><span class="Special">#{</span><span class="Special">'</span><span class="String">|</span><span class="Special">'</span>&nbsp;* <span class="Identifier">@age</span><span class="Special">}</span><span class="Special">&quot;</span>
  &nbsp;&nbsp;<span class="PreProc">end</span>
  <span class="PreProc">end</span>
</div>


(which renders as the ruby snippet I pasted above)

The only part of the process that is still kinda annoying is that you have to copy/paste the CSS into your header or external CSS file. Both :TOhtml and :DivHtml only output the CSS definitions for styles actually used in a snippet, so it's up to the user to check if all the definitions in a new snippet are already in your layout, or if you need to add them. The alternative would be to use inline CSS (:help :TOhtml), but that's ugly.

To finish things off, I gave the wrapping div the vim_block class, and gave that a background-color and border in my header, to make it look a little nicer. I also got rid of the "font-family: monospace" bit, because I thought it made the code hard to read.

Other than manually checking the CSS, it's a very painless process to put code snippets on the blog, and I'm happy with the look and feel of them.

Sunday, August 16, 2009

Ruby on Android -- Part 2

Continuing towards my goal of developing an application in ruby for my Android (see part 1), I've been getting into Rhodes, which is an open-source project that aims to allow you to build native apps for a variety of mobile platforms, all from a lightweight ruby framework. For Android, it uses xruby (not Jruby).

I started with the documentation at the Rhodes wiki, and checked out a Google Talk. I tried to duplicate the sample application in the video, but quickly starting hitting walls, which I have been trying to work through. I haven't had a ton of success with it yet, but I'll document what success I have had.

To start with, rhodes only seems to support Windows and Mac out of the box. I have forked the rhodes project, to add in linux support, as well as try to fix whatever isn't working for me (which I'll get to later). You can check out my forked rhodes my github page. The easiest way to use the repo version (rather than the gems) is to set up an alias in your bash_profile such as:
alias rh="RUBYLIB=/home/rellik/repos/rhodes/rhodes/rhodes-build/lib:/home/rellik/repos/rhodes/rhodes/rhodes-framework/lib"


Prefacing any command with 'rh ' will then use the repo version of the Rhodes libraries.

After generating an application with rhogen (found in the repo under rhodes-generator/bin/ or the rhodes-generator gem), the other platform-specific bit that you need to override is the build.yml file that gets generated for a new project. Since I'm only concerned about Android at the moment, the only line I needed to change here was the last one, to point to the android.jar that came with my SDK. For me, this amounts to:
    android: /home/rellik/Desktop/android-sdk-linux_x86-1.5_r3/platforms/android-1.5


With all this, I was still not having much luck with Rhodes. I used 'rh rake run:android:app' to generate the apk and push it to the emulator. I was getting an error about no space left on the virtual SD card, but I do like having the apk be signed as part of the rake task, so I usually just run:android:app (and let it fail), then install the application manually using 'adb install -r bin/target/Rhodes-debug.apk' (the emulator must already be running).

Once the application is successfully installed on the emualator, you can slide the applications tray open and click on the Rhodes icon. You'll see a 'Loading...' screen, then a blank screen. For me, that's all there is. I tried simplifying the index.erb file, and even taking out any logic, but still I get the same blank screen. I renamed the non-dynamic index.erb to index.html, and that worked fine (I saw that page after 'Loading...'), so there seems to be something funky about rendering the ERB file.

To debug this issue, I added a series of raises and debug statements to the rake task that builds the apk, and I discovered a few things. First, the erb file gets converted to ruby, which contains the use of String#force_encoding (which is ruby 1.9). I tried downloading and using ruby 1.9 for this, as well as just taking that line out of a generated erb file, but neither helped.

Second, I left #force_encoding out, and I added "puts _erbout" as the last line of the generated erb file (bin/RhoBundle/apps/app/index_erb.rb), and then used the following to test if the generated java was any good, and indeed it was (but failed when I put force_encoding back in):

$ java -cp /home/rellik/repos/rhodes/rhodes/rhodes-build/res/xruby-0.3.3.jar:bin/RhoBundle.jar xruby.apps.app.index_erb.main
<div class="toolbar">
   <h1 id="pageTitle">
       Contacter
   </h1>
</div>

<ul id="home" selected="true" title="FirstApp">
 <li>Hello Android! (This was generated dynamically and can add ten and two to make: 15)</li>
</ul>


So, that's pretty much where I am so far. I can build an app with static views, and I can confirm that the java that gets generated for my dynamic views seems ok (but won't display on the emulator, despite the apparent lack of exceptions). Maybe the force_encoding bit is the problem (that'll be my next avenue to explore).

Tuesday, July 21, 2009

Ruby on Android -- Part I

Of course, now that I have a new G1, I want to start writing apps for it. Android apps are written in Java, but not having touched that since college, I started looking at ways to get ruby onto it instead. There was plenty of chatter about the possibility of running jruby on the device, but there are also frameworks that claim to allow you to write apps in ruby and push them to a variety of mobile platforms (android, iPhone, etc). I'll be hearing a talk about Appcelerator Titanium at Ruby Hoedown in August, but I haven't gotten into their closed beta, so I'll have to wait to try that out. The other (similar) solution I came across was Rhodes, which is available now (so that's what I'll start with).

For Android, Rhodes compiles your ruby project to an APK (Android package), which you can push to the Android emulator to test (and eventually to the marketplace). So, before I get to Rhodes, the first step is to set up a local Android development environment. I can skip a lot of the stuff Google suggests, since I won't actually be doing java work -- for this first step, all I want is to be able to take an APK and get it running in a local Android emulator. It took me a while to get this figured out, so hopefully my notes will be helpful to others.

The first step is to download the Android SDK from Google. The installation instructions include a lot of good stuff, but it most of it isn't needed for what I plan on doing. I already had Java 6 (RE) installed, which seems to work ok. Since I'm on linux, I followed the linux directions (all the way at the bottom), but the general directions for any OS amount to:
  1. Make sure you have Java 1.5 or 1.6 installed
  2. Install (apt-get) ia32-libs if you're on a 64-bit processor (linux only)
  3. Set up your PATH to include the android tools. (Mac/linux: in your home dir's .bash_profile, add "export PATH=${PATH}:/tools"; Windows: right-click on My Computer, and select Properties. Under the Advanced tab, hit the Environment Variables button, and in the dialog that comes up, double-click on Path (under System Variables). Add the full path to the tools/ directory to the path.)
You can ignore all the stuff about Eclipse.

Once you have all that done, you should be able to type 'android' and see some help text (you may need to 'source ~/.bash_profile', or just open a new terminal to reread your profile and get your new PATH).

The first step is to create a new virtual device. There are currently three versions of the Android software in the SDK: Version 1.1 (the original), Version 1.5 (the current), and Version 1.5 + Google API. You can see your available targets using:

$ android list targets
Available Android targets:
id: 1
Name: Android 1.1
Type: Platform
API level: 2
Skins: HVGA (default), QVGA-P, HVGA-P, HVGA-L, QVGA-L
id: 2
Name: Android 1.5
Type: Platform
API level: 3
Skins: HVGA (default), QVGA-P, HVGA-P, HVGA-L, QVGA-L
id: 3
Name: Google APIs
Type: Add-On
Vendor: Google Inc.
Description: Android + Google APIs
Based on Android 1.5 (API level 3)
Libraries:
* com.google.android.maps (maps.jar)
API for Google Maps
Skins: QVGA-P, HVGA-L, HVGA (default), QVGA-L, HVGA-P


You can find the details about various options when creating an Android Virtual Device (AVD) here, but the following is good enough to get you started:

$ android create avd -n first_avd15 -t 2
Android 1.5 is a basic Android platform.
Do you wish to create a custom hardware profile [no]
Created AVD 'first_avd15' based on Android 1.5


To start up your virtual device:

$ emulator @first_avd15

(I get an error here about audio input missing, but that may be because I don't have a mic on this computer.)

Now you should see a red generic Android device, with just the word "ANDROID" in the middle. On my system it takes a while, but eventually you see the "Press menu to unlock" screen, and then you can get to the home screen, where you can do normal Androidy things. To finish the test, grab a simple Android program in APK form (I used this simple app). Download it, and run:

$ adb -e install Constitution_1.0.0.apk (or whichever APK you got)
738 KB/s (534155 bytes in 0.706s)
pkg: /data/local/tmp/Constitution_1.0.0.apk
Success


On your emulator, go back to the home screen, and you'll now the the app in the lower tray, with everything else, and it should run without incident.

If you have any problems duplicating this process, feel free to ping me (comment on this thread). I worked this out over two nights, and had many false starts, so I could have forgotten to put something that I did to get past a problem.

My next step will be try and write a simple ruby app and get it to an APK, so I can put it on my emulator.

Part 2