Manually installing ASDF packages for SBCL

When I started getting back into Lisp earlier this year, after a bit of reading I quickly found out that ASDF was the more popular method for installing third-party packages. I also quickly found a problem though: simply typing (asdf-install:install 'package-name) doesn't always work, because some packages, while they have ASDF system files written, are not uploaded to - or are not up to date on - the remote site (cliki.net) and thus fail when you try to install it. So when you're out of options and decide to download or checkout the source for the package, how the hell do you install it?

In your home directory you should have a .sbcl directory. Inside this directory you should have two more directories: site and systems. To manually install a package:

• Unzip/untar/checkout the source code into its own directory inside the site directory. For example, on my machine cl-ppcre is unpackaged into .sbcl/site/cl-ppcre-2.0.1/
• Create a symlink from the ASDF system file to the systems directory. For example, .sbcl/site/cl-ppcre-2.0.1/cl-ppcre.asd is symlinked to .sbcl/systems/cl-ppcre.asd

That should be it. Now doing a (require 'cl-ppcre) in SLIME locates the ASDF system file in .sbcl/systems/ and loads the library as expected.

Note: If your SBCL doesn't look in systems for packages, you can check the asdf:*central-registry* variable for the correct directory.

Another note: If you prefer to install packages for the entire system rather than just your specific user, ASDF system symlinks live in /usr/local/lib/sbcl/site-systems (assuming you installed SBCL in /usr/local).

Last note, I promise: The obvious shortcoming of manually installing libraries is it won't automatically retrieve dependencies for you :(

Twitter shell tools

I've posted my current shell tools for interacting with Twitter here. Still a work in progress, obviously, but it's progressing none-the-less!

URL Shortening Services

I'm currently brainstorming some ideas on determining who is linking to a domain that I own via shortened URLS (e.g. tr.im, is.gd, ow.ly and similar). Without approaching the owners of the site to purchase that data ;) Basically I'm trying to answer the question "who is linking to my site from Twitter" since Twitter clients all use URL shortening services.

If the URL shortening service sends the shortened URL as a referrer in the HTTP headers, and the web browser should do exactly that, then we can use Twitter's search interface to search on those shortened URLs when they pop up in our access logs. The obvious shortcoming is if nobody clicks on a particular shortened URL, it will never show up in the access logs and therefor we won't know about it.

More brainstorming is needed. And probably some reading.

SBCL and Command-line Common Lisp Apps

Ok, first things first: read Xach's steps to creating small Common Lisp projects. Without it, I was unable to find anything as straight-forward as that to explain a process for creating Lisp projects. There is a lot of very useful stuff in there.

Second thing: I am not a Lisp expert so this may not be The One True Way to do things.

Finished reading? Good. Ok. Xach's article is not aimed at creating utilities to be run from the command line, but in full Lisp environments, like SLIME. Xach's guide gives you all the information you need as far as laying out your Lisp code and setting up ASDF system files are concerned. The last bits and pieces you need are concerned with the shell environment. So, you will need:

• Your code (obviously)
• An ASDF system file (from Xach's article)
• A Makefile to compile your code to .fasl and a .core (for SBCL)
• A shell script to run your .core file

The contents of the Makefile for a project named "my-project":

SBCL = /usr/local/bin/sbcl

all:
    $(SBCL) --eval "(require 'asdf)" --eval " \
        (progn \
         (asdf:oos 'asdf:load-op 'my-project) \
         (save-lisp-and-die \"my-project.core\"))"

clean:
    rm -rf *.fasl
    rm -rf *.core

This will compile all of the packages you defined in your ASDF system file into .fasl bytecode files and then save a .core file. Now all you need is a shell script to run the .core:

#!/bin/sh

SBCL=/usr/local/bin/sbcl

$SBCL --core my-project.core --noinform \
    --eval "(progn (my-project:main) (quit))"

And that'll do it! This assumes that your code has an exported defun, 'main', as the entry point for your program. Now run the shell script and your application will run.

Edit: Xach mentioned in the comments that you can use the :executable argument to save-lisp-and-die to create an executable, thus removing the need for the wrapper script above.

This is the build process I use for my little cl-mines game, so you can check out the setup there if there is anything that was unclear.

cl-mines - A Minesweeper Clone

I uploaded my Minesweeper clone, cl-mines to GitHub a few days ago. It's just a simple, command-line Minesweeper game. It doesn't really have a proper build process yet.

I've only tested it on SBCL, but there isn't any SBCL-specific code that I'm aware of and it is, by far, not the most complicated piece of Lisp code out there, so it should work with other Common Lisp implementations.

Releasing Some Code to GitHub

I'm going to be releasing a bunch of code to my GitHub over the next few weeks, after I clean up some of it and fix a few design issues and small bugs. This should include:

• Minesweeper clone
• Command-line Twitter tools

I may also release my Cellular Automata simlator aswell. However, at the moment, it has performance issues due to the Common Lisp LTK libraries. I have also started designing a Tetris clone (to be written in either Python+PyGame or Perl+SDL). Any updates to that will be pushed to GitHub aswell.

Cute Perl Gem to Get the Minimum/Maximum Value

Saw this little nugget on #perl@irc.perl.org the other night. It determines the minimum of two values:

[$b, $a]->[$a <= $b]

It takes advantage of the fact that Perl doesn't have a boolean return type for true or false, so the comparison operators return 1 or 0 for true and false, respectively, which are then used by this code to index the array ref.

To get the maximum of the two values, just flip the operator to >=

Net::Twitter::Lite 0.05

The latest version of Net::Twitter::Lite has pretty much made my tweet script redundant! Or at least removed the need for half of the code: http://gist.github.com/143950 Enjoy!

Following a Topic on Twitter with Net::Twitter

Last week I was keeping up to date with Triple J's Hottest 100 of All Time for 2009 and the hashtag to follow was #hottest100, so that's what I typed into search.twitter.com and kept refreshing every few minutes to see the latest thoughts as the countdown progressed. However, it very quickly became annoying to constantly refresh the search page to view the latest tweets. So I threw together this quick Perl script to display Growl notifications for the latest updates:

#!/usr/bin/env perl

use warnings;
use strict;

use Config::ApacheFormat;
use Encode qw/encode/;
use HTML::Entities;
use Mac::Growl;
use Net::Twitter;

use constant {
    TWEETRC             => "$ENV{HOME}/.tweetrc",
    MAX_RESULTS_DISPLAY => 5,
    POLL_FREQUENCY      => 30
};

die "usage: tweet-follow \n" unless @ARGV > 0;

Mac::Growl::RegisterNotifications(
    'tweet-follow',
    ['tweet'],
    ['tweet']
);

my $num_results_display = MAX_RESULTS_DISPLAY;
my $poll_frequency      = POLL_FREQUENCY;

if ( -e TWEETRC ) { 
    my $config = new Config::ApacheFormat;
    $config->read(TWEETRC);
    my $block = $config->block('follow');
    $num_results_display = $block->get('max_display_results');
    $poll_frequency      = $block->get('poll_frequency');
}

my ($query) = @ARGV;
my $twitter = Net::Twitter->new( traits => [qw/API::Search/] );

my $since_id = 0;
my $max_id   = 0;
while (1) {
    my $response = $twitter->search(
        {   q   => $query,
            rpp => $num_results_display,
            $since_id ? ( since_id => $since_id ) : ()
        }   
    );  

    unless ( defined $response ) { 
        printf "error while following: %s\n",
            $twitter->get_error->{error};
        exit 1;
    }   

    for my $tweet ( @{ $response->{results} } ) { 
        next unless $tweet->{id} > $since_id;

        $max_id = $tweet->{id} if $tweet->{id} > $max_id;

        Mac::Growl::PostNotification(
            'tweet-follow',
            'tweet',
            decode_entities( $tweet->{from_user} ),
            encode( 'UTF-8', decode_entities( $tweet->{text} ) ) 
        );  
    }   

    $since_id = $max_id;
    sleep $poll_frequency;
}

Usage is simple:

tweet-follow hottest100

Add something like this to your ~/.tweetrc to tweak some basic settings:

<follow>
    max_display_results 7
    poll_frequency 30
</follow>

The options are pretty self-explanatory:

• max_display_results controls how many tweets to post Growl notifications for.
• poll_frequency configures how often, in seconds, to poll for, and display, new tweets.

If only I could get Unicode characters to display properly in the Growl boxes. At the moment they just display as question marks for me.

I made a design decision to only display the latest N tweets, rather than displaying the next N tweets since the last poll, otherwise if you are displaying 5 tweets every 30 seconds for a topic that receives 50 new tweets every 10 seconds (think #iranelection or celebrity deaths), the script will get far behind very quickly.

Unicode and Perl

Tonight I was having problems with different character sets while hacking together some code to display and colorize Twitter search results on the command line (will post code another time) and ended up finally, after all these years with Perl, reading up about Perl and unicode (see perlunicode, perluniintro, perlunifaq and perlunitut).

The Encode module makes dealing with different character encodings trivial. I wish I had known about it before. I remember using the bytes pragma a couple of years ago because it was the easiest and quickest solution and I was (am?) lazy, but it is recommended to not use it.

Project Euler problem 28

DISCLAIMER: This post contains a solution to Project Euler problem 28. Go away if you don't want to know. DISCLAIMER 2: I am not the most mathematically minded person, so this may/will seem slow for pro mathematicians.

See here for the problem description.

So I got around to solving problem 28 last night (14 more until I get to level 2!) and, like most problems, I quickly found out the reason why this problem is on Project Euler: there's a subtle and beautiful pattern hidden in the middle of what seems like a random math problem.

The first pattern is that as the pyramid builds, the top right corners are always perfect squares of 1, 3, 5, 7 etc...

43 44 45 46 47 48 49
42 21 22 23 24 25 26
41 20  7  8  9 10 27
40 19  6  1  2 11 28
39 18  5  4  3 12 29
38 17 16 15 14 13 30
37 36 35 34 33 32 31

So we start looping from 1 to 1001 by 2s.

The other subtle pattern is that the gap between the corners also increments by 1, 3, 5, 7 etc... with each new layer of the pyramid. Therefor if we are at the top right corner which is n², the top left corner is n² - n + 1, the bottom left corner is n² - 2n + 2, and the bottom right corner is n² - 3n + 3, which follows an easy summation pattern which we can represent with a simple Lisp function:

(defun sum-corners (n)
  (loop for x from 0 to 3
     sum (+ (- (* n n)
               (* x n))
            x)))

Note: That can be simplified further into one equation, rather than a loop, if you like.

And a simple loop, described earlier, to tie everything together:

(defun euler-28 ()
  (1+ (loop for n from 3 to 1001 by 2
         sum (sum-corners n))))

Done. A simple problem disguised as something a little more difficult than it really is and has a couple of simple but cool patterns.

Vim-style Line Numbers in Emacs

Being a heavy Vim user throughout my university degree and now throughout work, I get very used to having the line numbers down the side (with "set numbers" in a .vimrc) as I hack code. To get something like this in Emacs, there is linum-mode.

You can toggle linum-mode for any Emacs buffer with the command M-x linum-mode. Or if you want to set it globally for every file you edit, add this to your .emacs file:

(require 'linum)
(global-linum-mode 1)

linum-mode should come pre-installed with Emacs 22 and beyond. It is also bundled with Aquamacs 1.7 (and maybe earlier...)