Perl for DevOps: Mojo::UserAgent

There's no way anyone "doing devops" can work without needing to interact with products/systems remotely, and these days pretty much everything offers an API and that API is more than likely available over HTTP. The Mojolicious distribution provides a ton of useful modules for doing almost anything HTTP-related; both client- and server-side.

This post is going to mostly focus on writing client code with Mojo::UserAgent, and some utility modules that Mojolicious ships with. I will also touch on using Mojolicious::Lite for writing web interfaces and services, but deployment will be deferred until the next post, which will focus on Plack/PSGI, the various Plack servers available, and some of the web frameworks that support PSGI.

Mojo::UserAgent vs LWP::UserAgent

For the longest time, LWP::UserAgent has been the de-facto standard HTTP client library that the Perl community uses. But I've written before about why I prefer Mojo::UserAgent over LWP::UserAgent; it has both blocking and non-blocking interfaces, so you can write both kinds of applications and only need to know one API, should you get into writing non-blocking code in your devops journey.

But there's more.

Prototype with mojo

The mojo command is provided by the Mojolicious package, and is a great tool to query web services and pages. I wouldn't call it a curl or wget replacement (that's not its purpose), but it's a great tool for extracting data that is embedded in a response, for use in shell scripts, and for rapid prototyping before converting to Mojo::UserAgent in a Perl script.

As an example, if you wanted to use the MetaCPAN API to get the latest version of a package:

---

$ mojo get http://fastapi.metacpan.org/v1/package/Mojolicious
{
   "version" : "7.74",
   "module_name" : "Mojolicious",
   "dist_version" : "7.74",
   "file" : "S/SR/SRI/Mojolicious-7.74.tar.gz",
   "distribution" : "Mojolicious",
   "author" : "SRI"
}

---

This shows the JSON output of the API request. Mojolicious comes bundled with its own JSON decoding/encoding module, Mojo::JSON, which can be used directly in any application you want - perhaps as a replacement for the other JSON modules, if you so desired - but it's also integrated into Mojo::UserAgent, for decoding and easily extracting data from JSON responses.

The version is what I'm after. We can easily grab that with another parameter, utilising some simple notation.

---

$ mojo get http://fastapi.metacpan.org/v1/package/Mojolicious /version
7.74

---

But what if there is no nice JSON API, and we have to extract the same data from a web page? Well, we can do that too:

---

$ mojo get https://metacpan.org/pod/Mojolicious 'span[itemprop="softwareVersion"]' text
7.74

---

This fetches the Mojolicious documentation on MetaCPAN and looks for a span tag with an itemprop attribute value of softwareVersion and displays the text in the tag. In this case, we're pretty lucky that the MetaCPAN page gives us a friendly way to locate this data, but more complex queries can be used for less mojo-friendly websites.

The beauty of the mojo tool is that once you've prototyped how to extract the information that you want, you can either leave it in a bash script, or you can port the code to use the Mojo::UserAgent module and use it as part of a larger application.

---

#!/usr/bin/env perl

use v5.10;
use warnings;
use strict;

use Mojo::UserAgent;

my $ua = Mojo::UserAgent->new;
my $tx = $ua->get('http://fastapi.metacpan.org/v1/package/Mojolicious');

if ($tx->res->is_success) {
 say $tx->res->json->{version};
}

---

This is just a part of what the Mojolicious distribution has to offer; there's also an event loop and a promises implementation for writing non-blocking client and server code, and a whole lot more. Mojolicious wants to be a self-contained installation with as few external dependencies as possible, which makes it stable, and resilient to issues in the greater CPAN package ecosystem. Check out the other packages it provides.

Next in the series (whenever I get to it) I'll go through Mojolicious::Lite and Plack/PSGI, for when the time comes to write and deploy web sites and services.