Drupal and Front-End Web Developer

Front-end build tools

Saturday, August 6th, 2011

Now that our Front-End group at work is growing and needs to be involved in more and more complex projects, we’re starting to see the need for some tools to help us enforce some of the best practices we work with when we need to do hand-offs to other developers (either Back-End people, or internal teams with our clients). So what can we find that makes for a good way of enforcing some standards without making all our developers run screaming from having to jump through too many flaming hoops.

What should it do?

  • Let us work locally with multiple files for CSS and JS. The idea being that managing portions of a site’s functionality and supporting third-party libraries is an easier task in multiple files than it is in one giant one.
  • Automatically allow for merging and compressing CSS and JS.
  • Give us an automatic path for replacing external file references (CSS/JS) so that after generating merged versions they can be automatically referenced.
  • Be as simple to install and maintain as possible.
  • Be as simple to invoke and understand as can reasonably be expected. (Not all our FE devs are comfortable dealing with things on the command line, so if we can keep it to a minimum that’s for the best.)
  • Give us a reasonable first-pass at doing some sort of automated testing. Even if it’s just validation against standards or *Lint tools or something of the sort. Real automated testing (using something like Jasmine or QUnit) is still a little high for a target off the bat.

Okay so that’s some (sort of) requirements, now what?

Well, the good news is that all our FE developers are on Macs, so that does let us aim at most anything technology wise. I’d like to keep it as simple to install as possible (since I don’t want to be the one responsible for managing software on other people’s machines) and ideally something we can keep in version control somewhere as a way to distribute updates.

Real build-y tools

Apache Ant

One of the things that initially sent me this direction was coming across this video by Addy Osmani where he walks through setting up a Front-End build process using Apache Ant. I loved the idea, but knowing the other people on our team the idea of working with a number of separate applications loosely joined via Ant didn’t seem like the best way to encourage a common (and easily updateable) standard across the team.

Ruby Rake

Ruby’s Rake tool on the other hand, looked pretty promising. All the supporting pieces (doing things like CSS merging, JS compression, HTML DOM manipulation) all had gems available for them which put all the software installation process in the same mechanism. Conveniently, that same mechanism could also be used for updates to those packages as well. (Also, I have a copy of Learn to Program around and that seemed like a promising start for doing it in Ruby.)

Finding tools on the Ruby bandwagon

So Rake provides us a peachy basis for doing our actual build script, what about all that other stuff?

CSS compression and merging

Juicer does a nice job of this. You point it to a CSS file and it will actually pull all the @importdirectives in it into the file itself automatically. It can also run things through the YUI compressor to get them all minimized down. Another helpful aspect of doing it this way: we’re actually overriding a default behavior for CSS itself, so we don’t need to use excess markup in the page for many different tiny CSS files and then try to rewrite the markup later.

JS compression and merging

Juicer actually does this too, and provides a handy @dependsdirective you can place in a comment in your JS files that will tell the parser what files to merge together and then compress. The only drawback here is that in our local working/dev environment we still need to be using multiple SCRIPTtags which will need to be stripped out later.

Removing pesky excess markup

It actually took me a couple tries to find something for this that really suited my purpose. Originally I tried Nokogiri since it’s big and powerful and whatnot, but I had a couple concerns around it that I knew weren’t going to fly with the other front-end types. In an HTML5 document it kept adding an additional METAtag for setting character encoding in a format that’s not actually necessary in HTML5, and if you left comments in a document (like to mark the end of a specific DIVor section of markup) it would break them to a new line with no indentation (which sort of defeated the purpose of trying to leave yourself a breadcrumb trail).

So, back off to loot the corpse of _why I went and grabbed Hpricot instead. It did exactly what I needed for the most part (grab elements matching a CSS selector and remove them from the page) but it left behind white space. Which is what RegEx is for anyway, so I was alright with that.

Low-level testing and validation

Once again, Juicer lent a hand here. It actually has a built-in facility for running JS files through JSLint.

For HTML and CSS files I went with the w3c_validators gem which makes service request calls against the W3c’s official validation tools and gives some feedback on what’s what.

Enough chit-chat, build the damn thing!

Once I’d found some tools that did what we needed I broke down our tasks into a couple buckets and namespaced them.

Build tasks

  • Build HTML- Strip out stuff that’s useful or necessary in a front-end dev environment that shouldn’t be part of the final codebase.
  • Build CSS- Merge and compress the stylesheets.
  • Build JS- Merge and compress JS files.
  • Build all- Do all Build tasks

Testing tasks

  • Test HTML- bounce HTML files against the w3c validator and let the user know if there’s an issue in one. Unfortunately the Errorobject that’s returned from the HTML validator doesn’t let you target specific issues so this only goes so far as “You need to review [file x]” in terms of messaging.
  • Test CSS- w3c_validator returns line numbers and specific error messaging here.
  • Test JS- Use Juicer to do a command line call to JSLint. It’s pretty detailed, but since it has to do stuff outside the actual Rakefile it can’t be colored and formatted as nicely as the HTML and CSS errors.

Image management tasks

  • Compress Images: Pops open your image directory (as defined in the YAML config file) using ImageOptim to help strip out any excess metadata and whatnot to get the file sizes down.
  • Image Sync: Since the Dev environment and the Build environment live in different directories, this is just a shortcut for syncing the images from Dev (where you’re supposed to be working, you are working aren’t you?) to Build (what you’d hand off to someone else).
  • Backwards Image Sync: In case something goes into the Build images folder that needs to get pulled to the Dev folder.

Version control tasks

I included a couple “helper” shortcuts for generating tagged releases in SVN and Git. (I know, making them in Git is essentially trivial, but I wanted to be consistent.) It’s an “interactive” process that asks for a name and comment, etc.

Setup

Setup rolls through the file structure defined in the config.ymlfile and makes sure that all those directories actually exist. Trying to do a build when they didn’t would cause things to puke and for people who weren’t sure how to decode one of Rake’s error messages it was a little concerning.

Things I’d like to improve

There’s two areas that really stick out to me in terms of what is in the script right now that I’d like to revisit at some point. First, I’d like to be able to clean up the messaging in the testtasks a bit. Unfortunately as it stands right now, since they return such radically different data (in one case “useful” and in another “essentially none”) there’s not a whole lot I can do on that one right now.

The second is that since I wasn’t able to find any real docs around using Juicer within other Ruby scripts (it was essentially all “do this on the command line”) that’s how the script works, which means that any of the “raw Juicer” tasks end up being something that can’t be hidden from view or given a pretty output.

Shut it and give me the code already!

Certainly, you can get the code on Github.

Need a developer?

Good news, I'm available! You can try taking a look at my resume to get a feel for the kinds of things I know how to do.

I'm reading...

Listening to...

Individual Member of the Drupal Association