Update 2011-05-10: The code referenced in this post is now out of date. Please check Dojo Boilerplate’s AMD branch for up-to-date code.

For a large part of my Web development career, I’ve felt unease at the fact that every single Web app I’ve worked on has required two completely separate codebases: one for the server, usually written in a language like PHP, and one on the client, written in JavaScript. Over the years, thousands of frameworks have been written to try to streamline and automate this process, from crappy little AJAX page loaders to incredibly complex Java-to-JavaScript translators—but no matter how good any of these frameworks was, you were always writing to the language running on the server first, even though that language was (and likely always will be) incompatible with what runs in browsers on the client.

For the first time in over a decade, I’m happy to say that I believe this discontinuity has finally been solved in a way that will fundamentally change how all Web applications are written in the future. I say this because yesterday, for the first time ever, I succeeded in running a single JavaScript codebase seamlessly on both the client and server using a simple combination of Dojo Toolkit 1.6, Node.js 0.4.1, and RequireJS 0.24.

I’m sure this isn’t the first time someone has accomplished this, but the robustness of Dojo’s tools, the performance of Node, and the simplicity of RequireJS all combined in a way that made me feel that the time has finally come for Web development to shift to pure, seamless, fully-integrated JavaScript. Unlike past experiments that often involved gross hacks or the use of barely functional pre-alpha software, I would feel confident in deploying this solution to a production environment today. I hope this article gets others to start exploring and feeling excited about these technologies and how they’re changing Web development as we know it.

Dojo on Node.js

In Dojo 1.6, the Dojo team spent a huge amount of effort adding full support for the CommonJS AMD specification across all Dojo and Dijit modules; this change is the key that really puts Dojo worlds above any other toolkit when it comes to creating single codebase applications. Prior to this, Dojo was already the best RIA toolkit on the market, but running it in Node meant some ugly hacking to circumvent Node’s module sandboxing. With AMD, these hacks are no longer necessary.

Unfortunately, a tiny amount of modification is necessary in order to get Dojo 1.6.0 running in Node 0.4. Luckily, the changes amount to two one-liners and an alternative package main module, and these fixes are already scheduled to be added to the next version of Dojo.

RequireJS on Node.js

Node’s default module system is designed around the idea that dependencies can be loaded synchronously, which makes it a terrible choice for writing code modules that will run on the client. Swap it out with RequireJS—which still allows you to seamlessly use Node packages on the server-side—and writing highly modular code for both client and server suddenly becomes trivial. Roughly speaking, RequireJS provides a 1:1 mapping between filenames and defined modules, so if you had a component named my/app/FirstClass, you would just put it in my/app/FirstClass.js and RequireJS would load the definition from that file. Easy-peasy!

RequireJS also has support for basic loading of text files as dependencies and simple Object dictionaries for i18n. It also provides a highly robust build system, which means that when you’re ready to deploy to production, it’ll concatenate and minify all your code, intern external text, even let you split your code into arbitrary collections of modules that can be lazy-loaded on demand and minify your CSS. Amazing!

Putting it all together

In order to run a single codebase application using Dojo, RequireJS, and Node, you really only need 4 special things (and they really aren’t that special):

  1. A stub HTML file that loads RequireJS (index.html)
  2. A shell script that invokes Node with RequireJS (server.sh)
  3. A stub script that configures RequireJS for each environment and loads your main application code (js/my/_base.js)
  4. Some sort of logic within your application that branches depending upon the environment for any modules that need to load only in one place or the other (js/my/app.js)

In the examples given below, the application structure looks roughly like this:

  • app/
    • index.html
    • server.sh
    • css/
    • js/
      • lib/
        • requirejs/
        • dojo/
        • dijit/
      • my/
        • _base.js
        • app.js
        • nls/
          • app.js

There’s no particular requirement that the application be structured in this way, it’s just how I personally like to structure mine today.

index.html

A very basic HTML file; this loads RequireJS, then loads the application stub. Everything else is handled by Dojo/Dijit and RequireJS from this point on.

<!DOCTYPE html>
<html>
  <head>
    <title>my app</title>
    <meta charset="utf-8">
    <link rel="stylesheet" href="css/app.css">
  </head>
  <body class="claro">
    <!-- this will change to a single, minified
         JS file once you are in production -->
    <script src="js/lib/requirejs/require.js"></script>
    <script src="js/my/_base.js"></script>
  </body>
</html>

server.sh

A very basic shell script. r.js is one of the precompiled versions of RequireJS available on the download page and doesn’t come with the source download; if you do download the source version, use x.js from the bin directory, which serves the same purpose.

#!/bin/bash

JSENGINE=/path/to/node
REQUIREJS=js/lib/requirejs/r.js
REQUIREDIR=$(dirname "$REQUIREJS")

$JSENGINE $REQUIREJS $REQUIREDIR js/my/_base.js

js/my/_base.js

This configures Dojo by creating a global dojoConfig object, configures RequireJS, and kicks off loading of the main application code. Thanks to neonstalwart for his dojo-requirejs-template, which was the basis for this configuration code.

dojoConfig = {
  isDebug: true
};

require({
  baseUrl: 'js/',
  // set the paths to our library packages
  packages: [
    {
      name: 'dojo',
      location: 'lib/dojo',
      // these are loaded from js/lib/dojo/lib.
      // lib/main-commonjs is the alternative package
      // main module from ticket #12357;
      // you must place it there yourself (it does not
      // come with dojo yet)
      main: typeof window !== "undefined" ?
        'lib/main-browser' :
        'lib/main-commonjs',
      lib: '.'
    },
    {
      name: 'dijit',
      location: 'lib/dijit',
      main: 'lib/main',
      lib: '.'
    }
  ],
  // set the path for the require plugins—text, i18n, etc.
  paths: {
    require: 'lib/requirejs/require'
  }
});

// load the app!
require(['my/app']);

js/my/app.js

This is our application’s main application object. Really, you could load whatever you wanted from the stub, but this gives a very basic idea of what kinds of simple switching can be done to run different code depending upon the environment.

// requires dojo and our i18n dictionary
define(['dojo', 'i18n!my/nls/app'], function (dojo, i18n) {
  var isBrowser = dojo.isBrowser,
    mode = isBrowser ? 'Client' : 'Server',

    // our main application object; anything else that
    // requires 'my/app' in the future will receive this
    // object (because it’s returned at the end of this
    // function); all other defined modules work the
    // same way: the callback is invoked once and
    // the returned value is cached by RequireJS
    app = {
      onReady: function () {
        console.log(i18n.helloWorld);
      }
    };

  // loads either Client or Server class for Db and
  // Conduit depending upon if we are on the
  // client or server
  require(['my/db/' + mode, 'my/conduit/' + mode,
    'my/Baz'], function (Db, Conduit, Baz) {

    app.db = new Db();
    app.conduit = new Conduit();

    // this module works exactly the same on
    // both client and server, no extra code
    // necessary! NICE!
    app.baz = new Baz();

    // app has loaded, fire anything that has
    // connected to app.onReady!
    app.onReady();
  });

  return app;
});

I hope this article has raised your interest in Dojo, Node, and RequireJS, and that you’ll try building a unified codebase application of your own sometime soon. The ability to finally DRY out code between the client and server is here, and I’m incredibly excited about what it means for my applications and yours.

As a final note, if you’re a Web application developer and you’ve never used Dojo before (I see you there), you’re doing yourself a severe disservice if you are doing anything beyond very simple progressive enhancement of static pages. My colleague, Rebecca Murphey, will be doing a series of screencasts shortly on some of its killer features that make it the best choice for RIA development; I highly encourage you to watch her blog and check those out once they are available.

8 Responses to this post

  1. Nice work, perhaps now I'll have a reason to look more into Node - I've been putting it off for a while :)

    What do you think of Jaxer? It had this sort of feature, but then it faded into obscurity for some reason, and nowadays I don't think it's even supported anymore. It had an automatic proxy thing which would allow you to share code between client and server, and things that the client couldn't do (such as SQL) would automatically get proxied to the server.

  2. I've been very disappointed with server-side JavaScript for the most part. It hasn't been simple enough to avoid pushing peculiar requirements onto the developer.

    If there's one thing that SSJS needs next, it's good, clean software design. I'm hoping that there will be an apache module eventually that behaves a lot like PHP in terms of lifecycle.

    Node is a great start however.

  3. @Jani: I vaguely remember looking at Jaxer a long time ago and feeling quite underwhelmed for some reason. I don’t feel like SpiderMonkey is an appropriate engine for running production code on a server, especially these days. It’s even slower than PHP on nearly all of their own benchmarks, which is not great, since PHP is really the slowest language you should ever consider using in my opinion.

    Dojo actually seems to have support for Jaxer—I saw it when I was messing around in the loader—but like you said, it seems to be very much a dead project at this point. Luckily, Dojo has an fantastic data abstraction layer of its own that makes it very easy to plug in whatever you want and retrieve data using an agnostic query interface. (I was actually going to write more about how to do the server-client interaction, but my current app just uses a WebSocket interface, so I decided I needed a little more time to explore how traditional requests would best be completed before making any proclamations about how to do it.)

    @Alex: I’m not sure what sorts of peculiar requirements you’ve experienced; when working with Dojo and RequireJS, the code you write for the server looks and functions identically to the code you write for the client, which is very simple and straightforward to me at least. I’d be interested to know what you think is peculiar or problematic.

    I have to say that I don’t agree at all that an Apache module is something that should exist, especially one like mod_php which does a full buildup-teardown cycle every single time you handle a request. Node’s http server has fantastic concurrency and lets you actually, you know, bootstrap your app. Your database connection gets set up once, your routes get set up once, any helper components you need get loaded once, and any shared data (like common configuration data) can stay in memory without needing to use anything like memcached. PHP has an incredibly shitty lifecycle that means you’re constantly doing the same work over and over, even though probably 1/3rd of it is just application bootstrapping that should be done only once when the app starts up for the first time.

    More specifically to the suggestion of an Apache module, Apache httpd is quite slow and bloated, even when you can manage to get away from the prefork MPM and switch to the more efficient event MPM (which is never, if you’re using mod_php). If you don’t want Node to handle all requests, a simple reverse proxy is all you need, and nginx has got you covered there for everything except for WebSockets, which require HTTP/1.1 Keep-Alive in order to work (nginx’s proxy module currently only understands HTTP/1.0, so trying to proxy WebSockets just hangs).

  4. An elegant solution to this would be if we could use the same client side HTML5 APIs on the server. The biggest problem with Node and server-side JS is server-specific APIs that divide your code in two. HTML5 has the File API now, it has AJAX, it has database APIs (that could be rigged up to Couch or MySQL when code is running on the server).

    The server should just be a headless browser running the same code.

  5. @Colin

    Your points on PHP and Apache are totally right, and I agree. I wasn't saying that I wouldn't prefer node's approach. Perhaps I'm too comfortable with PHP. Indeed, lots of the things we try to do are to fix the things you mentioned.

    The previous SSJS implementations I've seen have had strange requirements as to what type of objects you have to create to bootstrap your app.
    Their dispatching bled into your own programs and put restrictions on it. Rather than being like node which is very flexible.

    Regardless, I'd love to see more comprehensive tutorials made. This is something I'll be watching closely. Thanks! :)

  6. Please can you provide archive with that project for fast start?

    Also I want say I also seek long time seamless JS stack to program web apps. Is it possible in that configuration work on it really as desktop application? I mean do not manually setup AJAX requests, send forms and process GET/POST? Something similar mentioned there GWT, but fully on JavaScript?

    Third question. What now with debugging such solutions? Is there present support to debug in seamless way server side JS together with client side? May be some plugins support in Netbeans (preferred for me) or Eclipse?

  7. @Hubbitus — Please see the Dojo Boilerplate link at the top of the article for up-to-date sample code. Dojo has many uniform mechanisms like dojo.store that abstract away the data layer very nicely (meaning no more manual AJAX requests and so on). There are also several ways to debug code in Node.js depending upon what environment you prefer to work in.

  8. Have you tried Dojo1.7.1 with require.js after 1.0.x
    I see your boilerplate uses dojo.js loader, but how about reference implementation from require.js?