We are the Dev Teams of
  • brands
  • ebay_main
  • ebay
  • mobile

Node.js and ES6 Instead of Java – A War Story

by Patrick Hund
in How We Do It
A thin layer

We created a Node.js web app from scratch within 9 weeks that now serves 10.81 million visitors each month*. We used the latest and greatest incarnation of JavaScript – ECMAScript 2015, a.k.a. ES6 – for the backend and frontend alike. We spent late nights coding in gleeful frenzy, stepped into pitfalls, moaned about WTFs, scaled the dizzy heights of asynchronicity. This is our story.

* AGOF Digital Facts 2015-06

Hi, I'm Patrick, software engineer. My team and I work for mobile.de, which is Germany's biggest online marketplace for cars and other vehicles.

In June 2015, we were given a task that every software developer dreams of: throw away all the old stuff and rewrite everything from scratch. The “old stuff” being the home page of mobile.de, the main entry point to our platform.

mobile.de before…

mobile.de before the relaunch

…and after the relaunch

mobile.de after the relaunch

Why Node.js?

We have traditionally been a Java shop since we migrated from a big steaming pile of Perl back in 2007. But engineers at mobile.de are encouraged to explore new technologies and empowered to choose the best tool for the task at hand and the given team setup.

It was not hard to convince the Powers That BeTM that we should go for Node.js, although it was still fairly new technology for us.

Our system architecture has been evolving from monolithic to service-oriented in the last years. The new home page was to become a fairly simple web application, basically a thin layer between a handful of service APIs and the client-side code. Node.js is great at integrating services and passing data through to the presentation layer, so it was an obvious choice.

A thin layer

A thin layer between services and client

We had a fairly tough deadline to meet, which is unusual for our company, but could not be avoided in this case. The new face of mobile.de needed to beam at the world on September 1st 2015, leaving us about 9 weeks.

At the time we started, our team included four JavaScript experts (including myself) and two Java experts, one of whom was on vacation until well into the project. After some experimentation with PayPal's kraken.js framework, I was confident that we could pull this off with much less deadline risk than using the “traditional” Spring/Java stack (more on why using Kraken turned out to be not such a great idea later).

Main concerns of our management were system stability, scalability and monitoring. We took a lot of effort to make sure these requirements were met. More on this in another blog article by my team mate Jonathan Krause (coming soon).

For further reading on why JavaScript is a good choice for backend and frontend alike, here is a link to a great article by Eric Elliott, How to Build a High Velocity Development Team, section “Why Language Matters”.

Setting Up the Build System

The Trouble With Kraken

kraken.js logo

The first phase of our project was to set up the basic scaffold of the application, including a build system that allows us to smoothly and quickly work on the software running in our local development environments.

When I evaluated PayPal's kraken.js framework early in 2015, I was euphoric about it: you run a Yeoman generator, and it sets up a basic skeleton app with Express, Dust.js templates, a Grunt build system, Bower package manager integration and a dev server that allows live reloading – everything we needed, ready to use within seconds. Heaven!

Unfortunately, when I set out to use Kraken in earnest, it turned out that the project had not been updated since April 2015 (v2.0.0-rc1). A new 2.x version has not been released ever since (at the time I'm writing this, October 2015).

So I had a choice to make:

  • use the old but stable 1.x version of Kraken
  • use the newer, but unstable RC version of Kraken 2
  • don't use Kraken framework as a whole, only single Kraken modules; set up the whole build system myself

My engineer's heart that wants to use the latest and greatest and try out cool and new stuff got the better of me and defeated the voice of reason (also known as “The You-Have-a-Deadline-Dude”) – so of course, I went for the latter option.

Engineering Man vs. Deadline Dude

Engineering Man vs. Deadline Dude

The reasoning behind this decision:

  • I did not want to risk being stuck with outdated software and be at the mercy of our (highly respected!) colleagues at PayPal to eventually provide an update that we can migrate to with reasonable effort
  • I wanted to use the latest version of JavaScript, ECMAScript 2015 (ES6) for backend and frontend, which would take a lot of customization to bake into the Kraken scaffold
  • Likewise, I wanted to use webpack for optimizing and bundling client-side resources; webpack had proven indispensable at my previous project, the relaunch of the search results page of mobile.de

webpack – Great for Frontend, Also Backend?

webpack logo

During my research, I came across an interesting series of blog articles by Mozilla developer James Long, Backend Apps with webpack.

James explores the possibility to use webpack not only to bundle and optimize client-side JavaScript and CSS resources, but also the backend JavaScript code that runs on NodeJS.

I set up our build system based on his article: npm run-script commands run Gulp tasks that use webpack for building and running the software.

I was aware that using webpack for the backend was an experiment, and it did turn out not to be feasible in the long run: maintaining the complex build configuration became too much of a hassle, so we recently decided to simplify things by using webpack only for the frontend.

For further reading on why webpack is awesome (for client-side resources), check out this blog article that I wrote on the subject.

Streaming Builds With Gulp

Gulp logo

We had previously used Grunt as frontend build tool in almost all our projects. Gulp is an interesting alternative, as it uses the streaming API of Node.js, allowing for some fascinating code solutions.

The Engineering Man in me still likes Gulp better than Grunt, but in hindsight, we should have gone with Deadline Dude's recommendation to stick to Grunt, as this would have allowed us to set up and maintain the build system much quicker – Gulp's streaming API has a steep learning curve.

It also turned out that some of the available Gulp plugins did not work 100% percent when running on our Jenkins CI system, causing hanging builds, which we only managed to fix by setting up some Gulp tasks to run sequentially. This defeats one of Gulp's selling points over Grunt, the ability to run tasks in parallel.

Demo: Local Development

To show you how nice it is to work on a Node.js project with a good build system, I've created a short demo video.

I especially like the live reloading provided by the webpack dev server that allows me to make changes to Less stylesheets and client-side JavaScript without having to hit the reload button of my browser all the time.

Thanks to sourcemaps, I can inspect my CSS and JavaScript code in the browser, which shows me the original ES6 and Less source files.

When I work on the backend, I also have very short turnaround cycles – nodemon detects changes of the backend ES6 code and restarts the application for me, which only takes 200-400 milliseconds. Compared to working on a Java application, where recompiling and restarting takes a perceived eternity, this is incredible.

To Be Continued…

That's it for now – I have lots more to talk about, so look forward to the other two parts of this series, coming soon to the eBay Technology Blog:

  • Part II: Practical application of ES6 features – working with Dust templates – MVC architecture with Express – unit tests with Mocha, Chai and Sinon
  • Part III: Lessons learned from building a Node.js application from scratch in 9 weeks

My team mate Jonathan Krause also wrote an excellent article about how we run our webapp in production (deployment, monitoring and logging): Node.js for the Real World

If you want to make sure you don't miss an episode, feel free to follow me on Twitter 😊


javascript, node.js, express