IconOf.Com

KOSTAS MAVROPALIAS

LEAD DESIGN ARCHITECT and DEVELOPER at CCKF. CYBERPSYCHOLOGIST.

From LAMP to the new full-JS web stack with Node.js, MongoDB, jQuery, Backbone.js and other cool stuff

in Development @ November 2, 2011

Since 1999 I have been developing my websites on the LAMP (Linux, Apache, MySQL, PHP) stack, and it has served me well. It was very easy to learn and get up to speed with PHP (a bit too easy maybe), while setting up Apache/MySQL for development work is trivial. The best part was, and still is, the plethora of straightforward and affordable LAMP hosting solutions, which have played a key role in the widespread adoption of PHP and its accompanying technologies.

As I like to keep myself up-to-date with new technologies and design/development paradigms, every time I want to learn a new technology I just create a new project (works better that a ‘hello world’ tutorial). When I was learning PHP/MySQL, I developed two gaming community websites: Noobwars and The-Core (the former has changed hands since then and the latter is no longer active), both full-blown news/community/CMS applications handwritten end-to-end. Then, as I was delving deeper into Flash & ActionScript, I redesigned my portfolio website in full-Flash (I know … I have seen the light since then!). When I wanted to get a better understanding of jQuery/CSS3/HTML5, I developed Penguspy.

Recently, I had an idea for a new project (more complex and involved), so I went out shopping for the latest and greatest in web technologies. With my background in PHP I started looking at PHP frameworks and one managed to impress me: Symfony 1.4. I went ahead, read the documentation, completed the tutorials and I was quite happy with it. But at that time (around May), a new version of Symfony was in the works; version 2.0. Based on the fact that my project will take about a year to go live, I thought it would be a good opportunity to start developing on the newer Symfony version, which is backwards-incompatible, in order to rip its benefits. Again, I went throught the documentation of the second version … twice. I want to emphasize their excellent documentation, where I learned lots of new things that will be useful to me even outside of Symfony. I particularly enjoyed the section on caching. In the end, I found Symfony 2 to be an amazing PHP framework, but it is so amazing (and so different than regular PHP) that in order to use it I basically have to re-learn everything from scratch and do things the Symfony-way. Well, if I have to learn everything from scrath I may as well go ahead and learn another language/framework.

I always had an interest in Python, which led me to the sexiness of Django. Django is great and coding in it is enjoyable. However, as the architecture of my project was maturing, I realised that I don’t really need an MVC framework running in the backend. I need my clients to run autonomously on many platforms (Android, iOS, Windows 8, Mac OS) and scale well across devices (mobiles, tablets, desktops), making HTML5/Javascript the only viable write once (almost), run anywhere solution.

Ultimately, I decided to develop a RESTful API on the server-side and move most of the business logic to the client, writing a middleware JavaScript library that will handle the client integration between the OS/device and the API, and creating the MVC architecture, again, on the client. This way, I only need to develop what is really needed on the backend (the API) and have a client code-base that is easily maintainable and can be ported to multiple platforms with little effort.

While I was deciding on the API approach. I came across Node.js and immediately fell in love with it. Its high responsiveness, asynchronous nature and performance, make Node an amazing platform to develop the API. As an added bonus I can re-use my experience in JavaScript on the server-side.

For the MVC client part I am using jQuery + jQueryMobile with Backbone.js and Mustache. Backbone.js has gained lots of momentum as the framework for client-side MVC apps, while Mustache provides a very good template engine for your HTML code. I also evaluated Sencha Touch as an alternative MVC engine, but I found that its cons (the significant learning curve,  the large file-size of the library and the fact that I would have to do everything the Sancha-way) outweighed the pros.

In regards to the database, the project requires lots of data manipulation due to its social nature and I was worried that a traditional relational database might not be the best solution, performance-wise, thus a NoSQL database seemed like a more appropriate fit (and of course, an opportunity to learn more about NoSQL DBs).

It was time to go out for NoSQL database shopping. Cassandra turned out to be the perfect match for the project’s requirements, however little support for Node exists as of yet. Instead, I settled down with the very close second choice MongoDB. What I find fascinating about MongoDB is that you get JSON objects as query results. Coming from a MySQL background, this is something very new for me and very welcome! With such a JavaScript-heavy app, getting JSON objects from the database makes the development process so much easier.

So, there it is, a full JS stack of technologies I am working with (and learning!) for the project:

  • Server:
    • MongoDB (database)
    • Node (web server & development platform)
  • Client:
    • jQuery + jQueryMobile
    • Backbone.js
    • Mustache (templating)
    • customProjectLibrary.js

This post turned out to be longer that I anticipated so I will leave out the part about choosing the right IDE, code repo/versioning, CSS/JS enhancers, hosting and more for a part 2. Overall, it is an extremely satisfying experience to work with and learn so many cutting-edge technologies on a very exciting project! I have learned so many new things in a very short period of time and although I had to spend some time searching for the most appropriate solutions, it was time well spent as it helped me improve my understanding on a number of development-related areas.

Although I am not an expert in Node (I am still learning), if you have been developing on LAMP since forever and want to try something cutting-edge and challenging (in a satisfactory way), I encourage you to give the full JS stack a try. It is a very different experience that might seem chaotic and overwhelming at first, but once you get used to it and understand the logic you will have a hard time going back.

  • http://www.facebook.com/shane.mcquillan Shane McQuillan

    Great post. I plan on using the same stack for my final year project which I shall be starting development on soon. I’ll be very interested to see how this works out for you.

  • Matt

    I’d love to know how you hang Backbone together with Node and MongoDB without using MVC on the server. Any chance of posting a sample?

    • http://www.iconof.com Kostas M

      Hi Matt. 

      All nodejs does on the server is the API stuff using json. Namely, it creates, reads, updates and deletes records & data in the database (mongo).On the client, I run an MVC using javascript (backbone, jquery, etc) that handles the UI templates, interactions and presentation of data. 

      I’ve also created a small javascript library (a single file) that is loaded in the client and creates an object that exposes some functions I’ve written, for quick data manipulation between the server and the client.

      For example, let’s say the project is called ABC and I have an ABC.js library. When it’s loaded it creates an ABC object that let’s me do things like: 

      – ABC.username (returns the logged-in user’s name as a string)
      – ABC.getNotifications (returns a json object with the notifications for the logged-in user)
      – etc.

      In ABC.js, there is a function (among others):

      this.getNotifications = function(callback) { … }

      Within the getNotifications function, I make a call to the web-server (e.g. to http://abc.com/user/userid/notifications?token=….), where Node receives the request, fetches the data from mongo and returns a json object with the data that is then passed-on to the MVC (or an error, if it was not successful). I use open auth to do the authentication between the client and the server. So, that’s all that Node does; fetches request and returns appropriate responses, leaving the client to take care of the rest.

      So really, backbone does nothing directly with Node. I could be using any other language in the backend instead of Node.

      Was that helpful and clear enough? :)

      • http://twitter.com/matt_ginty Matt Ginty

        Thanks, and sorry for the belated reply. I had forgotten I had posted a question and actually stumbled back to your blog when following your guide to getting node up and running on EC2 (best step-by-step article I’ve found on the process so far BTW – well done).

        I am actually running a similar setup for my learnings (Node + Express + MongoDB on the server and Backbone + Hogan on the client). As a .NET developer for the past 10 years it has been a rude awakening to the way things should be done and I’ve loving it… I’ve also made the move to CoffeeScript and LESS, both great additions to my stack.

        Thanks again for your reply,
        Matt

        • http://www.iconof.com Kostas M

          Thanks for the comments and for letting me know about Hogan.js! It looks promising; I’ll give it a try.

      • aBai Cheng

        “I use open auth to do the authentication between the client and the server……”, 
        Is open auth equal to OAuth? and how you do authentication between client and server?

        • http://www.iconof.com Kostas M

          Hi aBai,

          Yes, OAuth is a shortcut for open auth. The actual implementation is somewhat involved. Maybe you can try a library like jsOAuth: https://github.com/bytespider/jsOAuth

  • Darren

    Hi

    As a matter of interest what did use for “so I will leave out the part about choosing the right IDE, code repo/versioning, CSS/JS enhancers, hosting and more for a part 2″.

    Thanks
    Darren

    • http://www.iconof.com Kostas M

      Hi Darren, I use SublimeText, Git (on Github), SASS and I usually host my projects on Amazon EC2 & S3.

---