An Isomorphic Architecture for ASP.NET MVC
The decoupling of our back-end and front-end earlier this year led us to the creation of an innovative new platform architecture
Our new development approach was born out of a desire to overcome the headaches in the development workflow between front-end and back-end developers working in different technologies.
At the same time, we had long desired the dynamic user interface provided by a Single Page Application, but were unwilling to make the necessary compromises in SEO indexability and initial page load time. With our front-end tightly coupled to MVC’s server-side view engine, any experimentation was difficult.
- Decouple the front-end and back-end to create a clear separation of concerns and reduce duplication of efforts
- Combine the benefits of a Single Page App with the search-engine indexability and the speed of server rendering
- Create a platform-agnostic REST API that could be consumed not only by our web-application, but future native applications too
- Give front-end developers the freedom to autonomously develop and deploy front-end features
Our development team came together at the start of the year to think about how we could solve these problems collectively. We didn’t have a clear concept of where our goals would take us, but we began by looking into each problem in more detail.
Since the emergence of specialised front-end and back-end roles in web development, a commonly seen workflow has front-end developers working in conjunction with design and UX team members to produce HTML templates, while back-end developers concoct the database and behind the scenes application logic. At some point of course, the two worlds collide — the templates must be merged with server side code to create a dynamic application.
This process became a major pain point for our development team — a never entirely satisfactory workflow whereby pristine HTML was broken up into Razor templates, at the discretion of the back-end developer.
The final dynamic templates never quite matched the intention of the front-end developer — the breaking up of HTML templates into partials was usually driven by the back-end view of the world — and the inherent freedom of the Razor templating language meant that it was simply too tempting to put what should be server logic into templates. Logic-heavy templates indicated a poor separation of concerns, and the resulting mangling of C# and HTML became unapproachable to our front-end team should additional changes in markup need to be made.
There began the problem of duplication of effort: Front-end templates are built in isolation, but only Razor templates go into production. Every enhancement or bug fix needed to be made back within the static HTML environment before repeating the entire process to put it back into production.
This process became a major pain point — a never entirely satisfactory workflow whereby pristine HTML was broken up and inserted into Razor templates, at the discretion of the back-end developer
With a .NET back-end, a front-end developer using a Mac could only achieve a full-stack development environment by running Visual Studio through a sluggish Windows virtual machine, in addition to needing either a local or shared remote database in order to see their templates running in a production-like environment. This never proved a reliable workflow, and would often result in front-end code changes being pushed before they had been properly tested in the belief that it would be quicker for back-end to fix any issues than it would be to continually maintain the front-end developer’s VM setup each time configuration or DB changes had been made to the .NET solution.
Overall, we estimated that this duplication in effort was costing us between a half a day to a day per significant feature, while general frustration arising from the workflow did little to contribute to team morale.
- Use language-agnostic data formats wherever possible (e.g. JSON)
Shared Logicless Templates
The Handlebars templating language was in many ways the starting point for our whole solution. Our lead front-end developer had worked with it before in previous efforts to modularise bloated PHP templates in WordPress, and enjoyed the strictness that it enforced through its intentionally limited set of available logic (
#each), the idea being that if you find yourself needing to do anything more complex, the template is not the place for it.
Using Handlebars, we broke up our design into distinct reusable "modules", which could function in any context and in any order. Another level down, smaller reusable pieces of markup such as buttons and SVGs were turned into “snippets” for minimum duplication of code.
A Handlebars Module
A Global Data Structure
Handlebars is completely unopinionated about data structure and therefore allows complete freedom in the data models which are passed to it.
One of the drawbacks to our previous Razor templating solution was difficulty in accessing data that may not have originally been exposed or intended for a particular template. To make sure this problem was accounted for early on, we designed a global structure for our view models that would ensure that data remained organised and globally accessible at any time, by any module.
Consider the following object, made available to a Handlebars template:
site object contains all site-wide configuration data and text that isn’t specific to a particular entry or view (i.e. title, meta description, Google Analytics ID, feature toggles, etc).
entry object contains the data for the current resource that is being viewed. For example, the home page, or a particular film. As the user navigates around the site, its data is updated to reflect the currently viewed resource.
state object contains data pertaining to the state of the application, i.e. the currently active page or tab, or whether or not a modal is currently open and so on.
user object contains non-sensitive data about the currently signed-in user, such as their name, avatar, purchase history, etc.
module object allows us to import arbitrary data into a particular module without exposing it to any other modules. In other words, it is restricted to a module’s local scope. This is particularly useful within a loop of modules or when we want a generic module to be able to function in various different contexts by importing data into it.
We ensured that data over the API followed the exact same structure, which resulted the sharing of a single data service between the view controllers and API controllers on the back-end. Any data received via the API by the front-end would therefore be exactly the same as that used by the back-end when rendering a view.
Transpiled View Models
Our C# view models had always been the responsibility of the back-end team with the front-end team communicating any changes needed as new features were added (e.g. a new property being added, or existing data being made available where it wasn’t before).
This wasn’t particularly efficient and needed careful attention from both back-end and front-end to ensure that the desired properties were made available with the correct data and named consistently. It also broke our development goal of allowing front-end to push new features in their entirety when possible.
Retrospectively, it seemed obvious that front-end should be responsible for designing view models when creating or updating templates, and more generally responsible for determining the structure and semantics of data delivered to the front-end.
Both languages being fundamentally C-like, we were able to convert from one to the other comfortably through enumeration, type checking and a not-insignificant amount of regex
The final C# view models are then transferred into the .NET solution. When a view is requested, the appropriate data is mapped into them before being combined with Handlebars modules for rendering.
A C# View Model
We now had to find a way of dictating which modules should be rendered for a particular view and in what order, a task previously done by Razor “master” pages. Our first attempt was to hard-code arrays of modules into our view controllers, but this meant duplication of code between our back-end and front-end controllers, which again prevented the front-end from having complete ownership of the presentation layer of the application.
As these lists contained only data and not functionality, we decided that JSON would be the ideal format allowing the layouts to be easily consumed by either side of the stack. At this point we had basic JSON “layout” files that looked something like this:
However, we soon realised that if we added a certain amount of simple logic to our layout files, code duplication elsewhere could be drastically reduced and rendering performance improved.
Firstly, we wanted to render certain modules only if specific conditions in our data model were met. For example, a user menu should only be rendered if the user is signed in:
We also wanted the ability nest arrays of modules within other modules, and import arbitrary data into a module, which when combined with basic Handlebars-inspired logic such as
each would enable a powerful and expressive way of describing view structure and logic using a tree data structure.
The following example showcases the full range of functionality that evolved throughout the process of our rebuild:
A JSON Layout File
It’s worth noting that in the case of
unless, we could just have easily wrapped the entire HTML of the module in question within a Handlebars
#unless block, but apart from feeling like a hack that would reduce the readability of our modules, we realised that evaluating this logic earlier during the layout file parsing stage would avoid more expensive template evaluation later on by Handlebars, increasing rendering performance considerably.
The final class that we arrived at receives a JSON layout file along with data from a controller in the aforementioned structure, and iterates through the layout file rendering out modules, resulting in a the concatenation of a single, fully rendered HTML string.
We took the opportunity to write both implementations as group programming exercises between front-end and back-end in the hope of sharing knowledge
As the Page Maker iterates through each module in the layout file, it performs the necessary logic, loops and imports as instructed. If a module contains nested modules, it will recursively drop down a level and evaluate those modules before continuing with the original list.
Front-end Single Page Application
Our team had combined experience in Angular, Backbone, Knockout, and Durandal as potential candidates for a framework that could handle the various aspects needed for our front-end Single Page Application (SPA). This was however not long after the infamous Angular 2.0 announcement, and investing time to tie us to a third-party framework that could soon become obsolete was now very much a questionable decision.
Our lead front-end developer had recently began work on building an SPA framework from scratch as part of a side-project and the various lessons learned in routing, controllers, dynamic user interfaces and data-binding could easily be applied to our new architecture. We decided therefore to forge ahead with our own solution, with the peace of mind that fully understanding all aspects of our stack would lead to faster development in the long term, and the ability to swiftly adapt the framework the needs of an evolving platform — with minimal hacking.
We decided therefore to forge ahead with our own solution, with the peace of mind that fully understanding all aspects of our stack would lead to faster development in the long term
Much of our front-end SPA framework follows the typical MVC pattern, with the exception that the server has already rendered the view in its entirety before the application takes over.
- After the initial server rendering of a view, the application starts up in the background
- An identical data “scope” is created (via the API) to what has just been rendered by the server
- All link clicks are intercepted by a History API-enabled router
- The router then assigns a controller, coupled to a specific layout file
- The controller updates the entry scope via the API, and instantiates a Page Maker to render the view
A system of “behaviors” define and manage dynamic, replaceable user interfaces with isolated event-binding and garbage collection (similar to React), and also encompassing the functionality to data-bind elements to respond to changes in the scope, reusing the Handlebars logic already in the templates. DOM events are used to signal data changes rather than using observables or digest loops.
To maximize performance on the front-end, we implemented extremely aggressive caching (and pre-caching) of API resources including the data itself as well as all other components necessary to render views (layouts, modules and snippets). We also took advantage of HTML5 Storage to persist this entire cache between sessions, with the ability to flush specific elements on-demand when updates are made either to the database, or the front-end codebase.
Front-end & Back-end Development Workflow
We now had a fully independent front-end application containing the various shared components such as modules, layouts and view models. The problem remained of how to integrate those shared components into the .NET solution so that the full application could be deployed into production.
Initially we simply copied and pasted all relevant files into the solution, committed the code and deployed — but the inefficiency and error-prone nature of this workflow led us toward the idea of making the front-end a NuGet package that could be pulled in via our continuous integration server on TeamCity.
When front-end code is pushed, TeamCity creates a nuget package from the front-end Git repository. The files to be packaged are defined by a
.nuspec file which also lives in the respository. TeamCity then hosts the files on its own NuGet server, ready to be pulled down.
Whenever front-end changes are pushed and ready to be integrated, the package can be pulled in via the Nuget tool within Visual Studio. Once installed, a powershell script is run automatically to copy files into all relevant folders within the solution and remove any previous versions.
Node.js Development Environment
A crucial part of solving the initial workflow problems was the creation of a front-end development environment that was as close to the production back-end as possible, while avoiding use of Visual Studio or Windows virtual machines.
When we first prototyped the Handlebars module rendering concept, we built a very lightweight Node.js application using Express.js to provide some basic routes, data and view rendering.
As development progressed we expanded this application to mimic more and more features of the back-end (in a very stripped-down way), so that we could fully test things like API calls and routes, without having to commit unfinished code just to test integration with the back-end.
Of course this “psuedo-application” could have been written in any language, but using Node and Express allowed for minimal code duplication. For example, our Node.js app is also largely isomorphic, sharing the same controller and route files across the stack, as well as many of the same helper files and utility libraries. Therefore the overall size of the application is much smaller than it would be were we using a different server-side language, and much easier to maintain.
Additionally, using Node as a server allowed us to use to Gulp and all the great build tools available to it, resulting in a fully integrated development environment with cross-stack watch, lint and build tasks.
This “psuedo-application” could have been written in any language, but using Node and Express allowed for minimal code duplication
The Final Architecture
Separation, Not Isolation
We were very lucky to have an unusually quiet month at the start of the year before launching our public beta which enabled us to focus almost exclusively on the rebuild. Had things been busier, we may never have got past the initial early experiments.
With that said, the overall time saved in our development processes as a result has been considerable, and 10 months later, has more than vindicated our decision to invest time in what started off as a somewhat experimental and potentially risky endeavour.
There were some initial reservations (and a fair amount of world-domination related jokes) from the back-end team around relinquishing control of certain features entirely to the front end, but in hindsight these decisions now feel like obvious and logical choices which have not only helped to free up resources, but have also furthered each team’s understanding and appreciation of the other’s technology and methods.
Tackling platform architecture as a team effort has formed the basis for a much healthier knowledge sharing process, with developers now able to frequently dip into the each other’s teams to help out when needed. We have found that a decoupling and separation of concerns need not mean an isolation of teams and responsibilities, but leads instead to increased collaboration, understanding, and ultimately innovation.