Why is decoupled Drupal so hard?
Building decoupled applications is… hard.
That question gnawed at me. It kept me awake, staring at my ceiling. I’ve spent years working on Drupal to support decoupled use cases, but my own early experiences and the stories I hear at conferences and from colleagues all indicate that building a decoupled application involves many trade-offs and complications. How can I reconcile those things? Have I been wrong about decoupled Drupal? Am I chasing a mirage?
I’ve learned that it’s all too easy for decoupled projects to quickly became bloated with opaque state management, tedious request management, unweildy prop drilling, confused access logic, and more. Validated through DrupalCon presentations, Decoupled Dev Days talks, Slack conversations, and chatter around the now proverbial watercooler, my experience isn’t unique. Countless decoupled projects are mired in technical debt. Why?
The problem: mishandled mechanics.
What are mechanics? A project’s mechanics are its access controls, its menu hierarchy and depth, its authentication mechanism, its state management, its routing, etc.
I don’t hear it quite as often as I did a few years ago, but it is often said that “decoupling is the separation of content and presentation”. You may have heard a similar sentiment stated as: “just give me the data and let me do the rest.” Usually, “the rest” is a stand-in for presentation information that Drupal shouldn’t care about. Rarely does “the rest” encompass the full scope of work being hand-waved away.
These refrains reinforce the notion that, to build the platonic ideal of a decoupled application, developers must architect their projects so that the back end has no control over a application’s presentation (layout, color, font size, button placement, etc.) and the front end has no content (pages, menu titles, button text, etc.).
That’s true. But it’s not the whole truth. That pithy refrain: “decoupling is the separation of content and presentation” says nothing about mechanics. All projects have content, presentation, and mechanics.
I’ll wager that 8 times out of 10, a decoupled project gone wrong has mishandled mechanics. It will have a front end bloated by custom code and dependencies all working around the fact that mechanics don’t belong there. Mechanics should be driven by the back end.
It’s not the front end developer’s fault.
The hard path is the easy path.
The hard path is the easy path?
Yeah. Good tools make hard work easy and bad tools make hard work harder. I once was trying to loosen a bolt. It was too tight to take off with my fingers. Looking for some leverage, I looked in my toolbag and found pair of pliers. I ended up with a rounded bolt that I had to cut off. What I needed was a wrench… but my wrench was in the garage. The hard path was the easy path.
Drupal makes it very difficult to architect a decoupled project driven by the
back end. To do so, you need to rebuild or heavily customize the back-end API.
That’s because Drupal’s JSON:API handles only two mechanics: access and
pagination. That is, it omits inaccessible content and provides
prev collection links.
If you want Drupal to control routing in a decoupled context, it can’t. Authentication state? Nope. Contextual links? Sorry! Forms? stares
This functional vacuum begs to be filled. It’s a natural next step to refill it
with front end code. After all, the front end is already replacing the back end
presentation layer. It even seems like that’s the way you’re supposed to do
react-router handles routing and it’s very popular.
ocean of libraries claiming to solve your problems with just one more import.
Some front end developers don’t use libraries. They have a toolbag. One toolbag is named Ember, another Gatsby—Next, Nuxt, etc. And Drupal itself lures developers down the hard path with bolt-rounding “features”.
Perhaps this will be a familiar conundrum: A front end developer is looking at a wireframe with a Log in button. The developer looks but doesn’t find a wireframe with a Log out button but—surely—that’s implied… “Wait!” The developer’s mind pauses. “How will the front end ‘know’ which button state to display?”
Developer Googles: “how to get authentication state Drupal 9”
Enter stage left: the first spaghetti noodle.
What follows might lead to a custom
fetch request to the
Apparently, that response contains JSON and, in it, a link under
/meta/links/me. If that link is present then the user is authenticated; if it
is not then the user is anonymous.1 Great! Right?
Unfortunately, now the front end route must send and wait for an extra HTTP request. And the front end must poll that same endpoint in case the user logged in or out in another browser tab (you forgot about that, didn’t ya?). Moreover, since that fetch isn’t synchronized with the primary request, the front end must either hide the Log in link until it receives the second response or deal with the chance that button flashes Log in before updating to Log out (yuck!).
Perhaps the developer should look to NPM for a helpful library?
Developer Googles: “how to handle asynchronous requests React”
Enter stage right: the second spaghetti noodle.
These little conundrums keep compounding and the plate of code spaghetti keeps growing.
Yet, the back end always knows whether every request is authenticated or not. Why doesn’t Drupal put this information on every response so that front end developers don’t need the extra request?
Mechanics belong on the back end.
Here’s another short story: A front end developer built a decoupled web app
with a filtered content listing. In a server-rendered Drupal application, the
page would have been a View. However, this application is using Drupal’s
JSON:API. Apparently, “Collections are JSON:APIs API-First replacement for …
Views."2 and the way the content listing was built uses a custom-built
filter query parameter.3
Developer Googles: “how to invalidate cached JS file in the browser”
Enter from above: a spaghetti-code meatball.
Why doesn’t Drupal provide a static URL for the content listing whose filters are managed on the server so the front end isn’t entangled by this business logic in the first place?
Mechanics belong on the back end.
What’s the alternative? Progressive decoupling? For now, yes. Forever? I don’t think so.
When the day that you must add a moderately complex interactive flair to a webpage eventually arrives, modern JS rendering frameworks can do that with ease. PHP cannot.
If decoupling is imperative and decoupled applications are hard? How do we make decoupled Drupal easy? And if not entirely easy from start to finish, how can we at least create a shallow ramp that invites developers unfamiliar with the decoupled stack to produce valuable work at each step of their ascent?
What could be changed?
Here are some examples:
- Instead of placing a magical link on the
/jsonapiendpoint, it could provide an
authenticatelink on every anonymous response and a
logoutlink on every authenticated one. This would eliminate the required, secondary and asynchronous request.
- Menu links could be retrievable over JSON:API and conditionally attached to every response in a way similar to how a menu block is conditionally placed via Drupal’s block layout screen.
- Canonical entity URLs could return JSON:API responses so that front ends do not have to map URL aliases to JSON:API URL equivalents.
- Drupal could expose Views (or a simpler Views alternative) over JSON:API so that collection URLs don’t need to be hardcoded or compiled on the front end.6
These ideas need to be refined and implemented, of course, but I believe they would get Drupal to the point where 80% of its use cases are handled out-of-the-box, even in a fully decoupled architecture. I also believe that these features would make it easier for existing non-Drupal developers to build a decoupled application with Drupal than it would be to use any other headless CMS.
The intent of each proposal is to reduce the complexity of front end applications by moving mechanical complexity to the back end. Taken together, they permit front end applications to consume individual HTTP responses that each contain the information needed to render a single page/route/screen in its entirety. This means that front ends, and front end developers, will be able focus on presentation and interaction without having to worry about the additional complexity that comes with a decoupled application today.
Practically, the lifecycle a conventional React or Vue application might look something like this:
- The JS application is loaded and mounted to a root DOM element.
- The JS application inspects the browser URL and requests that same URL
from the back end with an
Acceptheader containing the JSON:API media type.
- Drupal will take that request and return the appropriate JSON:API response.
- That response data can be directly passed as a prop into the top-level component of the JS application.
- Elements of that response data will be passed as props to subcomponents (or provided via React Hooks or Vue’s Composition API).
- Subcomponents like the menu will be able read this response data to get the
menu tree. Since the menu tree was attached to the primary data, the menu
tree data can include an
activeboolean to indicate the active menu item7.
- Another subcomponent rendering authentication buttons can also observe the
response data and conditionally render a Log in/out button. Since every
response will include this data, there will be no need to poll the
/jsonapiendpoint or handle a secondary request.
- Finally, when a user follows a link, all that needs to be done on the front
end is to push the
hrefof the link into the browser’s History API and to request a new primary JSON:API document. Since Drupal can respond to aliased URLs and return JSON:API responses, a client side router to handle the navigation isn’t required. Instead of using URL path patterns to choose components, the JS application can observe the
typeof the response data and react8 to it by rendering a different component, if necessary.
- If the user bookmarks or shares the new URL, nothing needs to be done, even
without a client-side router. A browser request for that URL will return the
index.htmlwhich loads and mounts the JS application and the lifecycle returns to step #2.
As you can see, the role of the front end in this example is only to parse and beautifully render JSON-encoded data–not to reimplement application mechanics that were once exclusively back-end concerns.
This reduced complexity would not only be a welcome simplification for new developers, but also developers maintaining multiple front ends for different devices or use cases. Three simple clients are easier to maintain than three complicated ones, even if they share common functionality.
I think we need a new refrain: Decoupling is the separation of content and mechanics, from presentation. Drupal already handles the content, now it needs to handle the mechanics.
My friends Peter Weber and Ben Mullins were incredibly helpful with this post. Thank you!
Mea culpa ↩︎
Mea culpa ↩︎
Mea máxima culpa ↩︎
If you already know PHP and Drupal and you aren’t interested in “flair”, you might still be skeptical. A few weeks ago, I wrote about Using Drupal For Digital Experiences. It lays out a vision for the future of Drupal and I think it’s pretty exciting. I bet you you might like it too. Decoupling Drupal is a necessary step, though. ↩︎
This item represents the largest potential bucket of work. The others would not be very difficult to implement. ↩︎
Active menu items are another example of an application mechanic that is not easily handled on the front end. ↩︎
Pun intended. ↩︎