Why you should have at least one middleware with your E-Commerce
Don’t stuff your e-commerce with all the logic, split it up for stability, performance, security and maintainability.
This is quite an old subject of mine, one that I have trouble explaining to my clients. Often we are migrating from a Magento1 website, they are used to having all the development done in their monolith e-commerce solution. When they migrate to a new solution they expect to keep the same logic.
Times changes and we got more experience in building e-commerce websites. E-commerce websites are also more important for business then they used to be; their share in most companies income increases day by day. The current context with pandemic and restrictions have accelerated this even more.
The tendency today is for headless websites, with microservice architecture. But this is not possible for everyone; it’s still quite new and expensive; there are also very few ready to use software to make your e-commerce in such a manner.
Most websites even today end up therefore to rely on monolithic apps such as Magento, or Sylius. These applications are more flexible then ERP’s on which most, if not all the business of the company depends. The flexible e-commerce solutions, therefore, grows even more as more logic are added to them.
It is my opinion and experience that some of these logics can be externalised into an additional software that would be called a middleware. This is made easily possible by the new e-commerce software all supporting rest api’s natively.
First, What is a middleware?
Middleware is a very loose term, but basically, it’s a piece of software between 2 other pieces of software. Software A can’t communicate with software B; a middleware in between will allow them to communicate.
In someways a middleware is a translator, you speak English only, you need to talk to someone that speaks French only. There are 3 solutions:
- You learn French.
- The other person learns English.
- You hire a translator.
The middleware is the third solution. Sometimes it’s good to learn a new language but can you learn 10 new languages? Do you have the time to learn 10 languages?. T
Of course, a translator is not as good as having 2 people speaking directly with a common language because of communication issues. Even so, if you do learn a new language but are not fluent enough there might be miss understandings as well. The translator is simply the fastest and most efficient solution.
What can we do with a middleware?
Before seeing exactly how a middleware can help us let’s see an example of what a middleware can do for your e-commerce.
Your e-commerce usually needs to fetch products or at the very least the product stocks from an ERP. It will also need to export the orders to an ERP and synchronise the status of the order with the ERP.
The EPR already has api’s or file exports that allow external applications to communicate with it. The e-commerce solution also has such api’s.
Now the middleware can sit in between both these applications and call the ERP’s and the e-commerce solutions api’s. It can also do transformations, do additional checks, send report mails; it all depends on what you need.
So why should we use a middleware instead of doing custom code in the e-commerce solution?
Advantages of a Middleware
Stability
Not having the code for connecting the ERP to the e-commerce in the e-commerce, means less code in the e-commerce solution. Which means fewer chances for :
- the need to redeploy a new version of the e-commerce solution, which is always risky
- new code added to affect your customers and therefore your sales
Also, If the middleware fails your customers and your sales won’t be affected. You will continue to sell; you might just end up having some longer processing times.
Separating the code from the e-commerce solution doesn't make everything more stable, but it makes the most important part; the visible part of the website more stable. If your e-commerce is down for even 1H; during Christmas that is a nightmare; if the middleware is down, that is manageable.
Performance
Also, some e-commerce solutions will slow down as more code is added to them; even if that particular code is not executed in a particular page. In the case of Magento for example, this is mostly due to divers cache containing data from all the modules. So having less code in Magento will also increase performances.
Flexibility
A Middleware can also be very easily duplicated; imagine you are migrating to a new ERP, and a particular low traffic low sales country migrates before the others in order to test the new ERP.
You can easily deploy a new version of the Middleware while still keeping the old one running. Both could run at the same time; the old version continues to sync countries connected to the old ERP, and the new one the other countries. Once all the countries are switched to the new ERP you just throw away the old middleware.
With today's flexible hosting solutions having multiple instances is a breeze. There are multiple scenarios where this can be used. And of course, all these changes can be made with 0 risks for the customers visiting the website.
Maintainability / Security
Depending on your business model you e-commerce solution might not have a dedicated development team working on it full time. You are happy with your solution and do not need to make changes often. This does not mean you must not keep the solution up to date. The e-commerce application is exposed to the public and therefore to attacks. You must therefore update your solution constantly.
At each update even minor the development team will need to adapt code to make it compatible with the new version(particularly with Magento), and you will need to test the new release. The less code you have the faster you will be able to make these security updates and the less time your website will run in a vulnerable state.
The middleware is less concerned by security attacks, ideally, the middleware is not exposed to the public at all; and it’s also very improbable the middleware is storing any data. This means that security updates for the middleware are less critical and can be done less frequently.
Cons to a Middleware
There are 3reasons I can think of for not having a middleware:
- The development team looking after the project will have to master 2 “technologies”. The e-commerce and the middleware. This is half true, a middleware as described is usually very simple; the complex part is the business part. It’s my belief that any good developer should be able to jump into such a tool.
If you are using sylius for example, this is actually not even true because both the e-commerce solution & the middleware would be based on Symfony. - Additional hosting costs; this is more relevant. If you are using a modern hosting solution you can actually even use lambdas for executing the middleware which makes them cheap. We could install the middleware on the same server as the e-commerce solution but that’s not ideal as you would add common version dependencies between both application which would make maintenance more complex.
- Added complexity to the architecture.
Conclusion
In my opinion, these cons are non-issues, and any additional cost or the additional architectural complexity pays for itself very quickly.
There are very few reasons to develop connectors directly inside your e-commerce. Sometimes you might need to do it for performance if the native api’s doesn’t allow you to do bulk operations.
I have recently worked on a project for a new client, to connect a Magento that is at its end of life to an ERP. The deadlines were very short and we didn’t have access to the source code of the Magento as we were supposed to. We were nevertheless able to create the connector and deploy it. Because we used a middleware not having access to source code of the Magento didn’t affect us. There were custom devs but the api’s allowed us to find the information we needed without us having to do possibly false suppositions on how the data was stored.
The trick is to split things correctly, to remain in budget and not over complexify things for projects that possibly will never have dedicated development teams for their maintenance. So sometimes it’s better to put multiple things in one middleware then split them into multiple middlewares. These choices are very situational.
