Magento 2 — A dead giant?

First of all, this is my first article on medium. Hourayyy!

Until now I have been blogging on my personal website; https://oliver-decramer.com but without a lot of success.

I will be from now on, blogging here on what I consider “Important”. So some technical stuff or even maybe some fantasy. Time will tell exactly what I will publish.

To get into the subject; I am today, undecided whatever Magento2 is worse then Magento1.

Sacrilege you say?

Well, let’s see, In on one hand we have Magento1:

• stable
• relatively fast
• easy to deploy;

but

• it’s deprecated
• not xsd for xml’s
• no composer
• ….

and then we have Magento2 that is:

• maintained
• has varnish support
• composer for dependencies

but

• is complex to deploy
• has a slow BO
• it’s slow without varnish
• ….

I already wrote an article a few years back on what I thought about Magneto2, and time passed by, new versions were released and; it didn’t improve.

When we installed a Magento1 for our clients we basically had to do 2 mandatory changes:

• Prevent database update/install scripts to run on a page load and create a script to execute these scripts. This allowed us to do deployments with nearly no down time with only a bit of organization on our side.
• Add varnish support;

We of course have some other scripts to debug and so; but nothing critical, deployments were simple. We could build the css as we wanted, and do the same for javascript.

On magento2 on the other hand; we have tried different processes and all of them are either not working or complex. So that’s going to be our first point of concern.

We will then talk about native bugs, and the fix process; as well as the version numbers.

And end it with “community” extensions.

Let’s deploy Magento2

I want to start by saying there is lot’s of different processes for deploying a code. But there is one proper process; and that is by deploying every time all the code; in an automated and reproducible way.

This has some advantages, that we will discuss on a later chapter.

To implement this you can use shell scripts; ansible whatever you like. There is then intermediate steps that are application dependent but those are the details.

Building it on the servers

For our first projects, after tinkering quite a bit with the generate static commands we decided to build everything on the server beside dependencies. So these are the steps we did

Locally or using CI

• Composer install

On the server(s)

• Copy sources to the servers in /var/www/releases/<my-release>
• Compile the di (setup:di:compile)
• Deploy the static content (js & css)(setup:static-content:deploy)
• Put magento in maintenance
• Change symlinks to have /var/www/current/ => /var/www/releases/<my-release>
• Execute the database updated (setup:upgrade --keep-generated). The keep generated allows us to gain time and have less down time as we have already generated them before switching the symlink
• Disable maintenance.

With this procedure our only downtime is during the setup:upgrade so this at first seems to be a good solution. But it’s not without faults :

• The static deploy command is quite heavy on a websites with lots of locales. On some of our website with 30+ locales this can take over 30 minutes.
• We are running quite heavy commands on all our servers, which ends up adding load to our servers and therefore increases page load times and therefore lowers the service quality.
• The biggest issue is if something is wrong with our code, Magento in some cases will not be able to execute the compilation of the di or the static content. We only see this when we decide to deliver our code. If a CI ran those commands we could see it on the Merge request directly.

So back to the drawing board. We know that a full deployment is the way to go. We need to find a way to make it work with Magento.

Built it locally or in a CI

This seems to be the most obvious solution, and usually works fine for other websites I work on(drupal; symfony). First let me explain what we did then why it didn’t work.

To do this, there is an extra step the developer needs to take locally. The magento setup:static-content:deploy needs for some reason the list of stores, if this list is not present then it will try and connect to the database, but as we are building the package in the CI we don’t have a database. To get around this every time they we add a store, we need to dump the config files:

app:config:dump

Then the CI/CD will

• composer install
• Compile the di(setup:di:compile)
• Deploy the static contents (setup:static-content:deploy)
• Create an archive

Now this archive can be deployed:

- Copy sources to the servers in /var/www/releases/<my-release>

- Put magento in maintenance

- Change symlinks to have /var/www/current/ => /var/www/releases/<my-release>

- Execute setup:upgrade — keep-generated

- Disable maintenance.

We have moved our static generation to the CI, this works as the config.php committed contains the list of all the stores.

This looks to bee good solution, for we can now deploy one package on all environments, and we are sure that if there is a bug on one and that we can’t reproduce on the other, it isn’t a source code issue. If there is an issue with our code preventing it from being built we can also see it before deploying.

It sounds to be the proper solution? Well it doesn't work. There is 2 important issue with having the list of stores in the config.php.

• First of all if the magento has never been installed instead of running the setup:upgrade we will run the install command, this will fail as it will try to load stores from the database that doesn't exist. Logic as the stores will only be created later on with one of our install scripts. But the stores being in the config.php file, magento thinks they already exist
• Second issue is with the setup:upgrade, if you add a new store through a updata script, the store is in the config.php but not in the database which again cause an exception.
• There is also issues if the store ids are different in the config file & in the database. Not having all the stores in the config.php will also cause issues.

These shouldn’t be much on issue as we don’t run installs and create new stores that often do we? Well that’s debatable. Ideally, we would like for anyone that starts to work on the project, to simply run a install script without the need to fetch a database. The install should create most if not all the different cases and allow anyone to work on the project.

We could also bee using a automated CD capable of creating from a new branch in git a test environment with a single click. But with this method we can’t do it simply. We need to download a database during the creation process. We need to keep the database up to date with the proper stores. It make’s it “complicated”

These 2 solutions works without adding any additional code to Magento. We ended up coming to the conclusion that even through I don’t think that having an additional module just to make magento deliverable isn’t a good solution we ended up doing it. On our current procedure we have 2 config files; and a script that automatically updates the second one. The main config.php is the one we use to run the website everywhere. It does not contain the stores.

We automatically generate a second config file with all the stores. The CI uses this file. It’s the best we could find; it’s simple for the developer they don’t need to manually think about editing the file. But it’s additional code we need to maintain.

I had to choose the lesser evil. I would rather not choose if I had a choice. But I have to make it work so.

It’s natively bugged

All applications have bugs, well most do. And something as complex as Magento 2 when first published is bound to have quite a lot of them.

That is expected, the issue is the way bugs are regarded.

• They are ignored as user failure or as not reproducible even through they can be quite critical.
• Fixed in major versions; Forcing you to get new features you don’t need and possibly new bugs instead of just a fix.
• Fixed and broken in different versions. This is the case with the ES 5+ search engine. In 2.3.3 pagination on product search & category pages didn’t work. Only first page displayed products. With 2.3.4 the pagination works but the number of products is always incorrect.

There is no clear view for us integrating magento; to know if a bug we found and that is affecting our e-commerce website will ever be fixed. I don’t think that’s normal. And I am not talking about small bugs, there are bugs affecting payment methods that go unresolved for years. Github issues are affected obscure internal magento ticket numbers which are quite useless.

Just check this graph of open issues

Number of issues that are open at a given time (not number of new issues opened)

You can ignore the few abnormalities in this graph that I did using the github api’s. At some point end of 2017 they did a cleanup, during which many issues were closed without reason. But over 2000 open issues at date.

In average an issue takes 80 days to be closed. But there are issues that are more the 400 days old.

I agree that github issues are not always reflecting well the truth of what is going on. Most of those issues are indeed user issues or duplicates. I did compare this with a few other open source solutions and usually the number of open issues tends to stagnate at a certain point at much lower numbers. Magento is not doing that.

And don’t get me started with the Enterprise support. EE modules first of all are worse then the CE modules;

• Multiwarehouse not having proper elasticsearch support.
• The B2B functionalities not having elasticsearch support again. (Fixed in 2.4)
• Staging, and it’s nearly exponentially growing tables; I might actually do an article just on this. (2n-1 insert per version)
• Page builder and it’s loosing your contributions. Don’t spent to long to edit the page, or don’t do to much without saving you never now when the javascript will crash.
• …

And if you do get in contact with the EE support; best case scenario after proving multiple ways the the bug is real; you are going to get on of those “.patch” files used by pirates in the 1660’s. That’s going to create conflicts at each update and will therefore become part of what you need to maintain. The official fix will be implemented in a very far future and break something else.

I would like to end this with a last word on the version numbers. When you upgrade any application from 2.3.4 to 2.3.5 you do not expect to be making huge changes. After all it’s bugfixes? Well not with magento; because magento does not respect semantic versioning. magento/magento2-base 2.3.4 will for example require magento/module-catalog 103.0.4. So this means you need to check the change logs for each version.

Magento 2.3.5, that’s the commercial version number, it has no technical sense at all. A “Minor” commercial version can be a major technical version. So it’s very developer friendly.

Built without thought

That’s really what I think of when I think of Magento, it’s an overly complex tool; that has lots of features but with none of them inspiring trust when you start to dig in to see how they work.

It’s been now 4 years since the release of Magento2 and each year new features find it’s way into the tool, no refinement no improvements is done to the core, layers are added ontop of layers. And more new bugs are introduced then fixed.

Community modules

I would like to finish this article with one last rant on the community modules.

Why do we pay for this garbage?

I won’t give names, but we just bought a bunch of modules for our customer and issues are pilling up,

• Some of them can’t be installed with composer, they have their own ClassLoader and needs to have files to be added to the lib folder(which we don’t commit). I don’t ask them to have their own package repository, just to give us a proper archive that we can use with composer artifacts. That’s split up’s the “editable” code from the vendor code sufficiently.
• Outdated code, those modules that needs files to be added to the “lib” directory that they load them using their own Autoloader functions. Their code is pretty much the same as the M1 counterpart of the module. And the code is also probably used for woo-commerce and other php cms they support.

I am not talking about bugs, just how things are done and how changing behavior and maintaining those module is complicated.There is no proper centralized repository to have information on updates. At each magento update we manually need to check for the updates.

The situation of the modules make it even harder to work with magento; Wordpress and Drupal have similar issues with modules breaking native features but at least heir installation procedure remains unchanged.

With Magento each module has it’s own way of getting installed and only few of them is installed out of the box with composer.

There are some very good modules, usually open source modules are better then those you have to pay for.

Conclusions

Magento used to be one of the only options to create an online website for a reasonable price. Today there are other open source alternatives such as Sylius; but there is also SAS alternatives such as Shopify.

Whatever solution is chosen it should be done logically. In some cases Magento 2 remains the most obvious choice even through all it’s downfalls, because it’s so feature rich. But on some project it is actually easier to develop features from scratch on Sylius for example, then handle the edge cases magento was never meant to handle.

I think Magento has less a place in today’s world; where e-commere websites don't just need to work; they need to be special while remaining easy to use and fast(magento2 is by no means fast). They also need to be stable and have little downtime.

Most of the clients I work with requires Magento and won’t even consider an alternative. But I think most of them would profit by using Sylius. Most of them don’t use most of Magento’s functionalities; and most of what is used has been customized to great extent and great labor. There are of-course exceptions; In some cases Magento is a perfect fit.

I do hope that Magento will be able to reinvent itself in the future, for today in my opinion the only reason it’s still alive is the good reputation of Magento 1 which causes companies to wish to upgrade to Magento 2.