The Magic Quadrant (MQ) from Gartner is ubiquitous. Individuals may love and loath them, but many organizations rely on them as a starting point for short listing vendors in an evaluation and procurement cycle. MQ’s have a lot of flaws, which I will discuss below, but they are built on a rigorous research approach informed by thousands of end user client calls, extremely detailed vendor surveys and a deep understanding of the enterprise end of the specific market segment covered. The four-quadrant graphic is extremely quick and easy for people to understand (and to misinterpret, but that is a separate post).

Alongside the Magic Quadrant, Gartner has, until this year, always published a second document known as Critical Capabilities (CC). Think of the CC as the unpublicized sibling of the MQ.

Unlike the MQ, the CC does not lend itself to a simple graphic. It requires some nuance to interpret, and because of this it is not an attractive marketing asset. Hence you see a lot of MQ reprints across vendors, but no CC reprints.

However, the CC fills two very important functions.

  • CCs provide a technical evaluation against a set of use cases where various pieces of functionality from vendors are scored and calibrated with respect to each other.
  • CCs provide a valuable insight into the types of questions that analysts are receiving. The use cases are an aggregation of common questions that analysts receive from clients. Often these use cases seem prosaic, even mundane, to vendors – but they end up in CC’s as they are being asked about by Gartner’s end user client base.

The scores from a CC have, at least until now, directly flowed into an MQ. The weighting can vary a lot, but the scores can have more than enough impact to substantially move a dot in a complex market. More technical or product focused analyst’s value the CC as a useful tool to use during client calls. Or at least they did when I was at Gartner.

Savvier analyst relations teams and product managers value the CC as a document to get a comparative grounding on competitor product capabilities.

So, What Has Changed?

We have recently started to see some MQs, for example the recent Software Supply Chain and Cloud AI Infrastructure MQs (both paywalled), arriving with no accompanying CC, either at publication or planned for release relatively soon after the MQ.

Commercially I can see the reason why. MQs are already a massive lift for vendors and analysts, and the CC adds another layer of analysis. If Gartner’s goal is to produce more MQs, not having a CC will speed up the process.

However, and this is the rub, the lack of a detailed capability analysis and confirmation of how tools work for a set of a common use cases, and associated publication of this research, opens a huge gap in the process. It also, and this is just my opinion, dilutes the value of investing in the process in the first place from a vendor perspective.

I for one hope CC’s do not go gradually go away, and that the research leadership in Gartner finds a way to keep them. The problem is not the content, it’s the packaging.

The Flaws with Magic Quadrants

This list could be very long, and it is well debated in many quarters, but it is worth reiterating three of the key flaws with MQs.

  • They are a snapshot in time, which is fine in most technology markets, but in fast moving markets like we are seeing with AI it is a huge struggle to keep the research relevant and timely. To be clear the analysts covering this space, for example around coding agents, really know the enterprise market. They are, and trust me on this, talking to more organizations who are making purchasing decisions at scale than anyone else.
  • They implicitly reward large vendors – scale matters, and from the perspective of many technology buyers this makes complete sense. You need to know a vendor will still be around (or their technology is valuable enough to be acquired) for the medium to long term.
  • They are really easy to deliberately misinterpret – particularly when it comes to technical capabilities – hence the need for the CC. Technical capabilities are, rightly, only one component of an MQ

For all their flaws, MQ’s remain a valuable piece of research. A piece of research that is hugely popular with Gartner’s end user and vendor clients alike, and there is no sign of that popularity ending anytime soon.

Disclosures: I worked at Gartner as an analyst in the past, and now work at GitHub. I have seen the MQ process on both sides of the fence.