Content

• Continuous Functional Test Automation Suites (Q2 2020)
• Test Automation Becomes Smarter
• Eggplant on the Throne but…
• The Leaders in detail
• Summary and Outlook

Continuous Functional Test Automation Suites

Recently the new Forrester Wave report was published in our well-known area of functional test automation suites. We always watch those reports from Gartner and Forrester, to know what is going on in the market and to consult our customers as best as possible. This report ist special, as with Eggplant we have someone new on the throne. Time to have a closer look at this new ruler in „Westeros“ and its contenders.

In the report, you’ll find the 15 most significant continuous functional test automation (CFTA) providers which were evaluated via 26 criteria. I’d like to focus on the 4 leaders only and some surprising winners in one of the criteria.

But first, let’s have a look where Test Automation will move soon and if the COVID pandemic had any influence on that.

Test Automation Becomes Smarter

We continue to work in agile teams and due to the pandemic more remote. We have to become more adaptive and deliver software more quickly and continuously.

For a long time in the area of test automation, a very robust regression set was key. We had to deliver test cases that are error-prone and at the same time easy to maintain.

Those days are gone, at least in the area of web applications. In packages application testing, those robust regression sets are still a must and will stay that way for a longer time.

…we need an automation suite which enables fast and easy automation

Nowadays we have to automate fast. That means at the end of a user story a new automated test is ready to run or an existing one was updated. Therefore we need an automation suite which enables fast and easy automation. That means we need also fancy stuff like low-code, AI, and ML-enabled automation for GUI and API-Testing. And of course, we don’t want to rewrite those tests for our RPA suite.

So if you are looking for a new CFTA, look for a tool-provider which delivers:

• Make functional test automation smarter and more autonomous
• Provide multiple testing capabilities plugged into functional test automation
• Provide an engaging, uniform, and straightforward test experience

Eggplant on the Throne but…

The time for former leaders like Microfocus (Mercury Interactive > HP Software) and IBM (Rational) is long gone. Tricentis was very dominant in the last years but now we have someone new on the throne – Eggplant.

So let’s have a closer look at those leaders in the top right corner of the newest Forrester Wave.

Forrester Wave: Continuous Functional Test Automation Suites Q2’2020

The Leaders

The Forrester Wave looks at 16 main attributes inside 3 categories. We go through each category and look at which of the leaders is especially good or bad on a specific attribute.

CTFA: Current Offering

The max points you can get is always 5.0 per criteria. In average those are the winners in Current Offering:

1. Tricentis (4.52)
2. Eggplant (4.47)
3. Broadcom (3.85)
4. ACCELQ (3.67)
5. Parasoft (3.50)

Some highlights:

• Tricentis is the leader in packaged application testing, Parasoft is kinda weak in this area
• In Testing of BP and RP automation Eggplant and Tricentis got max points, while the other 2 only got 1 point.
• The only area where Eggplant is better rated than Tricentis is in Operating Environments because it is more flexible. This should change soon, as Tricentis uses their qTest platform knowledge to close this gap

CTFA: Strategy

In average those are the winners in Strategy:

1. Eggplant (4.40)
2. Saucelabs (4.40)
3. Parasoft (4.00)
4. Micro Focus (4.00)
5. mabl (4.00)
6. ACCELQ (3.80)
7. Tricentis (3.60)

Some highlights:

• So in this category, we see why Eggplant has surpassed Tricentis.
• Eggplant has the highest rating in Product Vision, Execution Roadmap, Innovation Roadmap while Tricentis only has 3.0 points
• Parasoft and Tricentis are winning in Partner Ecosystem, something which should be closed fast by Eggplant if they want to wish to succeed in longterm
• A side note here is, that Saucelabs, mabl and Micro Focus got a really strong Vision & Strategy, so they are eager to get (back) into the throne room

CTFA: Market Presence

This category is special because it has no impact on where you land on the wave. It only determines how big your circle (=market presence) is.

In average those are the winners in Market Presence:

1. IBM (5.00)
2. Micro Focus (4.50)
3. Broadcom (4.50)
4. Tricentis (4.00)
5. Smartbear (4.00)
6. Parasoft (3.50)
7. Sauce Labs (3.50)
8. Ranorex (3.50)
9. Eggplant (3.00)
10. ACCELQ (1.50)

Summary and Outlook

I think Eggplant did hit Tricentis from behind. Tricentis was focusing on its RPA and SAP-partnering and was not ready anymore to get a clear vision and strategy. Eggplant, on the other hand, has a really innovative and easy AI-Screen-recording technology and did shape a great vision which was not only noticed by Forrester.
The AI-driven test automation Tricentis now tries to close with its launch of NEO.
What is even more surprising is that also new starters like ACCELQ are suddenly in the throne room of the leaders. As they have a fresh product with a clear vision behind it, time will tell if customers want to start a journey with them.

So Tricentis is overall still the best solution but the gap is a lot smaller now…

But one thing is clear. Everyone wants to sit on the iron throne and I’m sure in a year from now, we will have a quite different picture than today.

Automate Now!

If you would like to know more about the topic or do you need specific support, contact us today!

• Test Automation
• Robotic Process Automation
• Tosca Consulting