The world of mobile app development is fast-paced, demanding rapid iterations and flawless user experiences. But what happens when your mobile app testing process cannot keep up? For Wahed, a fintech specializing in Islamic finance with a global user base reliant on their React Native mobile app, slow and cumbersome testing was a significant bottleneck. This case study explores how Maestro, an open-source mobile test automation framework, transformed their approach, leading to faster mobile app deployment and superior quality.
Impact Highlights: A Snapshot of Wahed's Transformation with Maestro
- Test Writing Time Slashed: From 3-4 hours per test with Appium to just 10-15 minutes with Maestro.
- Release Cadence Accelerated: Weekly releases are now the norm, a significant improvement from bi-weekly cycles.
- Test Case Coverage Quadrupled: Expanded from 50 manual smoke tests to approximately 250 automated tests, covering major app journeys and regression.
- Reduced Bug Catching: Bugs are caught earlier, often before releases, thanks to nightly test runs.
- Enhanced Team Collaboration: Developers now actively write tests, a shift from previous reluctance due to complex tooling.
The Problem: The Limitations of Wahed's Previous Mobile Testing Workflow
Before adopting Maestro, Wahed relied on a third-party testing company for their mobile app quality assurance. This approach, while providing regular testing, was plagued by significant inefficiencies:
- Slow Feedback Loop: Getting feedback on issues was a multi-layered process, making it difficult to quickly address problems or change tests.
- Time-Consuming Releases: The entire testing and release cycle was protracted, contributing to a bi-weekly release schedule.
- Limited Control and Visibility: Wahed had to go through external teams to understand testing progress and issues.
Recognizing these limitations, Wahed explored mobile test automation, initially turning to Appium, a popular mobile testing framework.
Appium vs. Maestro: Why Wahed Switched for Better Mobile Test Automation
Wahed's initial foray into automated mobile app testing with Appium presented its own set of challenges. As David White, Engineering Manager at Wahed, recounts, "It was hard going, not very friendly. You had to know Java and JavaScript just to understand some of it, so it was quite a challenge." This steep learning curve made it difficult for the team to adopt and utilize effectively.
The search for a more accessible solution led them to Maestro.
Furqan Daud, QA Automation Engineer at Wahed, initially had reservations, thinking "what could beat Appium?". However, the immediate impact of Maestro was undeniable:
Feature/Metric |
Appium |
Maestro |
Ease of Use |
Required knowledge of Java/JavaScript; not user-friendly |
Simple, YAML-based scripts; intuitive commands like tapOn , swipe . |
Test Creation Time |
Approximately 3-4 hours per test |
As little as 10-15 minutes per test |
Team Adoption |
Difficult to convince front-end engineers to write tests |
Developers actively write tests due to its simplicity |
Flexibility |
More complex for custom API calls |
Easy to integrate custom API calls directly from scripts |
“We found it super simple, a to-the-point YAML-based script. What used to take us three or four hours to write with Appium, we could now do in hardly 10 to 15 minutes. That was the main point, the time cut off, and we impressed our managers right away!"
How did Maestro improve test writing speed?
Maestro's appeal lay in its intuitive design, which closely mirrored what David White had envisioned for an Appium abstraction. The ability to use simple, human-readable commands like tapOn
and swipe
was a game-changer. Beyond its ease of use, Maestro's efficiency in test creation, reducing hours to minutes, was a compelling factor that quickly won over the team and management. The timing of Maestro's emergence also played a crucial role, aligning perfectly with Wahed's growing need for a more streamlined and developer-friendly React Native testing framework.
Post-Implementation Transformation: A Culture of Quality and Accelerated Releases
The implementation of Maestro brought about a profound transformation in Wahed's mobile app testing strategy and overall development workflow.
Initially, Wahed focused on automating their core onboarding flows, which are critical for a financial app and involve detailed KYC (Know Your Customer) processes. By translating existing manual test cases into Maestro's YAML scripts, they quickly built a robust suite of tests. "Each and every step was being automated from the manual test case."
A significant achievement was the ability to integrate custom API calls within Maestro tests, allowing for actions like depositing test funds directly, which was previously a more cumbersome process.
The impact on their release cycle has been substantial. Wahed now operates with a weekly release cadence, a notable improvement from their previous bi-weekly schedule. While a full test suite still takes 3-4 hours to run, they leverage nightly runs to ensure a high level of confidence in each build.
"Something is always caught earlier, and we catch bugs and resolve them. That is where Maestro has truly helped us".
This proactive bug detection has significantly improved the quality of their releases.
"It Sold Itself"
“Maestro was a no-brainer. This was so much fun that we simply carried on. While we had to work to convince everyone, given our prior investment in Appium, Maestro truly came along at just the right time. We were really lucky.”
- David White, Engineering Manager at Wahed
"When they realized Maestro was YAML-based and they could write a test in just 15 to 20 minutes, that's where the difference truly became apparent. Within a year, we saw developers taking a keen interest in testing. Maestro's simplicity had a significant impact, making everyone eager to get involved."
- Furqan Daud, QA Automation Engineer at Wahed
Conclusion: The Path Forward for Automated Testing
With approximately 250 automated tests covering all major app journeys and regression, Wahed has achieved an impressive 80-85% automation coverage. The team is now looking to explore edge cases and further refine their testing strategy, focusing on the usefulness of each new test. They are also beginning to explore Maestro's capabilities for web testing, extending the benefits they have seen on mobile to other internal tools.
Maestro for Web
Exploring Maestro for Web Testing
The vision of using AI to generate Maestro tests from design flows is also on their horizon, showcasing their commitment to continuous innovation in testing.
Ready to transform your mobile and web app testing? Maestro is an open-source framework for mobile and web test automation that allows you to run tests directly on your local machine or leverage the power of our cloud platform.
Experience the difference firsthand by starting your free trial of Maestro Cloud today!