Testing mobile applications, especially in highly regulated industries like healthcare, can be a daunting, time-consuming, and repetitive task. But what if there was a way to automate much of that process, freeing up developers to focus on building innovative features instead of manually ticking off test cases? That’s exactly the transformation Doccla achieved using Maestro, a simple, powerful E2E testing framework for Mobile and Web apps.
In this post, we dive into a conversation between Jake, co-founder of Mobile.dev (the creators of Maestro), and Dan, Head of Testing at Doccla. They discuss Doccla’s journey from manual testing struggles to a streamlined, automated workflow, and the significant impact it’s had on their development cycle.
Watch the full conversation here:
The Challenge: Regulatory Compliance and the Burden of Manual Testing
Doccla provides a vital service: remote patient monitoring through a mobile app connected to Bluetooth medical devices (blood pressure monitors, thermometers, etc.). This allows patients to stay at home while still receiving the necessary medical oversight. However, as Dan explains, this comes with stringent regulatory requirements:
…basically every time you ship the app, you have to test everything. All of your requirements have to be tested and you need to keep that paper, you keep that record.
Before Maestro, this meant a grueling manual testing process:
…every time you shipped, someone used to sit there with, with a massive sheet of test rail and do 120, 130 tests. It was half a day, a day of sitting there and taking things off…
This was not only incredibly time-consuming but also demotivating for the development team:
…all that time spent doing tests… was time not spent building features and it made you not want to release because, like, it was hard.
This is where the team decided to look at test automation to greatly reduce the manual testing overhead.
The Solution: Finding Maestro and Embracing Automation
Doccla needed a testing solution that met several key criteria:
- Open Source: To avoid vendor lock-in and ensure long-term viability.
- Accessible to Developers: Easy to learn and use, without requiring deep expertise in complex testing frameworks.
- Device-Level Control: The ability to interact with the device itself (e.g., Chrome, Google Play), not just the app.
- User-Perspective Testing: The team wanted a framework that tested the app like a real user to provide full confidence.
Maestro fit the bill perfectly. Its ability to drive the device, not just test the app, was a crucial factor:
…Maestro, being able to test the device rather than test an app. That was really, really powerful…
The Rollout: From Champion to Contagion
Dan championed the adoption of Maestro, initially building out test coverage himself. The impact became clear as automation adoption and coverage increased:
Initially when we were kind of 5%, 10%, people were like, well that’s nice. Once you got to about the 20% mark and that like that day of testing, people felt that kind of like I’ve got an hour or two back because I’ve got, and all of a sudden [adoption] shoots up.
The ease of use, with its YAML-based configuration, was key to wider adoption:
People are, people are invested. Like Maestro makes it easy. It’s YAML. Like people don’t want to learn, like didn’t want to learn, like the intricacies of selenium and that kind of thing. Like the fact that they could just read a. And it’s pretty much like self explanatory about how this is going to work.
Soon, mobile developers were writing Maestro tests as part of their feature development, integrating testing directly into the development workflow.
like within, within a month we were like great. Other people are writing, they’ve taken up to 30%… Instead of writing tests, I’m talking to other people about how to, how to run it locally, just these little bits, mostly sending them links to the docs and, and now if I look at how the team works now, I’m almost completely uninvolved.
The Impact: Faster Releases, Increased Confidence, and a Focus on Innovation
The results of implementing Maestro were transformative:
Bi-weekly Releases
Doccla went from struggling to release every few months to consistently releasing every two weeks, even over the Christmas period.
I talked about we wouldn’t dream of releasing every two weeks. They have released every two weeks for months, including over Christmas, because they’ve just got that level of confidence.
Shifted Focus
Developers could focus on building features instead of manual testing.
…the team is spending their time not focusing on the testing. They’re focusing on building cool stuff.
Improved Engineering Culture
The conversation shifted to engineering best practices, risk assessment, and quality.
…there’s a mind shift. Everybody’s talking about engineering, good engineering and risk and quality and all the conversations are the right kind of conversations.
Reduced Risk Decision Fatigue
Automation reduced the need to make constant risk-based decisions about what to test manually.
The cool thing about having all the automation in place is that you don’t. There’s a bunch of that decision you don’t make. There’s a bunch of the decisions you go like oh, which test do I run? All of them.
Cost Savings
Hosted execution on Maestro’s cloud infrastucture significantly reduced costs by freeing up time for developers.
The cost of Maestro Cloud as it was when we made the decision, like it was practically a no brainer because you kind of go like, well if you’re saving a developer day a week, like have you seen how much mobile devs get paid?
The Development Process Integration
Maestro tests are now an integral part of Doccla’s development process:
- Tests Alongside Code: Tests live in the main repository, alongside the code they test.
- Local Execution: Developers can easily run tests locally during development.
- Continuous Automated Testing: The main branch is tested constantly using GitHub Actions.
- Traceability: Tests are linked to requirements, ensuring full traceability for regulatory compliance.
Looking Ahead
Doccla plans to expand its use of Maestro to other apps, including their carer app, leveraging the reusability of tests for their React Native codebase.
Key Takeaways
Doccla’s story demonstrates the power of automation in mobile app testing. By choosing the right tool, and fostering a culture of automation, they were able to:
- Dramatically reduce manual testing time.
- Increase release frequency and confidence.
- Free up developers to focus on innovation.
- Improve overall engineering practices.
- Improve the end-user’s experience.
This case study provides a compelling example for any team struggling with the burden of manual testing, especially in regulated industries. Maestro’s ease of use and powerful features can be a game-changer, transforming testing from a bottleneck into an enabler of rapid, high-quality development.