Modernizing the app delivery foundation at NTT Docomo with UiPath Test Suite

NTT Docomo, a leading mobile phone operator in Japan, provides a variety of telecommunications services and applications. Given that their service and app provisioning spans over 30 years, NTT Docomo has faced legacy issues in their underlying systems. To overcome this challenge, the Service Design Department at NTT Docomo decided to focus on advancing their system modernization through automating their testing environment. By adopting UiPath Test Suite, they have worked towards eliminating individual dependence on testing, as well as reducing the load during large-scale testing.

Transforming legacy systems through agile development and test automation

NTT Docomo not only provides mobile communication services for smartphones, but also a variety of services through apps. The service infrastructure that supports these diverse services and apps is one of the most critical infrastructures for NTT Docomo, along with the communication infrastructure.

The Service Design Department leads the development of Docomo's services and apps, as well as the maintenance and operation of the underlying infrastructure.

"Development is important, but our biggest mission is to maintain operations and ensure that customers don’t experience any inconveniences," says Takanori Nagahara, Application Development Team Leader in the Service Design Department at NTT Docomo. "During our 30-year history, many of our services and app mechanisms have become legacy systems. However, if they are important services for our customers, we can't just simply stop supporting them. My job is to smoothly modernize these legacy services and apps.”

Numerous underlying systems reinforce NTT Docomo's main system—one of these underlying systems is the app delivery infrastructure. Mr. Nagahara has been working on the Legacy System Structural Reform for these systems since 2019, as the legacy system that was built more than 10 years ago has reached its end of life (EOL). He's in charge of the transformation towards creating a foundation that enables smoother operations.

Throughout the Legacy System Structural Reform project, the team aims to solve the challenges inherent in the legacy system by shifting towards a cloud native environment and adopting an agile development approach. For example, the team simplified the system by decluttering specific functions, and improved productivity through containerization. Furthermore, Mr. Nagahara states, "By automating the tests that were previously performed manually, we tackled the fundamental issue of spending significant time and effort on testing.”

Reliance on manual testing created various issues. In legacy systems developed using the waterfall model, there were cases where releases occurred only once or twice a year, even with multiple release opportunities. With such infrequent releases, it proved difficult for individuals to absorb all documented knowledge related to testing the system. Mr. Nagahara explains, "We needed to read through documents from scratch every time or seek out people with expertise. If we had accumulated automatically executable test cases, such efforts wouldn’t be necessary, and anyone could execute tests immediately."

The event where the team most anticipated the effects of test automation was the major migration of the underlying systems, which takes place once every five years. This migration involves a massive amount of testing concentrated on upgrading hardware, operating systems, and other components. Mr. Nagahara states, "During each migration, we encountered numerous issues and unexpected situations, and creating tests placed a significant burden on us.”

Selecting Test Suite for mobile app testing

When the Service Design Department tackled test automation as a core element of the Legacy System Structural Reform, they chose UiPath Test Suite as their solution of choice.

According to Mr. Nagahara, "The main reason for choosing Test Suite was its ability to effectively test mobile apps."

The department conducts many tests that involve connecting mobile device applications with servers to verify their operation. However, the mobile apps are developed by separate departments or external parties, so the Service Design Department often lacks detailed testing expertise about them. Therefore, they needed a tool that could automate the testing of mobile apps and inherit that expertise.

"Among the test tools we tried, only UiPath Test Suite was capable of testing mobile apps. Moreover, many of the tools were open-source software that did not have built-in support for mobile app testing," says Mr. Nagahara. Test Suite provided the ability to perform end-to-end testing of both the app and server simultaneously, as well as tools for the server-side.

While impressed by Test Suite capabilities, the team at NTT Docomo also had expectations for vendor support from UiPath, which differs from open-source software. Mr. Nagahara states, "After we began using Test Suite, UiPath provided robust support. Even when our technicians asked numerous questions at a high frequency, UiPath patiently provided guidance to overcome challenges that often arise during test scenario creation. By receiving assistance, our technicians felt more confident in approaching automation. We also received support in terms of nurturing talent," he explains.

Development testing isn't based on documentation, but rather based on executable test cases. By utilizing the accumulated test assets in a timely manner, the modernization of the system can be efficiently streamlined.

Mr. Takanori[TN(崇1] Nagahara • Application Development, Service Design Department

18 months to faster test execution with Test Suite

The Service Design Department adopted Test Suite in the spring of 2021, and by fall of 2022, they had accumulated about 1.5 years of experience using it. While Mr. Nagahara acknowledges that they are still in the middle of their test automation journey, they have high expectations for the future based on the progress achieved so far.

One of the effects is the significant reduction in test execution time. "For new features, the responsible individuals learn how to use Test Suite, then create test cases, and then execute automated tests. For existing features, we gradually increase the number of automated tests while leveraging the expertise of knowledgeable individuals for regression testing. As we continue to reduce the time required for learning and scenario creation, the cumulative impact of automation will become even more significant," explains Mr. Nagahara. During the initial introduction of Test Suite, it took about two weeks for learning, specification creation, and execution for a single test. However, recently the team has reduced this process to only a few days.

In the scheduled migration planned for 2025, which occurs once every five years, the team expects to see even greater impacts. Mr. Nagahara expresses his enthusiasm, saying, "Regarding the tests created with UiPath Test Suite, there won't be any additional effort required to re-run them during the migration. If we can use automated tests for the migration, we anticipate significant effects."

For testing the delivery platform specifically, 80 scenarios have already been built as the basic set. The team estimates that the number of tests required for the migration will be three times that, taking into account various combinations. "We can already eliminate one-third of the entire tests through automation, reducing the need for manual work," explains Mr. Nagahara.

Test automation also contributes to reducing the time required from everyday service and app development for release. "Previously, it took one day to perform manual test execution, but with Test Suite, it can be completed in one hour. With the transition from a waterfall development process to an agile one, including changes in the development structure, we estimate that the total time from development to test completion can be shortened, and the release time can be reduced to 1/8 of the original," says Mr. Nagahara.

Mr. Nagahara further discusses the benefits of test automation in agile development. He says, "In waterfall development, it’s assumed that there will be no need for rework after a single test, but in agile development, testing is repeated. Even while facing such changes in the development process, Test Suite proves its effectiveness. In comparison to waterfall development, where a single release took eight months, agile development allows for releases every two weeks. However, to maintain quality, it’s necessary to engage in test automation."

Enhancing personnel development with Test Suite

Testers at NTT Docomo have also experienced positive impacts in their role development from using Test Suite. Through the utilization of Test Suite, which can be operated through a GUI, even new employees can quickly create multiple test cases. These opportunities have been helpful for sharing business know-how and personnel development.

"Even if our team members encounter difficulties along the way, they can successfully complete tests thanks to strong support from UiPath. We’ve heard a lot of positive feedback saying that this has given them confidence in testing," says Mr. Nagahara.

Personnel development has also contributed to the enrichment of test scenarios. In the era of manual testing, a single skilled individual oversaw all testing. Now, the transition to automation has allowed for multiple people to tackle test creation. As a result, the team at NTT Docomo has experienced a steady increase in the number of possible scenario creations.

Looking to the future, Mr. Nagahara states, "We anticipate expanding the test automation introduced in the delivery platform to other platforms." Furthermore, he identifies applying Continuous Integration/Continuous Delivery (CI/CD), which is currently implemented only on the server side of the delivery platform, to the mobile app side as another challenge. The team aims to establish a consistent environment of test automation and CI/CD from mobile apps to servers, allowing people to focus on more valuable tasks.

Regarding the selection of test automation tools, Mr. Nagahara explains, "It’s important to clearly identify the challenges you want to solve. In our case, Test Suite served as a solution to the challenge of automating tests for mobile apps. If it can be determined that UiPath can solve that challenge, then the functionality of Test Suite and the strong support of UiPath will provide a clear path to success.”