Since Perfecto is sponsoring and attending the annual Appium 2018 conference today, it’s a great time to take a look at some pros and cons, how it stacks up against the competing tools and what might be around the corner.
Benefits of Appium
What’s great about Appium:
It has a strong active open source community
Appium is by far the leading open-source test framework for cross-platform(mobile) native test automation (iOS, Android)
Appium is consistently backed by a large, very dynamic community, with steady support, commits, etc.
It has strong support for end-to-end testing in multiple programming languages
Appium can cover black box end-to-end test flows including outside-the-app scenarios (e.g. initiating a call, sending a text message)
Challenges with Appium
Some areas where Appium could be better:
Setting up Appium locally can be a challenge: Teams are required to download, install and configure their environment which means having a local Android and iOS device available and connected.
Working with app objects for both iOS and Android isn’t easy, and falls into the top challenges (below) coming from practitioners. Since the Appium framework relies on iOS XCUITest and Espresso (Android’s UI automation framework), users needs to be familiar with the object structure, and know how to use the Appium object spy correctly.
Slow test execution. Tests can be slow due to the remote webdriver dependency, network issues and processing commands.
Test framework stability – Stability issues can occur when executing through CI or at scale and in parallel.
Test automation coverage and keeping up with latest mobile OS – Being able to fully cover gestural inputs, environment conditions, device settings and more, as well as support immediately latest Beta and GA versions of iOS and Android. Community supported solutions frequently move slower than handset vendor/mobile OS innovations.
Top Challenges from users on the Appium discussion board
Comparison of Mobile App Testing Frameworks
Despite Appium’s leadership today, DevOps teams are also adopting Espresso and XCUITest.
Since there is no perfect solution with regards to testing frameworks, the best solution for your needs might be best met by mixing various test frameworks across the DevOps pipeline.
Here’s a comparison of the leading testing frameworks:
Where is Mobile Application Testing Headed?
My testing framework intuition tells me that:
Appium will share more of the testing framework market with Espresso and XCUITest.
Functional testing using Espresso/XCUITest will become part of commit- triggered built-testing.
Full end-to-end testing using Appium will be leveraged during full regression testing.
Appium stability will improve and execution times will shrink – it will get better & faster!
Hybrid test execution will become supported. Appium scripts will be able to trigger embedded Espresso/XCITests.
Appium is great! It’s got:
A strong open source community
Outstanding support for a number of programming languages
The ability to handle end-to-end test flows
Appium is weak:
In test performance and stability
In keeping up with the latest OS features (e.g. gestural)
In setup time
XCUITest and Espresso are also leading the market with strong user bases and helps fill in the gaps where Appium falls short – so keep an eye out for those other tools!!
It will be interesting to hear the upcoming Appium roadmap at the Appium 2018 Conference. Sign up for our live webinar to hear our take on the current and future state of testing frameworks and how Appium might fit into your DevOps toolchain.
Apple has unveiled “ClassKit” at its Field Trip education event. It’s basically a framework for educational apps and works pretty much like HealthKit and SiriKit.
The tool allows developers to create apps that can work with Schoolwork, which enables teachers to assign handouts to students and also track their progress. Schoolwork also lets students report their progress to teachers.
Apple ClassKit: The Framework for Educational Apps
“Schoolwork only receives and displays student progress data for activities a teacher explicitly assigns, and only when students use the Managed Apple ID that was created for them by their school on their device. If your app already uses a student account to enable certain features, ClassKit does not associate that account with the Managed Apple ID that the student uses.” – Apple
When Will ClassKit Be Available?
The ClassKit framework will be available with iOS 11.4 and work in conjunction with Schoolwork. According to Apple, developers can build apps that work with Schoolwork using Xcode 9.4 beta and iOS 11.4 beta.
They also require the access to Apple School Manager and a beta version of Schoolwork for testing.
Some apps have already begun using the tool. Teachers and students can take full advantage of them to collaborate on assignments, report and track progress.
ClassKit and Schoolwork offer desired privacy to students’ privacy. Schoolwork will receive and display student progress data for activities only when a teacher assigns them, and students use the Managed Apple ID.
Just in case an app already uses a student account to enable certain features, the tool won’t associate that account with the Managed Apple ID.
What’s your take on ClassKit? Toss up your thoughts in the comments.
CHICAGO—Educators will have new ways to use iOS in schools thanks to Apple’s latest software update. On a stage at a school in Chicago today, Apple announced ClassKit, a new framework for making educational apps on iOS.
We first learned of ClassKit when Apple users were sifting through a beta of iOS 11.3. It’s a software development framework, in the tradition of prior Apple-developed frameworks like ARKit, HealthKit, and SpriteKit. This framework is focused on taking care of some of the heavy lifting for developers who are making apps for use in schools and other educational contexts.
Schoolwork and Classroom
Apple created a few native school programs with ClassKit and promised that more apps using the API are forthcoming. The new Schoolwork app gives students and teachers a digital space to assign, complete, and review assignments. The free, cloud-based app lets teachers assign work to students and see each student’s progress while they complete the assignment in their own version of the Schoolwork app. Teachers can even grade these assignments directly in the app, making assignment management much easier than if dozens of students were to hand in individual papers.
Apple has announced its latest foray into open source software, this time with SwiftNIO, a new framework for writing network applications using the Swift programming language. Both the framework and its open source nature were announced by Apple software engineer Norman Maurer at the try! Swift Conference.
The framework is now available on Apple’s GitHub page. Here’s what the company has to say about SwiftNIO:
SwiftNIO is fundamentally a low-level tool for building high-performance networking applications in Swift. It particularly targets those use-cases where using a “thread-per-connection” model of concurrency is inefficient or untenable. This is a common limitation when building servers that use a large number of relatively low-utilisation connections, such as HTTP servers.
For those familiar with the Netty framework, Apple also describes SwiftNIO as “like Netty, but written for Swift.”
The documentation for SwiftNIO notes that the framework is not geared towards building web applications, but rather for providing underlying network support for those applications. Currently, SwiftNIO supports macOS 10.12 or higher and Ubuntu 14.04 or higher.
See, there are two main paths to app development. “Native” development, where you use the default tools and languages provided by Apple or Google for their respective platforms. But then there’s… everything else. You can make apps for Android and iOS in C#…
After being announced last year, Google took to the stage at MWC 2018 to announce a beta release of Flutter, its free and open-source framework for building iOS and Android apps with a unified codebase. The SDK lets devs code their apps in the Dart programming language, and packages them along with a rendering engine, as well as the native code needed to run those apps on Android or iOS. It’s designed to offer top-shelf performance, while also making it easy to create interfaces suitable for both platforms, thanks to included UI widgets for each. Flutter plays nice with a range…
Developing for both Android and iOS usually involves working with two codebases, two UI frameworks, and two different design languages. There have been a few efforts over the years to remedy this problem, but they usually result in apps that don’t work well and don’t look particularly native. Google unveiled its ‘Flutter’ framework at Google I/O last year, which allows developers to quickly create native iOS and Android apps.
The first beta version of Flutter is now available, signaling that Google thinks the project is ready for more widespread use.
Google has released its first beta of Flutter, a mobile user interface framework that intends to help developers create interfaces for their apps on iOS as well as Android smartphones, a toolkit that could speed up and simplify cross-platform app development. AppleInsider – Frontpage News
In the last year, there has been a growing trend of iOS development teams adopting XCUITest and additional frameworks built on top of XCTest interface.
Development teams have started to adopt XCUITest to get fast and reliable feedback. There are a few clear drivers to this growing adoption:
Intuitive – Using XCUITest is quite intuitive for developers as it runs from within XCode IDE
Fast – Test execution against iOS devices is faster than any other UI test automation tool due to the framework architecture
Reliable – Due to the architecture of the framework, test execution using XCUItest generates more reliable results and eliminates flakiness
Mature – The API’s and the framework became significantly more mature during the last year
Test maintenance – Since the app is instrumented, the framework works directly in the object level which reduce maintenance efforts that usually happens due to changes in the applications.
A quick iOS instrumented testing frameworks and terminology review
XCTest – Apple’s official framework for writing unit tests for classes and components at any level. These tests, like the app itself, can be written in Swift/Objective C.
XCUITest – a UI testing framework that is built on top of XCTest. Itincludes additional classes (such as UIAccessibility); these tests can be written in Swift or Objective C. The tests are packaged in a test ipa (iOS packaged application) runner that execute the tests on the AUT(application under test) ipa.
KIF(Keep It Functional) – A iOS native app that warps XCTest as well using undocumented iOS APIs. It requires the developer to add KIF framework to the project. It has simple and intuitive syntax.
[tester enterText:@”email@example.com” intoViewWithAccessibilityLabel:@”Login User Name”]; [tester enterText:@”thisismypassword” intoViewWithAccessibilityLabel:@”Login Password”];
Earlgrey – Similar to KIF although developed by Google. EarlGrey has an advanced synchronization mechanism which means you don’t need explicit waits / sleeps. (For example, if tapping a button triggers a network request, EarlGrey will wait for the network request to finish before proceeding with the test). EarlGrey uses matchers extensively (Read selection API this), these give you the flexibility to interact with elements and write assertion logic in a variety of ways with simple APIs.
Cucumberish – Test automation framework for Behavior Driven Development (BDD) that integrates into XCode and uses the iOS interfaces XCTest/XCUITest.
The challenge we find is although the above test frameworks can solve significant challenges that other test automation frameworks cannot, in many cases, teams adopt these frameworks before considering the proper setup and infrastructure. The promise of XCUITest depends on the fact that the execution of the tests will be on a robust, reliable, and scalable lab infrastructure.
Even though the adoption of those automation frameworks grow, many teams still execute their tests on simulators / local device from the developers workstation. Those teams understand that they get significantly more value from executing XCUITests and therefore continue to consider leveraging them even more by executing them as part of the CI processes to provide continuous feedback on real devices and end-user conditions.
XCUITest advanced capabilities
Perfecto recently released advanced support for the above frameworks in order to enable development teams to leverage the advantages mentioned above, while leveraging Perfecto’s cloud based capabilities. In addition, Perfecto extended the XCUITest framework by adding the ability to control and setup the device the same way end users do, by which enables teams to validate that their apps will function as expected in the real world.
To learn more about the Perfecto solution please visit our documentation website or to read more about the differences between XCUITest and Appium: The Rise of Espresso & XCUITest; The Fall of Appium, click here.
Xposed, a framework that allows users to install heavy modifications to Android, usually takes a few months to be updated for every new Android release. However, it took over a year for the creator of Xposed (rovio89) to update the framework for Nougat, due to the major changes Android 7.0 made to the app runtime.
Thankfully, it looks like we won’t be repeating that long wait, as a beta version of Xposed is already available with Oreo support.