Q&A – Cypress.io UK Community Meetup – Model-Based Testing for Cypress

Yesterday, I had the immense pleasure of presenting on the second Cypress.io UK Community Meetup. My talk was called “Introducing model-based testing to your Cypress framework: Why and how?”. It focused on the value of introducing model-based testing design to Cypress frameworks, with a live demo of Test Modeller auto-generating and maintaining test scripts and data from flowchart models.

Want to try model-based testing for Cypress?

There were lots of interesting questions posed during the session, more than we could ever have time to answer live. I’ve therefore answered the questions asked yesterday in this article. Whether you use Cypress, are considering adopting it, or use another automation framework all together, I hope that the below answers help show how model-based testing can be applied to test automation. I hope that they also indicate its value in reducing test maintenance, increasing test creation speed, and generating accurate tests data for tests.

Thank you to Marie and Giridhar for running the Cypress.io UK Community Meetup – check out their Meetup page for more great events in future. A huge thanks also to everyone who joined and asked questions! If you didn’t have a chance to join, you can watch the session recording below:

Q&A

How does the scanner choose which selectors to use? Especially if the elements have random ids/classes/etc that can’t be used, will it avoid them?

The scanned elements can be edited and prioritised during the scan process. This means you have full control over which identifiers are used in the resulting automation.

How ‘clean’ is the output Cypress code? Often auto generated code is far more complicated than it needs to be.

The code Test Modeller produces well architecture code which follows the page-object model pattern. Of course, a great engineer can probably write better code than us; however, the code produced by Test modeller is not messy or overly complicated. The exported Cypress code is also fully configurable using code templates. code that matches best practices and architecture. This means that you can adjust the code generated to meet any architecture or coding standards within your organisation.

Does Test Modeller also scan custom data attributes? For example, if you have data-testid or data-cy attribute?

Yes, this is particularly important for applications like Dynamics 365 where only specific identifier types (e.g. aria-label and data-id) are static between each execution. 

Can we use Test Modeller with Visual Testing tools like Applitools and Percy?

Yes, typically you would build this integration into the framework you are using. Test Modeller then produces the automation code to be executed within this framework.

This demo shows how Test Modeller embeds Applitools’  visual assertions in auto-generated test scripts:

Won’t this introduce a lot of E2E UI tests? Some of the test cases can be moved to unit level, so it can execute faster.

Models are not just for UI automation; they can also be used for API testing and data testing. UI tests are generally slow, and a distribution of different types of testing is best. Model-based test design is flexible and can be used to generate a wide-range of test types of tests.

How does the scanner handle complex pages that have a lot of react or JS on (for example)?

The automation framework needs sufficient code to implicitly wait for elements. If that code exists in the framework, Test modeller will reuse it.

The UI scanner supports React, AngularJS, Iframes and Shadow Dom elements.  This demo shows Test Modeller scanning Shadow Dom elements to generate page objects:

How would one use this MBT to test legacy software such as Peoplesoft?

Yes, the art of testing software is finding the best software to automate that type of application. Test Modeller integrates with a lot of different automation execution engines, including open source, commercial and home-grown technologies. That means that you can generate tests to execute against whichever software those frameworks support. You can see a list of integrations here.

What is the term ‘Shift Left’?

Shift left is a term which represents shifting parts of the SDLC to much earlier in the process, often even before a line of code is written. It means “shifting” testing “left” to much earlier in the development lifecycle. The benefits of this include faster and earlier feedback, as well as detecting defects before they significantly impact code.

If you scan two pages and there are similar things done on both, will the tool duplicate it? Or turn it into a Cypress custom command so it only exists once?

Yes, it will duplicate them if you scan the two pages. The art is knowing how to appropriately segment your objects. A good example is the product pages in an ecommerce system. It would not be good to scan every product page. Instead a generalised product object would be better suited.

Can you also demonstrate importing from existing gherkin and/or BPMN models?

Here is a video demo demonstrating Test Modeller importing Gherkin to flowchart models, generating new specifications and matching tests:

Does this work for API testing?

Yes, Test Modeller generates API tests and works with multiple frameworks for this. In fact, the complexity of interconnected API calls and their associated test data requirements makes model-based testing a great fit for model-based test design.

Here’s a video of Test Modeller working with one API test framework, API Fortress: Here

I co-presented a webinar on model-based testing earlier this year. You can watch the webinar recording here.

How would you do Mobile (Android and iOS) testing with Test Modeller?

You can utilise an Appium based framework in Test Modeller, or any integrated automation framework that supports mobile.

Does Test Modeller update page objects automatically when it is changed in the application

Yes, there is the possibility of having intelligent page objects which use a mixture of locators and update as automation is being executed. If one of the identifiers has been removed or changed, Test Modeller updates it during execution, using the remaining, valid identifiers. Here’s a demo videos showing the approach:

Does this reduce flaky tests? I’m not convinced that auto-generated code will be any more reliable. Does the generated code need any ‘tinkering’?

The model is used as a central resource. Instead of updating code, only the models need to be updated, which will then update any corresponding scripts utilising that component. This often significantly reduces time spent on test maintenance after a system change, typically without any ‘tinkering’.

What is the cost for pricing licenses? 

Pricing information is available on request – sales@curiosity.software

A free 14 day trial is available at testmodeller.io

If you start a free trial, have a look at the free tutorials to get started. They include a free Cypress tutorial.

Will model based testing benefit teams, if so how?

Modelling benefits teams by enabling collaboration (important in the current climate). Teams can work together on creating models which become a shared artefact across the SDLC. This “Living Documentation” helps remove siloes and avoid miscommunication, reducing the frustrating rework they create.

Does Test Modeller handle test framework updates automatically?

Yes, if you update the framework you can resynchronise objects to Test Modeller.

Can Test Modeller generate cypress statements?

Yes, the objects in Test Modeller are fully extendable. You can also create your own objects and corresponding code then overlay them on the model.

what alternative MBT tools exist, and why is this unique/superior?

GraphWalker is a popular opensource alternative. Strengths of Test Modeller includes its extensive integrations with other CI/CD tooling (e.g. JIRA, Git, ALM, and more), along with open source, commercial and homegrown test automation frameworks. The intuitive interface of Test Modeller often also makes it a preferred choice, along with the wide range of accelerators for building accurate models and tests quickly.

Can you mix mocked and real data returned from the API in the tests like Cypress supports ?

Currently, no. This is something we hope to include in a future release.

Thanks for reading! Want to try model-based testing for Cypress?