FAQ

testRigor is different from Cucumber or Gherkin in fundamental way specifications are executable out of the box. There is no need to write the code. Whereas Gherkin is just a language to express specification. And then engineers have to write the code to implement it, which is not just a longer process, it's a three-step process, you read specification, write code, and then write test code, but also, what happens from time to time is that QA engineers misunderstand what needs to be done. Then once they coded it, it doesn't match what business meant, and therefore what's happening is just engineers would need to rewrite it again as you all know engineers' time is very expensive and hard to come by, so with testRigor, it makes this order of magnitude more efficient once baseline tests are all automated. What our customers do is they start to write specifications in English before functionality is coded by engineers. What it helps them to achieve is that not only, of course, is the order of magnitude less effort, because there's no QA engineering involved but most importantly, since those specifications are executable in English, if it's impossible to do by engineers, engineers will be able to discover those issues and discrepancies early on whereas no miscommunication between product management and engineering it kills ambiguity and what needs to be done.

Acceptance-level functional UI-level regression tests. We test that functionality works via UI. For example, for Amazon we would test that a person can successfully purchase a product from log in to purchase confirmation.

Our system is particularly adept at testing form-based UI or functionality that has a predictable input/output. A few other strengths unique to our tool are...

1. ...cases which require multiple users to interact be the same flow be it via email, sms, or instant messages.

2. ...cases built for platforms with constantly changing code and HTML IDs.

3. ...complex flows that need to combine actions like image comparison, audio quality, dragging & dropping all into one.

We don't test games, real-time graphs, video content. Stuff like that.

Even if your business is heavily dependent on any of the few things we do not test well, there is a good chance that you also have a lot of UI that we do test well.

Rather than dedicating testing resources to this type of tedious, repetitive & time-consuming work, you can unload that testing to testRigor and re-deploy your testers to your core testing needs.

TestRigor does test mobile applications both Android applications and iOS applications, both pure native applications and hybrid applications easily, even if hybrid applications might contain iframes, shadow doms, and so on.

Yes, we can offer an on-premise solution.

Installing our JavaScript library in your production environment will allow us to cover your most frequently used functionality & flows (up to 1,000 test cases). Secondly, you can also manually create custom tests using plain English or our browser plugin.

Beyond that, tests are run in parallel on our infrastructure, so prioritization is less relevant. We'll execute all tests every time unless you specify differently.

If you are using our JavaScript library & browser plugin, you can be assured that we are covering your most frequently used AND business critical functionality & flows. The best way to mange this is to name & tag these tests with the same naming system you are using in your current test management suite. Test cases are easily searchable.

We also offer a "tree" view that allows you to quickly see what areas of your application are being tested on any given test run.

By default up to 1,000. If you need more, just let us know.

It will obviously vary by application, but ideally you want to make sure you have covered your most frequently used functionality & flows AND all of your most critical business test cases.

Where you draw the line here is up to you, but it is unlikely that testRigor will ever be your roadblock. We can automatically discover as many edge cases beyond this as you'd like, but there is a law of diminishing returns beyond 1,000 test cases for most web & mobile applications.

Great question. Most teams have engaged in some level of attempts at automating their tests. For those who have, you've no doubt run into the challenges of creating, maintaining & scaling these tests.

The sad reality is legacy test automation approaches do not scale. Your application development will always outpace your ability to cover new functionality while maintaining flaky test suites. Legacy test automation approaches are a Sisyphean task!

testRigor will get you out of this vicious cycle with tests that mostly build themselves and self-maintain. This reduces your team's level of effort around regression testing so they can be re-deployed to more impactful testing initiatives.

Yes, we have a browser plugin that allows you to capture a manual test case with the click of a button. Once created, the test will automatically execute and maintain itself, allowing you to move on to more interesting test work.

For complicated test cases, you can use our Reusable Rules & Test Cases interface to create sophisticated tests using plain English commands. No code to learn!

With minimal training, all manual testers can be automation engineers!

TestRigor can handle complex user flows. You would think that, if you need something complex, you would need engineering. But that's not necessarily true because as soon as you can explain things in English, even on a higher level, this testRigor will allow you to do that. And therefore, because it's explained in English, non-technical people, you'll be able to use testRigor to achieve those goals. In addition to that, testRigor allows our customers to build tests for mobile, Web, APIs, text messages, phone calls, emails, you can access databases, and so on.

testRigor will automatically cover frequently used functionality up to 1,000 test cases. If there are specific edge cases you want to discover beyond this, you can quickly create a test by using our browser plugin or modifying existing tests.

New features will be surfaced by testRigor automatically on every new run with a suggested test script. Your team will then verify that this is indeed new functionality and approve/edit the test script. Boom! Test covered.

This is testRigor's secret sauce!

We are NOT dependent on XPath. Instead, our AI was trained to look at your web & mobile applications the same way a human would. Our tests are designed to emulate the way a manual tester would execute a test.

This approach means that our system can figure out UI changes in a way that conventional solutions are simply unable to do.

testRigor extracts visible elements like texts, inputs and buttons/links and allow you to work with just visible elements based on their type.

Nope. Since we emulate a real human running a test, your framework does not matter. We work great with SPA frameworks like React & Angular.

Very unlikely. Our clients have changed development platforms and UI frameworks and seen virtually all of their tests survive. We specifically built testRigor with adaptability and stability as our #1 priority.

You'll probably want to build several suites; one for common functionality and several for some customer-specific features. This is easy to build and customize in our test creation interface.

We handle everything on our servers. No need to worry about infrastructure headaches or cost-overruns. Call our APIs from Jenkins, CircleCI or any other CI/CD system you use.

We handle everything on our servers. No need to worry about infrastructure headaches or cost-overruns.

Email, Slack, API, and/or our web app. Totally up to you.

We can easily integrate with test-case management systems like TestRail and Zephyr.

Custom test cases are a way to create a test script using plain English instructions. Reusable rules help our system generate better tests and can be used as subroutines for any test.

You can even streamline your customizations by having custom test cases/reusable rules reference other test cases/rules.

TestRigor provides fanatical Support through Slack and MS Team channels. We strive to be exceptionally responsive to make sure that the pace of business of our customers is not disturbed.

We generate data based on detected field type (we support about 50 types), and you can provide your own test data as well as allow us to generate random data conforming to a specific format every time we run.

1. We emulate the way a human would interact with the application instead of hard-coded XPaths.

2. We have a sophisticated system which ensures that the page is fully loaded before proceeding.

3. We do reasonable retries for timeouts, browser crashes, etc.

4. Our system emulates the same behavior on the page that you as a human display by interacting only with visible elements (with some exceptions) on the screen or page.

The test creation is completely autonomous and comes from anonymized metadata about how your users are using your applications in production as well as from AI-enabled rules-based engine to cover most important functionality for your app.

We also provide you with a browser plugin to automatically capture tests your manual test team is executing today.

However, you can create/record your tests yourself using our Chrome Extension or just using plain English.

TestRigor is extremely secure. We take security very seriously. That is why we are SOC2 TYPE2 compliant and we run external security tests on a regular basis as well as security is built into our architecture technical reviews.

It's a question I can't answer a question because we have companies with as few as two people and all the way to Netflix.

You should use testRigor in order to be able to achieve your QA automation goals faster and if you want to reduce your test maintenance to almost none.

With testRigor you'll be able to involve your manual QA as well as business analysts, and product managers to be able to build your executable specifications in English even though you do not have QA automation engineers yet.

Any person in your company can write tests in testRigor. Either it's a manual tester, or a customer success person or even sales people if it's a smaller company. So it's totally up to you because tests are all in plain English and do not require any technical knowledge at all.

And testRigor will help you to hire directly people who are certified with testRigor, so we will connect you and help you to interview those people. They'll be able to build tests for you, potentially on your production systems, so we can assess how good we are and when you will be able to hire them permanently.

TestRigor is very different from any solution that is on the market and our main veiling proposition is the pretty much absence of test maintenance. So if you migrate all of your tests to testRigor, you will forget what maintenance is.

TestRigor differs from UFT, Tosca Ranorex, Test Complete, and others quite significantly, because in order to use those tools you would need to hire engineers. Whereas in case of testRigor, you'll be able to use your non-technical resources to build tests quickly. And those tests would be about 200 times more stable compared to the tests you were able to build in your systems.

testRigor uses a lot of machine learning models in order to be able to provide our customers with an unprecedented level of stability and service.

It uses a lot of AI to be able to understand what you're trying to say in English, how to find things on the screen, classify different types of images that are otherwise impossible to describe such as a shopping cart icon, or user icon as such. Plus, there are other additional models to help automatically detect that your screen is broken from the end-user perceptive.

testRigor is based on AI. We do have machine learning and test creation integrates with AI. We have crawler-based generated tests as well as behavior-driven test creation. With behavior-driven test creation, the tool analyzes the most frequent scenarios that your end-users are taking and creates adjustive tests based on those most frequent scenarios.

TestRigor does handle accessibility testing it's very easy to turn on. What kind of accessibility tests you would like to run and what is the level of accessibility errors you would want to get. We help our customers to stay compliant with FDA and other certifications.

And each company decides for themselves how deep they want to test accessibility, set up the settings, and there you go.

Speaking about integration with Jira we do have it out of the box. Our tool creates tickets in Jira and you get the URL of the error so it's just as simple as that.

TestRigor includes very exciting two ways integration with TestRail. Not only testRigor will set the status of TestRail test cases and move them to run as soon as tests in testRigor are executed, but also you can literally go to TestRail write your English executable specifications in TestRail itself and execute test cases directly from TestRail without even touching testRigor whatsoever.

You create tests in the TestRail, and then they automatically appear in testRigor. And if you match testRigor language, you already have a test and testRigor that is ready to go.

Our Price model is very simple we have SSO. So once you purchase the license, then anybody in your company can log in. We do not care how many tests you have. All that we charging for is the infrastructure. So that's the number of servers that run your tests in parallel.

TestRigor pricing is very company friendly. TestRigor only charges for infrastructure to execute tests for parallelizations. So the more parallelizations you purchase the faster your tests will be executed. We do not charge for number of users and we do not charge for the number of executions. You will be able to enable single sign-on so anybody in your company will be able to login into testRigor and collaborate together without additional charges.

For testRigor CI/CD integration is absolutely seamless. You can literally copy and paste already pre populated script into your CI/CD. We have a bash script for Linux-based systems and we have a PowerShell script for Microsoft systems.

There's a special bash script code that you copy and paste into your existing CI/CD pipeline. And it kicks the retests.

Those scripts will trigger testRigor, and you fail if testRigor fails and succeed if testRigor succeeds.

We help our customers to ease test maintenance by providing tools to help them to maintain their own tests. Moreover, we are the only company on Earth today that provides tools for customers to maintain their own tests. This way we achieve 200 times less time spent on test maintenance compared to other frameworks.

Talking about maintenance, there is no maintenance associated with the details of implementation. So no flaky failure maintenance. There's all maintenance that is connected to your business processes or something changes in your UI and that is easily corrected in the plain English body of a test case and it literally takes minutes to rephrase or change the processes.

Our stability is really our strongest feature. In fact, our customers use it as a monitoring tool.

TestRigor helps our customers to keep their tests stable just simply because testRigor executes tests emulating human interaction with a browser and testRigor does not rely on details of implementation like all other frameworks minuet details and how things are changing in HTML structure, XML structure or anything like that, do not affect the stability of the test whatsoever.

So no matter what happens in your XPath or selectors, tests are always stable. Now with the UI changes, we have some features that enable you to make changes really fast, and it groups arrows so that you spend minutes instead of days implementing those changes.

TestRigor can help you automate end-to-end tests. Even if you have something automated, we can help you transfer those tests into testRigor or create end to end tests with testRigor.

They oftentimes have a lot of manual tests which are not automated. So we help our manual QA to be able to quickly automate away your most automated parts and if you desire to do so, we can help you to quickly transition all existing tests and existing automation frameworks into testRigor, using our technology.

Our end-to-end tests are testing the functionality of web applications, mobile applications, implementations of SAP, Salesforce, Microsoft Dynamics, and so on and so forth. API tests, emails, text messages, phone calls, and anything else you can think of, what would you express from an end user's perspective in test case description.

Those end-to-end tests test things like the flows that your end users are taking. Let's say an example of Amazon that can take the whole procedure of buying and checking out the item. So that's really needed tests for majority of companies. And if you don't have those automated, you'd better do with testRigor.

The biggest problem that we face with customers is that once you create automation, you're stuck with maintenance. It's all those flaky failures that really drive you crazy. And with testRigor, there is no such problem because we do not rely on implementation details.

It helps our customers to be able to quickly and efficiently to get to 90 plus test automation test coverage. Often within under a year. And the problem we're solving is test maintenance. What we see is a lot of customers before the use of testRigor would be stuck at lower percentages of test automation, because all various sources are tied up in test maintenance. This is the problem we solve for our customers. They will be able to like nearly continue building tests using the same resources.

We are the only company that not only helps your non-technical employees to be able to effectively build automated tests 15 times faster than Selenium but also we are the only tool on the market that gives you tools to help you to reduce test maintenance.

TestRigor will be able to achieve automation in a matter of months using plain English with just non-technical people.

Not just that, our customers on average spend 200 times less time maintaining our automated tests compared to others.

And what makes us different from the rest of the companies is that once you create those tests, those tests require no maintenance.

TestRigor tests are so stable because we are running on the human level. The problem with Selenium and other frameworks on the market is they hook up with details of the implementation of the specific page. So once an engineer builds a page, engineers would hook up to details like XPath, CSS selectors, IDs, etc. As soon as those details change, test breaks, even though from a user's perspective there is no difference. Since testRigor operates on the end user level you are writing specifications in English and testRigor executes those specifications emulating how a manual tester would do that. It doesn't matter if implementation changes as soon as from the end users' perspective everything is the same, testRigor tests will succeed.

TestRigor does support BDD; however, BDD is a very broad term and there’s a little bit of mixing misconception about BDD is Gherkin Cucumber, which is not true. BDD is a process, which was designed by the creators of BDD to involve business owners into creating specifications that can be used to write tests. In that sense testRigor is a pure BDD framework. Moreover, it makes BDD order of magnitude more efficient because business owners sometimes with the help of QA and sometimes themselves can write those specifications in English the way that it's executable out of the box. So it actually would cut out the part where QA engineers would have to implement your specifications because the specifications themselves are executable. That's the point of testRigor and because of this difference with commonly used BDD, they call it SDD Specification Driven Development because your specifications are executable out of the box. So it's technically BDD. However, it emphasizes the fact that it's a two-step process, built specification, and code, rather than three steps built specification code write QA in addition to those specifications.

There's a very interesting thing about test coverage in the QA industry. It's not exactly well-defined. Because when you're talking to engineers, they mean one thing for test coverage; however, QA can mean a completely different thing. So for engineers, test coverage usually applies to unit tests indicate aligned coverage of lines of code. So how many lines of code are covered? You can measure that precisely, and you can calculate with precision. However, then it goes to QA because QA often deals with applications on a business level, and it deals with end-to-end tests rather than unit tests for QA Test coverage means how many business scenarios that had been outlined are covered.

As you can imagine, it's not a precise enough function because it's the function of how much time was invested into outlining all-important business scenarios to cover because you usually it's almost impossible to digest all of those. It usually covers only the most important ones. So in that regard, they believe that the right amount of test coverage is 100% of what can become, of course, various scenarios that account thought separately currently recovered, but what out of what can be carried this should be the reason why people think it shouldn't be 100% is because we used to do with older technologies, like Selenium, with Selenium's really a huge pain and effort to build tests, which is technically a struggle to get to high coverage percentage and is almost impossible to achieve; however, with testRigor you can achieve 100% test coverage within a month by copy-pasting your own test case written in English into testRigor and making executable using plain English as testRigor plain English execution system. This way it will be pretty quick and easy to automate everything you have, whatever can be automated. Moreover, they believe that all of those test cases must be automated because it improves the speed of business and if there are no issues with test maintenance anymore, as it would be with Selenium, you should cover all of your end-to-end tests scenarios because we believe end-to-end tests are the most critical tests, you can run them because they're validating actual business functions that your application functions from the end user's perspective end-to-end and your users can achieve what they set out to accomplish in your application.

We at testRigor believe that the order should be the most important functionality first so whatever is most business critical must be automated first such as you're making sure that your main core business functions correctly. TestRigor, of course, provides a super easy way for non-technical users including on the managers to do but it also provides a way to autonomously generate tests for you. You can deploy our analytics library in your production that analytics will gather how your users are using your application and your production and then testRigor use that information to automatically build tests covering most frequently used end-to-end scenarios. The most frequently used end-to-end scenarios are not necessarily the most important business functions, but they definitely do help, and it's always cool to get tests for free, isn't it?

We have two most efficient ways to use testRigor. The first part is how you automate your existing test cases. And for that, you can literally outline your own terminology in testRigor to be more specific actions, and then literally copy-paste your own test cases, which will become executable in testRigor out of the box. After that is done the next step would be writing test cases before engineering started working on a feature because your own terminology has already been outlined. Your product managers can write up how new features should work using their own terminology. This test would be executable before engineering starts and as soon as engineering starts there is no ambiguity about what needs to be done. And that specification itself is a test so you'll have test coverage going before engineering ATDD style for UI tests which will allow you to achieve as efficient engineering as it’s technically possible.

TestRigor can run test cases the way you write them. The way it works is very interesting. By default, testRigor has a specific language allowing you to express low-level steps such as clicks and entering data. However, you can group your actions into higher-level actions and give them arbitrary means. For example, if we were to test Amazon we would have actions like "find the product", "add to the cart", "checkout", and stuff like that. And each of them you would outline and say, Okay, what does checkout mean? And what our customers do is they usually have a lot of test cases already documented. In those test cases, we have a specific language, like as I mentioned, for Amazon, add to cart checkout, and so on. What we do is copy and paste that language into testRigor and outline in more specific steps what each chart means more specifically. After they outline their own terminology, what they do is they literally come back to where all test cases are, copy-paste the test cases into testRigor, and as a minimal cleanup, as it's technically possible, their own test cases are executable out of the box. Literally, your own test cases are written in your own language and will run on testRigor using this technic.

TestRigor has been built from the ground up to run tests in parallel. In fact, this is how tests run by default because in the realm of end-to-end tests, which would wait until your application would react to certain actions. The only way to achieve high enough speed in order, for example, to run the test on every commit is parallelization. And this is why testRigor had been always running the test in parallel.