Skip to content
May 15, 2012 / linkconsulting

Testing ADF + BPM from end-user perspective

The challenge

Every application MUST be tested. Like any other application, tests are ran during development, or short development periods (http://en.wikipedia.org/wiki/Test-driven_development). But unit tests are not all there is to it. How can you test an application through the client’s point of view? And at the same time ensure correctness of both ADF and BPM components?

The solution

Tools

First of all, it is not feasible to define tests on the fly. Tests must be defined, agreed upon, and repeat/improved as the solution grows.

Therefore a testing tool must be used. Most common tools not only allow you to defined tests and store that definition; they also keep track of test history, evolution and completeness.

Several tools are available for our goal: tests from the client’s point of view. The most known ones are debatably Selenium and HP Quality Center. Different in size and complexity, these tools provide features for robust tests.

In our case, HP Quality Center was chosen for several reasons:

  • Enterprise grade;
  • Enterprise support;
  • Internal knowledge;
  • Set of features;
  • Distributed approach to testing.

Organization

As a solution grows, so does the test range. A common malpractice is to define tests “as we go”, which in a worst case scenario might lead to an unmanageable state down the road.

Organizing your tests is crucial! But how to properly organize?

We found this division to work well for us:

  • First of all, our solution is scheduled in different phases;
  • Each phase has its own set of requirements and features that must be implemented;
  • Each feature/requirement might have business constraints.

Furthermore, a single solution phase might in itself contain so many requirements to be satisfied that considering them as being in the same level might slightly complicate organization of tests.

We defined the following hierarchy:

  • Project: the project we want to test. This allows our tool to manage multiple solutions with a well defined boundary between them;
  • Phase: the phase of the solution the tests refer to. This is especially relevant for regression testing;
  • High level feature: this represents a high level separation of the solution. For example, “Authentication” would contain all sub-features and requirements associated with this topic, same for “Cars”, “Vegetables”, “Juices”, “Reporting”, etc. Obviously it is strongly dependent on the solution context, but this first high level allows for a simpler obvious organization that counters feature/requirement overcrowding in on level;
  • Specific features: groups up specific features considering the high level one. A high level feature “Car” might contain specific features such as “list cars”, “search cars”, “validate car forms”, etc;
  • Requirements: bottom level where the business requirements are stored. Each requirement is easily identified as to which feature/part of the solution it refers to.

Definition

Having our organization, the test definition must follow it. In this stage we must define what the tests are. At this point our organization eases our task. We know what to test in a finite set of requirements and features.

Considering the requirements associated with the “Car” high level feature (and subsequent specific features and requirements), we might want to test listing cars through using different sorting criteria, search cars through different parameters, etc.

This is the core issue with client’s perspective testing, achieving an exhaust enough test set that comfortably produces confidence in the solution. Additional issues however arise as well:

  • Ensure BPM correctness (ADF correctness is only considered at the “Execution” level);
  • Ensure coverage;
  • Ensure “out of the box” cases.

Most straight forward features don’t need to be tested (but tested them anyway). The client’s inexperienced/unexpected behavior is the one that must be tested. I’ll leave some examples.

Concurrency

Imagine your BPM flow has event triggers for when a notification is answered (e.g. replying to a customer inquiry). If two employees access the notification simultaneously, and answer simultaneously, how does your flow deal with it?

Oracle BPM has concurrency management as any decent enterprise grade framework, but does your application deal gracefully with it? Either by merging notifications, discarding one with a meaningful error message, or just leave one of the clients in the blank?

Multiple paths

In BPM it is possible to reach the same target through different paths. It is tightly coupled with the solution in hands, but it is a frequent enough case. Imagine you can consult properties of a car through a car detail page, or through a search by property. Changing a property will reflect in both paths?

You might be able to answer a client’s inquiry through a popup on your application, through a list of inquiries, or through a worklist approach. Using each option leads to the same result?

These are just a few examples of not so trivial tests that effectively cover BPM through the client’s perspective. Even if they do not understand what is happening under the hood, they know that one action must produce a result, whatever method used. Take in consideration your context and think of the different ways the same action can be performed, how many people can perform it, and how different parts of the solution deal with change.

Execution

Test execution is debatably the most boring task regarding tests. It’s a repetitive task with a strict set of steps. It is through execution however that you ensure that everything is working as they are supposed to.

A normal execution consists of two parts, steps and expected result.

Steps

The steps must be as explicit and exhaustive as possible. Obviously your consider the person testing has some intelligence (don’t write steps such as “click on the ‘G’ letter on your keyboard”), but do not make assumptions on their knowledge of the solution. Any person should be able to take follow your steps and achieve the expected result even if it is the first time they are interacting with the solution.

Expected result

This part ensures ADF correct implementation. The expected result must be detailed.

It must consider error cases as well. Error messages/pages, validation inconsistencies, etc, must be part of both the test plan and test execution.

Automation

We all agree that strict repetitive tasks are boring, so why not let the computer do the job for us?

There are several tools for test automation. The inner workings are similar between them. Usually it can be broken down to these parts:

  • Record an action or several actions;
  • Record how the expected result should look like;
  • Replay the recorded action(s).

In our environment we use QuickTest Pro which is easily integrated with HP’s Quality Center.

Automation has several limitations however. In a frequently changed solution (in terms of interface) worst case scenario is that any change requires a re-recording of the test, defeating the purpose of automation.

Automated tests are particularly suited for regression tests, ensuring that previously implemented and working features remain unchanged despite new feature additions.

Another approach (dependant of the tool used) is to divide a recording in smaller parts, allowing the reuse of some steps, e.g. logging in, accessing an option on the menu, etc. With this approach, changes might only require the re-recording of the smaller step set associated with that part.

Reports

Finally, you have to show your tests results. You can explain your managers that testing is great, and the tools are awesome, but nothing says more to them than a nice graph which they actually understand.

HP Quality Center provides reporting tools which produce useful information regarding the solution. You can easily see how your features/requirements coverage is being solved over the time, tests ran, etc. (see figure 1).

Additionally to the usefulness of graph representation for management purposes, it is useful as well for development, so they can understand how a solution is evolving during its lifetime.

Conclusion

This post was a combination of theoretical and practical matters. Hopefully the concepts and fundamentals will be passed on.

On Testing ADF + BPM from end-user perspective part 2 we will look on how to implement some of the concepts used here.

Until then I hope you enjoyed!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: