Ruthlessly Helpful

Stephen Ritchie's offerings of ruthlessly helpful software engineering practices.

Monthly Archives: October 2020

Boundary Analysis

For every method-under-test there is a set of valid preconditions and arguments. It is the domain of all possible values that allows the method to work properly. That domain defines the method’s boundaries. Boundary testing requires analysis to determine the valid preconditions and the valid arguments. Once these are established, you can develop tests to verify that the method guards against invalid preconditions and arguments.

Boundary-value analysis is about finding the limits of acceptable values, which includes looking at the following:

  • All invalid values
  • Maximum values
  • Minimum values
  • Values just on a boundary
  • Values just within a boundary
  • Values just outside a boundary
  • Values that behave uniquely, such as zero or one

An example of a situational case for dates is a deadline or time window. You could imagine that for a student loan origination system, a loan disbursement must occur no earlier than 30 days before or no later than 60 days after the first day of the semester.

Another situational case might be a restriction on age, dollar amount, or interest rate. There are also rounding-behavior limits, like two-digits for dollar amounts and six-digits for interest rates. There are also physical limits to things like weight and height and age. Both zero and one behave uniquely in certain mathematical expressions. Time zone, language and culture, and other test conditions could be relevant. Analyzing all these limits helps to identify boundaries used in test code.

Note: Dealing with date arithmetic can be tricky. Boundary analysis and good test code makes sure that the date and time logic is correct.

Invalid Arguments

When the test code calls a method-under-test with an invalid argument, the method should throw an argument exception. This is the intended behavior, but to verify it requires a negative test. A negative test is test code that passes if the method-under-test responds negatively; in this case, throwing an argument exception.

The test code shown here fails the test because ComputePayment is provided an invalid termInMonths of zero. This is test code that’s not expecting an exception.


[TestCase(7499, 1.79, 0, 72.16)]
public void ComputePayment_WithProvidedLoanData_ExpectProperMonthlyPayment(
  decimal principal,
  decimal annualPercentageRate,
  int termInMonths,
  decimal expectedPaymentPerPeriod)
{
  // Arrange
  var loan =
    new Loan
    {
      Principal = principal,
      AnnualPercentageRate = annualPercentageRate,
    };

  // Act
  var actual = loan.ComputePayment(termInMonths);

  // Assert
  Assert.AreEqual(expectedPaymentPerPeriod, actual);
}

The result of the failing test is shown, it’s output from Unexpected Exception.


LoanTests.ComputePayment_WithProvidedLoanData_ExpectInvalidArgumentException : Failed
System.ArgumentOutOfRangeException : Specified argument was out of the range of valid
values.

Parameter name: termInPeriods
at
Tests.Unit.Lender.Slos.Model.LoanTests.ComputePayment_WithProvidedLoanData_ExpectInvalidArgu
mentException(Decimal principal, Decimal annualPercentageRate, Int32 termInMonths, Decimal
expectedPaymentPerPeriod) in LoanTests.cs: line 25

The challenge is to pass the test when the exception is thrown. Also, the test code should verify that the exception type is InvalidArgumentException. This requires the method to somehow catch the exception, evaluate it, and determine if the exception is expected.

In NUnit this can be accomplished using either an attribute or a test delegate. In the case of a test delegate, the test method can use a lambda expression to define the action step to perform. The lambda is assigned to a TestDelegate variable within the Act section. In the Assert section, an assertion statement verifies that the proper exception is thrown when the test delegate is invoked.

The invalid values for the termInMonths argument are found by inspecting the ComputePayment method’s code, reviewing the requirements, and performing boundary analysis. The following invalid values are discovered:

  • A term of zero months
  • Any negative term in months
  • Any term greater than 360 months (30 years)

Below the new test is written to verify that the ComputePayment method throws an ArgumentOutOfRangeException whenever an invalid term is passed as an argument to the method. These are negative tests, with expected exceptions.


[TestCase(7499, 1.79, 0, 72.16)]
[TestCase(7499, 1.79, -1, 72.16)]
[TestCase(7499, 1.79, -2, 72.16)]
[TestCase(7499, 1.79, int.MinValue, 72.16)]
[TestCase(7499, 1.79, 361, 72.16)]
[TestCase(7499, 1.79, int.MaxValue, 72.16)]
public void ComputePayment_WithInvalidTermInMonths_ExpectArgumentOutOfRangeException(
  decimal principal,
  decimal annualPercentageRate,
  int termInMonths,
  decimal expectedPaymentPerPeriod)
{
  // Arrange
  var loan =
    new Loan
    {
      Principal = principal,
      AnnualPercentageRate = annualPercentageRate,
    };

  // Act
  TestDelegate act = () => loan.ComputePayment(termInMonths);

  // Assert
  Assert.Throws<ArgumentOutOfRangeException>(act);
}

Invalid Preconditions

Every object is in some arranged state at the time a method of that object is invoked. The state may be valid or it may be invalid. Whether explicit or implicit, all methods have expected preconditions. Since the method’s preconditions are not spelled out, one goal of good test code is to test those assumptions as a way of revealing the implicit expectations and turning them into explicit preconditions.

For example, before calculating a payment amount, let’s say the principal must be at least $1,000 and less than $185,000. Without knowing the code, these limits are hidden preconditions of the ComputePayment method. Test code can make them explicit by arranging the classUnderTest with unacceptable values and calling the ComputePayment method. The test code asserts that an expected exception is thrown when the method’s preconditions are violated. If the exception is not thrown, the test fails.

This code sample is testing invalid preconditions.


[TestCase(0, 1.79, 360, 72.16)]
[TestCase(997, 1.79, 360, 72.16)]
[TestCase(999.99, 1.79, 360, 72.16)]
[TestCase(185000, 1.79, 360, 72.16)]
[TestCase(185021, 1.79, 360, 72.16)]
public void ComputePayment_WithInvalidPrincipal_ExpectInvalidOperationException(
  decimal principal,
  decimal annualPercentageRate,
  int termInMonths,
  decimal expectedPaymentPerPeriod)
{
  // Arrange
  var classUnderTest =
  new Application(null, null, null)
  {
    Principal = principal,
    AnnualPercentageRate = annualPercentageRate,
  };

  // Act
  TestDelegate act = () => classUnderTest.ComputePayment(termInMonths);

  // Assert
  Assert.Throws<InvalidOperationException>(act);
}

Implicit preconditions should be tested and defined by a combination of exploratory testing and inspection of the code-under-test, whenever possible. Test the boundaries by arranging the class-under-test in improbable scenarios, such as negative principal amounts or interest rates.

Tip: Testing preconditions and invalid arguments prompts a lot of questions. What is the principal limit? Is it $18,500 or $185,000? Does it change from year to year?

More on boundary-value analysis can be found at Wikipedia https://en.wikipedia.org/wiki/Boundary-value_analysis

Better Value, Sooner, Safer, Happier

Jonathan Smart says agility across the organization is about delivering Better Value, Sooner, Safer, Happier. I like that catchphrase, and I’m looking forward to reading his new book, Sooner Safer Happier: Antipatterns and Patterns for Business Agility.

But what does this phrase mean to you? Do you think other people buy it?

Better Value – The key word here is value. Value means many things to many people; managers, developers, end-users, and customers.

In general, executive and senior managers are interested in hearing about financial rewards. These are important to discuss as potential benefits of a sustained, continuous improvement process. They come from long-term investment. Here is a list of some of the bottom-line and top-line financial rewards these managers want to hear about:

  • Lower development costs
  • Cheaper to maintain, support and enhance
  • Additional products and services
  • Attract and retain customers
  • New markets and opportunities

Project managers are usually the managers closest to the activities of developers. Functional managers, such as a Director of Development, are also concerned with the day-to-day work of developers. For these managers, important values spring from general management principles, and they seek improvements in the following areas:

  • Visibility and reporting
  • Control and correction
  • Efficiency and speed
  • Planning and predictability
  • Customer satisfaction

End-users and customers are generally interested in deliverables. When it comes to quantifying value they want to know how better practices produce better results for them. To articulate the benefits to end-users and customers, start by focusing on specific topics they value, such as:

  • More functionality
  • Easier to use
  • Fewer defects
  • Faster performance
  • Better support

Developers and team leads are generally interested in individual and team effectiveness. Quantifying the value of better practices to developers is a lot easier if the emphasis is on increasing effectiveness. The common sense argument is that by following better practices the team will be more effective. In the same way, by avoiding bad practices the team will be more effective. Developers are looking for things to run more smoothly. The list of benefits developers want to hear about includes the following:

  • Personal productivity
  • Reduced pressure
  • Greater trust
  • Fewer meetings and intrusions
  • Less conflict and confusion

Quantifying value is about knowing what others value and making arguments that make rational sense to them. For many, hard data is crucial. Since there is such variability from situation to situation, the only credible numbers are the ones your organization collects and tracks. It is a good practice to start collecting and tracking some of those numbers and relating them to the development practices that are showing improvements.

Beyond the numbers, there are many observations and descriptions that support the new and different practices. It is time to start describing the success stories and collecting testimonials. Communicate how each improvement is showing results in many positive ways.

Fakes, Stubs and Mocks

I’m frequently asked about the difference between automated testing terms like fakes, stubs and mocks.

The term fake is a general term for an object that stands in for another object; both stubs and mocks are types of fakes. The purpose of a fake is to create an object that allows the method-under-test to be tested in isolation from its dependencies, meeting one of two objectives:

1. Stub — Prevent the dependency from obstructing the code-under-test and to respond in a way that helps it proceed through its logical steps.

2. Mock — Allow the test code to verify that the code-under-test’s interaction with the dependency is proper, valid, and expected.

Since a fake is any object that stands in for the dependency, it is how the fake is used that determines if it is a stub or mock. Mocks are used only for interaction testing. If the expectation of the test method is not about verifying interaction with the fake then the fake must be a stub.

BTW, these terms are explained very well in The Art of Unit Testing by Roy Osherove.

There’s more to learn on this topic at stackoverflow https://stackoverflow.com/questions/24413184/difference-between-mock-stub-spy-in-spock-test-framework