Testing of calculator №6.2. Testing using pseudo-random data

Original in Russian: http://programmingmindstream.blogspot.com/2014/06/62.html

Table of contents

Having 4 tests for 4 operations we do not check the “great spread” of the input data accurately. To expand our testing coverage we get to our new chapter – “Testing using pseudo-random data”. It appears from the name that we need some random data set for testing. Test results will be written in the output file as in the previous chapter.

First of all, we perform a slight refactoring of our TCalculatorOperationViaEtalonTest class.
We introduce “function type” TCalcOperation:

type
  TCalcOperation = function (const A, B: string): string of object;

We rewrite AddArgumentsToLog function and add it to TCalculatorOperationViaEtalonTest class:

procedure TCalculatorOperationViaEtalonTest.CheckOperation(
                                                    aLogger: TLogger;
                                                    aX1, aX2: string;
                                                    anOperation : TCalcOperation);
begin
  aLogger.OpenTest(Self);
  aLogger.ToLog(aX1);
  aLogger.ToLog(aX2);
  aLogger.ToLog(anOperation(aX1,aX2));
  CheckTrue(aLogger.CheckWithEtalon);
end;

Our tests now look like this:

procedure TCalculatorOperationViaEtalonTest.TestDiv;
var
  x1, x2 : string;
begin
  x1:= cA;
  x2:= cB;
 
  CheckOperation(g_Logger, x1, x2, TCalculator.Divide);
end;

Next, we define a new class responsible for “tests with pseudo-random data” TCalculatorOperationRandomSequenceTest.

type
 TCalcOperation = function (const A, B: string): string of object;
 
 TCalculatorOperationRandomSequenceTest = class(TTestCase)
  private
   procedure CheckOperation(aLogger: TLogger;
                            aX1, aX2: Double;
                            anOperation : TCalcOperation);
   procedure CheckOperationSeq(aLogger: TLogger;
                               anOperation : TCalcOperation);
  published
   procedure TestDiv;
   procedure TestMul;
   procedure TestAdd;
   procedure TestSub;
 end;//TCalculatorOperationRandomSequenceTest

Now we have a new procedure CheckOperationSeq, which has “taken away” part of CheckOperation features, namely:

procedure TCalculatorOperationRandomSequenceTest.CheckOperationSeq(
  aLogger: TLogger;
  anOperation: TCalcOperation);
begin
  aLogger.OpenTest(Self);
  CheckOperation(aLogger, 5, 10, anOperation);
  CheckTrue(aLogger.CheckWithEtalon);
end;

procedure TCalculatorOperationRandomSequenceTest.CheckOperation(
  aLogger: TLogger;
  aX1, aX2: Double;
  anOperation : TCalcOperation);
begin
  aLogger.ToLog(aX1);
  aLogger.ToLog(aX2);
  aLogger.ToLog(anOperation(FloatToStr(aX1),FloatToStr(aX2)));
end;

As you can see, we have to call CheckOperationSeq for testing. In its turn, it will call CheckOperation with the parameters specified on launch. Along with it, twice we pass the required anOperation : TCalcOperation function for call. Next, we “reload” the procedure of filing so that it could “understand” Double:

...
   procedure ToLog(const aParametr: Double); overload;
...
 
procedure TLogger.ToLog(const aParametr: Double);
begin
  Writeln(FTestFile, FloatToStr(aParametr) + ' ');
end;

Finally, and that is the reason why we ventured the previous changes, we change the procedure of “checking the sequence” so that it checks random arguments. As we can see, the second argument equals 2000 * Random + 1, one is added to avoid random division by 0. However, we’ll mention the issue of processing the exceptions while testing in the coming articles.

I’d like to mention separately the first line in procedure RandSeed := 40000; - in this way, we fix “Random” and so our sequence is always the same. We need it in order to avoid repeated “uploading” of our etalons.

procedure TCalculatorOperationRandomSequenceTest.CheckOperationSeq(
  aLogger: TLogger;
  anOperation: TCalcOperation);
var
  l_Index : Integer;
begin
  RandSeed := 40000;
  aLogger.OpenTest(Self);
  for l_Index := 0 to 10000 do
    CheckOperation(aLogger,
                   1000 * Random,
                   2000 * Random + 1, anOperation);
  CheckTrue(aLogger.CheckWithEtalon);
end;

As a result, 10k alternatives of testing will be done for each operation. In theory, we can do more; it depends on the power of the machine you run tests on. I could sum up at this point but, after Alexander has uploaded the source code, I’ve discovered a problem with regional adjustments: on Alexander’s computer decimal numbers were written with comma, on my PC – with a point. All this might pass, but our “etalons” were already uploaded to git with a point. Alexander “patched it in rough haste” using the new method of class TCalculator:

class function TCalculator.FloatToStr(aValue: Double): string;
var
 l_FS : TFormatSettings;
begin
  l_FS := TFormatSettings.Create;
  l_FS.DecimalSeparator := '.';
  Result := SysUtils.FloatToStr(aValue, l_FS);
end;

The whole listing of our new class:

unit CalculatorOperationRandomSequenceTest;
 
interface
 
uses
  TestFrameWork,
  Calculator,
  Tests.Logger;
 
 type
  TCalcOperation = function (const A, B: string): string of object;
 
  TCalculatorOperationRandomSequenceTest = class(TTestCase)
   private
    procedure CheckOperation(aLogger: TLogger;
                             aX1, aX2: Double;
                             anOperation : TCalcOperation);
    procedure CheckOperationSeq(aLogger: TLogger;
                                anOperation : TCalcOperation);
   published
    procedure TestDiv;
    procedure TestMul;
    procedure TestAdd;
    procedure TestSub;
  end;//TCalculatorOperationRandomSequenceTest
 
implementation
 
  uses
    SysUtils;
 
 
{ TCalculatorOperationRandomSequenceTest }
procedure TCalculatorOperationRandomSequenceTest.CheckOperationSeq(
  aLogger: TLogger;
  anOperation: TCalcOperation);
var
  l_Index : Integer;
begin
  RandSeed := 40000;
  aLogger.OpenTest(Self);
  for l_Index := 0 to 10000 do
    CheckOperation(aLogger,
                   1000 * Random,
                   2000 * Random + 1, anOperation);
  CheckTrue(aLogger.CheckWithEtalon);
end;
 
 
procedure TCalculatorOperationRandomSequenceTest.CheckOperation(
  aLogger: TLogger;
  aX1, aX2: Double;
  anOperation : TCalcOperation);
begin
  aLogger.ToLog(aX1);
  aLogger.ToLog(aX2);
  aLogger.ToLog(anOperation(FloatToStr(aX1),FloatToStr(aX2)));
end;
 
procedure TCalculatorOperationRandomSequenceTest.TestDiv;
begin
  CheckOperationSeq(g_Logger, TCalculator.Divide);
end;
 
procedure TCalculatorOperationRandomSequenceTest.TestSub;
begin
  CheckOperationSeq(g_Logger, TCalculator.Sub);
end;
 
procedure TCalculatorOperationRandomSequenceTest.TestMul;
begin
  CheckOperationSeq(g_Logger, TCalculator.Mul);
end;
 
procedure TCalculatorOperationRandomSequenceTest.TestAdd;
begin
  CheckOperationSeq(g_Logger, TCalculator.Add);
end;
 
initialization
 TestFramework.RegisterTest(TCalculatorOperationRandomSequenceTest.Suite);
end.

So, we’ve added 40k tests by 10k for each operation. Due to implementing “testing using etalons” the results of all tests are committed in version control system. While writing this testing, we kept business logic the same and found out it is correct basing on a previous, though minimal, testing.

In this way, we’ve moved from minimal testing of the application to regression testing, that will be useful throughout the whole “lifecycle” of our software.

Repository