net.grinder.script

Interface Statistics

public interface Statistics

Script statistics query and reporting API.

An implementation of this interface can be obtained by calling getStatistics on the grinder context object.

Statistics

The following table lists the statistics provided by The Grinder. Each statistic has a unique name. Basic statistics either hold long integer values or double floating-point values. Sample statistics are a special type of statistic that hold aggregate information about a series of long or double sample values; specifically count (number of samples), sum (total of all sample values), and sample variance.

Name	Type	Description
errors	basic long	The number of tests performed that resulted in an error.
timedTests	sample long	Sample statistic that records successful tests. A test is considered successful if it is not marked as an error.
userLong0, userLong1, userLong2, userLong3, userLong4	basic long	Statistics that scripts and custom plug-ins can use for their own purposes.
userDouble0, userDouble1, userDouble2, userDouble3, userDouble4	basic double	Statistics that scripts and custom plug-ins can use for their own purposes.
untimedTests	basic long	The number of successful tests performed that have no timing information. This statistic only used when reporting statistics to the console, it has no meaning in a worker process.
period	basic long	The sampling period duration, in milliseconds. This statistic is used to define views for evaluation in the console and for the overall time used to calculate rates in the Totals line in the final summary table.
peakTPS	basic double	The highest Tests Per Second figure in the current sampling period. This statistic is used to define views for evaluation in the console and has no meaning in a worker process.

The getForCurrentTest() and getForLastTest() methods allow scripts to query or update statistics for a single test. Consequentially, the statistics should be interpreted as follows when using this interface.

Name	Type	Meaning for a single test
errors	basic long	1 if the test resulted in an error, otherwise 0.
timedTests	sample long	If the test was successful, the count is 1 and the sum is the test time in milliseconds, otherwise the sum and the count are zero. The variance is always 0.
userLong0, userLong1, userLong2, userLong3, userLong4	basic long	Depends on how the script or custom plug-in chooses to use the statistic.
userDouble0, userDouble1, userDouble2, userDouble3, userDouble4	basic double	Depends on how the script or custom plug-in chooses to use the statistic.
untimedTests	basic long	Not relevant.
period	basic long	Not relevant.
peakTPS	basic double	Not relevant.

The Grinder updates the statistics for each test just before it is recorded as follows:

• If errors is 0, the elapsed time of the test is added to the timedTests sample statistic.
• If errors is not 0, the timedTests and untimedTests statistics are reset to zero, and errors is set to 1.

If the grinder.reportTimesToConsole property (see The Grinder manual ) is false, the statistics sent to the console are further modified by setting untimedTests to the count of the timedTests statistic, and resetting timedTests.

HTTP Plug-in Statistics

The HTTP plug-in adds a number of basic long statistics. Although these will be updated for a single test, many HTTP requests might be wrapped in that test. Where there is more than one HTTP request, the statistic values will reflect the sum for all requests unless otherwise stated.

Name	Type	Description
httpplugin.responseStatus	basic long	The HTTP status code of the response to the last HTTP request wrapped in the test.
httpplugin.responseLength	basic long	The length of the HTTP response in bytes.
httpplugin.responseErrors	basic long	1 if the HTTP response status code was greater or equal to 400, otherwise 0.
httpplugin.dnsTime	basic long	The time taken to resolve the host name in milliseconds.
httpplugin.connectTime	basic long	The time taken to establish the HTTP connection in milliseconds. (This includes time to resolve the host name).
httpplugin.firstByteTime	basic long	The time taken to receive the first response byte in milliseconds. (This includes time to resolve the host name and establish the connection).

Querying statistics

Statistics.StatisticsForTest.getDouble(String) and Statistics.StatisticsForTest.getLong(String) provide basic statistics about the current or last test performed by the calling worker thread. There is also Statistics.StatisticsForTest.getSuccess(), which is equivalent to getLong("errors") != 0.

Example of querying the result of a test:

 result1 = test1.doSomething()

 statisticsForTest = grinder.statistics.forLastTest

 if statisticsForTest.success and \
   statisticsForTest.getLong("httpplugin.responseStatus") == 200:
   # ...
 

Calling query methods on the result of getForLastTest() provides information about the last test performed by the calling worker thread.

Calling query methods on the result of getForCurrentTest() from within code wrapped by a Test proxy provides information about the test in progress. This information may be partially complete.

There are no general methods to access the count, sum, and variance of sample statistics; these terms aren't that meaningful for a single test. Instead, there are specific methods which influence the only sample statistic used by The Grinder - the timedTests statistic. The time of the last test can be obtained with Statistics.StatisticsForTest.getTime() (or the elapsed time of the current test when used with getForCurrentTest()}, and Statistics.StatisticsForTest.getSuccess() returns whether the test was successful or not.

There's a subtle difference between the sum of timedTests and the result of Statistics.StatisticsForTest.getTime(). Statistics.StatisticsForTest.getTime() always returns the time taken by the test, even if the test was an error and the time will not be added to timedTests. This allows the script to access the time taken by a failed test, even though it's not recorded anywhere else.

Updating statistics

The following methods allow basic statistics to be updated.

• Statistics.StatisticsForTest.setSuccess(boolean)
• Statistics.StatisticsForTest.setLong(String, long)
• Statistics.StatisticsForTest.setDouble(String, double)
• Statistics.StatisticsForTest.addLong(String, long)
• Statistics.StatisticsForTest.addDouble(String, double)

The only way to influence the timedTests sample statistic through this interface is to mark the test as an error.

By default, test statistics reports are automatically sent to the console and data log when the test proxy call completes, so the script cannot modify the test statistics after the call. Scripts can turn off this automatic reporting for the current worker thread by using setDelayReports(boolean). Having done so, the script can modify or set the statistics before they are sent to the log and the console. The statistics reports are sent at a later time as described in setDelayReports(boolean). For example:

 grinder.statistics.delayReports = 1

 result1 = test1.doSomething()

 if isFailed(result1):
   # Mark test as failure. The appropriate failure detection
   # depends on the type of test.
   grinder.statistics.forLastTest.success = 0
 

It is also possible to set the statistics from within test implementation itself using getForCurrentTest(). Changes to the standard statistics will be modified by The Grinder engine when the test finishes as described above.

Registering new expressions

New statistics expressions for the console and the data log can be registered using registerSummaryExpression(String, String) and registerDataLogExpression(String, String).

Author:

Philip Aston

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	Statistics.StatisticsForTest Query and update methods for the statistics relating to a particular call of a test.

Method Summary

Methods

Modifier and Type	Method and Description
Statistics.StatisticsForTest	getForCurrentTest() Access to the statistics for the current test.
Statistics.StatisticsForTest	getForLastTest() Access the statistics for the last completed test.
boolean	isTestInProgress() Returns whether there is a test in progress.
void	registerDataLogExpression(String displayName, String expression) Register a new "detail" statistic expression.
void	registerSummaryExpression(String displayName, String expression) Register a new "summary" statistic expression.
void	report() Send any pending statistics for the last completed test to the data log and the console.
void	setDelayReports(boolean b) Use to delay reporting of the last test statistics to the log and the console so that the script can modify them.

Method Detail

getForCurrentTest

Statistics.StatisticsForTest getForCurrentTest()
                                               throws InvalidContextException

Access to the statistics for the current test.

This is only valid when called from code wrapped within a Test. If this is not the case, InvalidContextException will be thrown. E.g.

 def foo():
   statistics = grinder.statistics.getForCurrentTest()
   t = statistics.time      # Time since foo() was called.
   statistics.success = 0   # Mark test as bad.

 instrumentedFoo = Test(1, "My Test").wrap(foo)

 instrumentedFoo()              # OK.
 foo()                          # The statistics.getForCurrentTest() call in
                                # foo() will throw exception as there is no
                                # current test.
 statistics.getForCurrentTest() # Will throw exception, no current test.
 

Returns:

The statistics for the current test.

Throws:

InvalidContextException - If not called from a worker thread.

InvalidContextException - If there is no test in progress.

See Also:

getForLastTest(), isTestInProgress()

getForLastTest

Statistics.StatisticsForTest getForLastTest()
                                            throws InvalidContextException

Access the statistics for the last completed test. These can only be updated if reporting has been delayed.

Returns:

The statistics for the last test.

Throws:

InvalidContextException - If not called from a worker thread.

InvalidContextException - If called before the first test has started.

See Also:

getForCurrentTest()

isTestInProgress

boolean isTestInProgress()

Returns whether there is a test in progress.

Returns:

true getForCurrentTest() will not throw InvalidContextException.

setDelayReports

void setDelayReports(boolean b)
                     throws InvalidContextException

Use to delay reporting of the last test statistics to the log and the console so that the script can modify them. Normally test statistics are reported automatically when the code wrapped by the test completes.

With this set to true the statistics for a completed test will be reported at the following times:

1. When the next test begins.
2. When an enclosing test ends.
3. When the current run completes.
4. When the script calls report().
5. When the script calls setDelayReports(false).

This method only changes reporting for the calling worker thread.

Parameters:

b - false => enable automatic reporting when the code wrapped by a test completes (the default behaviour); true => delay reporting.

Throws:

InvalidContextException - If not called from a worker thread.

See Also:

report()

report

void report()
            throws InvalidContextException

Send any pending statistics for the last completed test to the data log and the console.

Calling this does nothing if there are no pending statistics to report. This will be the case if setDelayReports(boolean) has not been called to delay reporting.

Throws:

InvalidContextException - If not called from a worker thread.

registerSummaryExpression

void registerSummaryExpression(String displayName,
                             String expression)
                               throws GrinderException

Register a new "summary" statistic expression. This expression will appear in the worker process output log summaries and the console.

Statistic expressions are composed of statistic names (see Statistics) in a simple post-fix format using the symbols +, -, / and *, which have their usual meanings, in conjunction with simple statistic names or sub-expressions. Precedence is controlled by grouping expressions in parentheses. For example, the error rate is (* (/ errors period) 1000) errors per second. The symbol sqrt can be used to calculate the square root of an expression.

Sample statistics, such as timedTests, must be introduced with one of sum, count, or variance, depending on the attribute of interest.

For example, the statistic expression (/ (sum timedTests) (count timedTests)) represents the mean test time in milliseconds.

Parameters:

displayName - A display name. In the console, this is converted to a key for an internationalised resource bundle look up by prefixing the string with statistic. and replacing any whitespace with underscores.

expression - The expression string.

Throws:

GrinderException - If the expression could not be registered.

registerDataLogExpression

void registerDataLogExpression(String displayName,
                             String expression)
                               throws GrinderException,
                                      InvalidContextException

Register a new "detail" statistic expression. This expression will appear in the worker process data log. Each test invocation will have an value displayed for the detail statistic expression.

You should call registerDataLogExpression from the top level of your script. It cannot be called from a worker thread - the data logs are initialised by the time the worker threads start.

Parameters:

displayName - A display name.

expression - The expression string.

Throws:

GrinderException - If the expression could not be registered.

InvalidContextException - If called from a worker thread.

See Also:

for details of the expression format.