Overview  Package   Class  Tree  Deprecated  Index  Help 
 PREV CLASS   NEXT CLASS FRAMES    NO FRAMES    
SUMMARY: NESTED | FIELD | CONSTR | METHOD DETAIL: FIELD | CONSTR | METHOD

net.grinder.script
Interface Statistics

public interface Statistics

Script statistics query and reporting API.

An implementation of this interface can be obtained by calling getStatistics on the grinder context object.

Statistics

The following table lists the statistics provided by The Grinder. Each statistic has a unique name. Basic statistics either hold long integer values or double floating-point values. Sample statistics are a special type of statistic that hold aggregate information about a series of long or double sample values; specifically count (number of samples), sum (total of all sample values), and sample variance.

Name Type Description
errors basic long The number of tests performed that resulted in an error.
timedTests sample long Sample statistic that records successful tests. A test is considered successful if it is not marked as an error.
userLong0, userLong1, userLong2, userLong3, userLong4 basic long Statistics that scripts and custom plug-ins can use for their own purposes.
userDouble0, userDouble1, userDouble2, userDouble3, userDouble4 basic double Statistics that scripts and custom plug-ins can use for their own purposes.
untimedTests basic long The number of successful tests performed that have no timing information.
This statistic only used when reporting statistics to the console, it has no meaning in a worker process.
period basic long The sampling period duration, in milliseconds.
This statistic is used to define views for evaluation in the console and has no meaning in a worker process.
peakTPS basic double The highest Tests Per Second figure in the current sampling period.
This statistic is used to define views for evaluation in the console and has no meaning in a worker process.

The getForCurrentTest() and getForLastTest() methods allow scripts to query or update statistics for a single test. Consequentially, the statistics should be interpreted as follows when using this interface.

Name Type Meaning for a single test
errors basic long 1 if the test resulted in an error, otherwise 0.
timedTests sample long If the test was successful, the count is 1 and the sum is the test time in milliseconds, otherwise the sum and the count are zero. The variance is always 0.
userLong0, userLong1, userLong2, userLong3, userLong4 basic long Depends on how the script or custom plug-in chooses to use the statistic.
userDouble0, userDouble1, userDouble2, userDouble3, userDouble4 basic double Depends on how the script or custom plug-in chooses to use the statistic.
untimedTests basic long Not relevant.
period basic long Not relevant.
peakTPS basic double Not relevant.

The Grinder updates the statistics for each test just before it is recorded as follows:

If the grinder.reportTimesToConsole property (see The Grinder manual ) is false, the statistics sent to the console are further modified by setting untimedTests to the count of the timedTests statistic, and resetting timedTests.

HTTP Plug-in Statistics

The HTTP plug-in adds a number of basic long statistics. Although these will be updated for a single test, many HTTP requests might be wrapped in that test. Where there is more than one HTTP request, the statistic values will reflect the sum for all requests unless otherwise stated.

Name Type Description
httpplugin.responseStatus basic long The HTTP status code of the response to the last HTTP request wrapped in the test.
httpplugin.responseLength basic long The length of the HTTP response in bytes.
httpplugin.responseErrors basic long 1 if the HTTP response status code was greater or equal to 400, otherwise 0.
httpplugin.dnsTime basic long The time taken to resolve the host name in milliseconds.
httpplugin.connectTime basic long The time taken to establish the HTTP connection in milliseconds. (This includes time to resolve the host name).
httpplugin.firstByteTime basic long The time taken to receive the first response byte in milliseconds. (This includes time to resolve the host name and establish the connection).

Querying statistics

Statistics.StatisticsForTest.getDouble(String) and Statistics.StatisticsForTest.getLong(String) provide basic statistics about the current or last test performed by the calling worker thread. There is also Statistics.StatisticsForTest.getSuccess(), which is equivalent to getLong("errors") != 0.

Example of querying the result of a test: 

 result1 = test1.doSomething()

 statisticsForTest = grinder.statistics.forLastTest

 if statisticsForTest.success and \
   statisticsForTest.getLong("httpplugin.responseStatus") == 200:
   # ...

Calling query methods on the result of getForLastTest() provides information about the last test performed by the calling worker thread.

Calling query methods on the result of getForCurrentTest() from within code wrapped by a Statistics.StatisticsForTest proxy provides information about the test in progress. This information may be partially complete.

There are no general methods to access the count, sum, and variance of sample statistics; these terms aren't that meaningful for a single test. Instead, there are specific methods which influence the only sample statistic used by The Grinder - the timedTests statistic. The time of the last test can be obtained with Statistics.StatisticsForTest.getTime() (or the elapsed time of the current test when used with getForCurrentTest()}, and Statistics.StatisticsForTest.getSuccess() returns whether the test was successful or not.

There's a subtle difference between the sum of timedTests and the result of Statistics.StatisticsForTest.getTime(). Statistics.StatisticsForTest.getTime() always returns the time taken by the test, even if the test was an error and the time will not be added to timedTests. This allows the script to access the time taken by a failed test, even though it's not recorded anywhere else.

Updating statistics

The following methods allow basic statistics to be updated.

The only way to influence the timedTests sample statistic through this interface is to mark the test as an error.

By default, test statistics reports are automatically sent to the console and data log when the test proxy call completes, so the script cannot modify the test statistics after the call. Scripts can turn off this automatic reporting for the current worker thread by using setDelayReports(boolean). Having done so, the script can modify or set the statistics before they are sent to the log and the console. The statistics reports are sent at a later time as described in setDelayReports(boolean). For example: 

 grinder.statistics.delayReports = 1

 result1 = test1.doSomething()

 if isFailed(result1):
   # Mark test as failure. The appropriate failure detection
   # depends on the type of test.
   grinder.statistics.forLastTest.success = 0

It is also possible to set the statistics from within test implementation itself using getForCurrentTest(). Changes to the standard statistics will be modified by The Grinder engine when the test finishes as described above.

Registering new expressions

New statistics expressions for the console and the data log can be registered using registerSummaryExpression(String, String) and registerDataLogExpression(String, String).

Author:
Philip Aston

Nested Class Summary
static interface Statistics.StatisticsForTest
          Query and update methods for the statistics relating to a particular call of a test.
 
Method Summary
 Statistics.StatisticsForTest getForCurrentTest()
          Access to the statistics for the current test.
 Statistics.StatisticsForTest getForLastTest()
          Access the statistics for the last completed test.
 boolean isTestInProgress()
          Returns whether there is a test in progress.
 void registerDataLogExpression(String displayName, String expression)
          Register a new "detail" statistic expression.
 void registerSummaryExpression(String displayName, String expression)
          Register a new "summary" statistic expression.
 void report()
          Send any pending statistics for the last completed test to the data log and the console.
 void setDelayReports(boolean b)
          Use to delay reporting of the last test statistics to the log and the console so that the script can modify them.
 

Method Detail

getForCurrentTest

public Statistics.StatisticsForTest getForCurrentTest()
                                               throws InvalidContextException
Access to the statistics for the current test.

This is only valid when called from code wrapped within a Test. If this is not the case, InvalidContextException will be thrown. E.g. 

 def foo():
   statistics = grinder.statistics.getForCurrentTest()
   t = statistics.time      # Time since foo() was called.
   statistics.success = 0   # Mark test as bad.

 instrumentedFoo = Test(1, "My Test").wrap(foo)

 instrumentedFoo()              # OK.
 foo()                          # The statistics.getForCurrentTest() call in
                                # foo() will throw exception as there is no
                                # current test.
 statistics.getForCurrentTest() # Will throw exception, no current test.

Returns:
The statistics for the current test.
Throws:
InvalidContextException - If not called from a worker thread.
InvalidContextException - If there is no test in progress.
See Also:
getForLastTest(), isTestInProgress()

getForLastTest

public Statistics.StatisticsForTest getForLastTest()
                                            throws InvalidContextException
Access the statistics for the last completed test. These can only be updated if reporting has been delayed.

Returns:
The statistics for the last test.
Throws:
InvalidContextException - If not called from a worker thread.
InvalidContextException - If called before the first test has started.
See Also:
getForCurrentTest()

isTestInProgress

public boolean isTestInProgress()
Returns whether there is a test in progress.

Returns:
true getForCurrentTest() will not throw InvalidContextException.

setDelayReports

public void setDelayReports(boolean b)
                     throws InvalidContextException
Use to delay reporting of the last test statistics to the log and the console so that the script can modify them. Normally test statistics are reported automatically when the code wrapped by the test completes.

With this set to true the statistics for a completed test will be reported at the following times:

  1. When the next test begins.
  2. When an enclosing test ends.
  3. When the current run completes.
  4. When the script calls report().
  5. When the script calls setDelayReports(false).

This method only changes reporting for the calling worker thread.

Parameters:
b - false => enable automatic reporting when the code wrapped by a test completes (the default behaviour); true => delay reporting.
Throws:
InvalidContextException - If not called from a worker thread.
See Also:
report()

report

public void report()
            throws InvalidContextException
Send any pending statistics for the last completed test to the data log and the console.

Calling this does nothing if there are no pending statistics to report. This will be the case if setDelayReports(boolean) has not been called to delay reporting.

Throws:
InvalidContextException - If not called from a worker thread.

registerSummaryExpression

public void registerSummaryExpression(String displayName,
                                      String expression)
                               throws GrinderException
Register a new "summary" statistic expression. This expression will appear in the worker process output log summaries and the console.

Statistic expressions are composed of statistic names (see Statistics) in a simple post-fix format using the symbols +, -, / and *, which have their usual meanings, in conjunction with simple statistic names or sub-expressions. Precedence is controlled by grouping expressions in parentheses. For example, the error rate is (* (/ errors period) 1000) errors per second. The symbol sqrt can be used to calculate the square root of an expression.

Sample statistics, such as timedTests, must be introduced with one of sum, count, or variance, depending on the attribute of interest.

For example, the statistic expression (/ (sum timedTests) (count timedTests)) represents the mean test time in milliseconds.

Parameters:
displayName - A display name. In the console, this is converted to a key for an internationalised resource bundle look up by prefixing the string with statistic. and replacing any whitespace with underscores.
expression - The expression string.
Throws:
GrinderException - If the expression could not be registered.

registerDataLogExpression

public void registerDataLogExpression(String displayName,
                                      String expression)
                               throws GrinderException,
                                      InvalidContextException
Register a new "detail" statistic expression. This expression will appear in the worker process data log. Each test invocation will have an value displayed for the detail statistic expression.

You should call registerDataLogExpression from the top level of your script. It cannot be called from a worker thread - the data logs are initialised by the time the worker threads start.

Parameters:
displayName - A display name.
expression - The expression string.
Throws:
GrinderException - If the expression could not be registered.
InvalidContextException - If called from a worker thread.
See Also:
for details of the expression format.
Overview  Package   Class  Tree  Deprecated  Index  Help 
 PREV CLASS   NEXT CLASS FRAMES    NO FRAMES    
SUMMARY: NESTED | FIELD | CONSTR | METHOD DETAIL: FIELD | CONSTR | METHOD