Core Activity Parameters
Activity parameters are passed as named arguments for an activity, either on the command line or via a scenario script. On the command line, these take the form of
<paramname>=<paramvalue>
Some activity parameters are universal in that they can be used with any driver type. These parameters are recognized by NoSQLBench whether or not they are recognized by a particular driver implementation. These are called core parameters. Only core activity parameters are documented here.
NOTE:
To see what activity parameters are valid for a given activity type, see
the documentation for that activity type with
nb help <activity type>
.
When starting out, you want to familiarize yourself with these parameters. The most important ones to learn about first are driver, cycles and threads.
driver
For historic reasons, you can also use type
. They both mean the same
thing for now, but driver
is more descriptive. The type
parameter will
continue to be supported in this major version (3.x), but it will be an
error to use it in 4.x and newer.
driver=<activity type>
- default: inferred from
alias
oryaml
parameters, or unset - required: yes, unless inferred
- dynamic: no
Every activity is powered by a named ActivityType. Thus, you must set
the type
parameter. If you do not specify this parameter, it will be
inferred from a substring match against the alias and/or yaml parameters.
If there is more than one valid match for a valid type value, then you
must set the type parameter directly.
Telling nosqlbench what type of an activity will be run also determines
what other parameters are considered valid and how they will be used. So
in this way, the type parameter is actually the base parameter for any
activity. When used with scenario commands like run
or start
, an
activity of the named type will be initialized, and then further activity
parameters on the command line will be used to configure it before it is
started.
alias
alias=<alias>
- default: inferred from yaml, or 'UNSET'
- required: no
- dynamic: no
You should set the alias parameter when you have multiple activities, when you want to name metrics per-activity, or when you want to control activities via scripting.
Each activity can be given a symbolic name known as an alias. It is good practice to give all your activities an alias, since this determines the named used in logging, metrics, and even scripting control.
default value : The name of any provided YAML filename is used as the basis for the default alias. Otherwise, the activity type name is used. This is a convenience for simple test scenarios only.
threads
threads=<threads>
- default: 1
- required: no
- dynamic: yes
You should set the threads parameter when you need to ramp up a workload.
Each activity can be created with a number of threads. It is important to adjust this setting to the system types used by nosqlbench.
default value : For now, the default is simply 1. Users must be aware of this setting and adjust it to a reasonable value for their workloads.
threads=auto
: When you set threads=auto
, it will set the number of
threads to 10x the number of cores in your system. There is no distinction
here between full cores and hardware threads. This is generally a
reasonable number of threads to tap into the procesing power of a client
system.
threads=__x
: When you set threads=5x
or threads=10x
, you will set
the number of threads to some multiplier of the logical CPUs in the local
system.
NOTE:
The threads parameter will work slightly differently for activities using
the async parameter. For example, when async=500
is provided, then the
number of async operations is split between all configured threads, and
each thread will juggle a number of in-flight operations asynchronously.
Without the async parameter, threads determines the logical concurrency
level of nosqlbench in the classic 'request-per-thread' mode. Neither mode
is strictly correct, and both modes can be used for more accurate testing
depending on the constraints of your environment.
A good rule of thumb for setting threads for maximum effect is to set it relatively high, such as 10XvCPU when running synchronous workloads (when not providing the async parameter), and to 5XvCPU for all async workloads. Variation in system dynamics make it difficult to peg an ideal number, so experimentation is encouraged while you dial in your settings initially.
cycles
cycles=<cycle count>
cycles=<cycle min>..<cycle max>
- default: same as
stride
- required: no
- dynamic: no
The cycles parameter determines the starting and ending point for an activity. It determines the range of values which will act as seed values for each operation. For each cycle of the test, a statement is built from a statement template and executed as an operation.
If you do not set the cycles parameter, then it will automatically be set to the size of the sequence. The sequence is simply the length of the op sequence that is constructed from the active statements and ratios in your activity YAML.
You should set the cycles for every activity except for schema-like activities, or activities which you run just as a sanity check of active statements.
In the cycles=<cycle count>
version, the count indicates the total
number of cycles, and is equivalent to cycles=0..<cycle max>
. In both
cases, the max value is not the actual number of the last cycle. This is
because all cycle parameters define a closed-open interval. In other
words, the minimum value is either zero by default or the specified
minimum value, but the maximum value is the first value not included in
the interval. This means that you can easily stack intervals over
subsequent runs while knowing that you will cover all logical cycles
without gaps or duplicates. For example, given cycles=1000
and then
cycles=1000..2000
, and then cycles=2000..5K
, you know that all cycles
between 0 (inclusive) and 5000 (exclusive) have been specified.
stride
stride=<stride>
- default: same as op sequence length
- required: no
- dynamic: no
Usually, you don't want to provide a setting for stride, but it is still
important to understand what it does. Within nosqlbench, each time a
thread needs to allocate a set of cycles to operate on, it takes a
contiguous range of values from a shared atomic value. Thus, the stride is
the unit of micro-batching within nosqlbench. It also means that you can
use stride to optimize a workload by setting the value higher than the
default. For example if you are running a single-statement workload at a
very high rate, it doesn't make sense for threads to allocate one op at a
time from a shared atomic value. You can simply set
stride=1000
to cause (ballpark estimation) about 1000X less internal
contention.
The stride is initialized to the calculated sequence length. The sequence length is simply the number of operations in the op sequence that is planned from your active statements and their ratios.
You usually do not want to set the stride directly. If you do, make sure it is a multiple of what it would normally be set to if you need to ensure that sequences are not divided up differently. This can be important when simulating the access patterns of applications.
NOTE: When simulating multi-op access patterns in non-async mode, the stride metric can tell you how long it took for a whole group of operations to complete.
async
async=<ops>
- default: unset
- required: no
- dynamic: no
The async=<ops>
parameter puts an activity into an asynchronous dispatch
mode and configures each thread to juggle a proportion of the operations
specified. If you specify async=500 threads=10
, then each of 10 threads
will manage execution of 50 operations at a time. With async mode, a
thread will always prepare and send operations if there are fewer in
flight than it is allotted before servicing any pending responses.
Async mode also puts threads into a different sequencing behavior. When in async mode, responses from an operation may arrive in a different order than they are sent, and thus linearized operations can't be guaranteed as with the non-async mode. This means that sometimes you use want to avoid async mode when you are intentionally simulating access patterns with multiple linearized operations per user as you may see in your application.
The absence of the async parameter leaves the activity in the default non-async mode, where each thread works through a sequence of ops one operation at a time.
cyclerate
cyclerate=<cycle_per_second>
cyclerate=<cycles_per_second>,<burst_ratio>
- default: unset
- required: no
- dynamic: yes
The cyclerate parameter sets a maximum op rate for individual cycles within the activity, across the whole activity, irrespective of how many threads are active.
NOTE: The cyclerate is a rate limiter, and can thus only throttle an activity to be slower than it would otherwise run. Rate limiting is also an invasive element in a workload, and will always come at a cost. For extremely high throughput testing, consider carefully whether your testing would benefit more from concurrency-based throttling as with async or the striderate described below.
When the cyclerate parameter is provided, two additional metrics are tracked: the wait time and the response time. See the 'Reference|Timing Terms' section for more details on these metrics.
default: None. When the cyclerate parameter is not provided, an activity runs as fast as it can given how fast operations can complete.
Examples:
cyclerate=1000
- set the cycle rate limiter to 1000 ops/s and a default burst ratio of 1.1.cyclerate=1000,1.0
- same as above, but with burstrate set to 1.0 (use it or lose it, not usually desired)cyclerate=1000,1.5
- same as above, with burst rate set to 1.5 (aka 50% burst allowed)
Synonyms:
rate
targetrate
burst ratio
This is only an optional part of the cyclerate as shown in examples above. If you do not specify it when you initialize a cyclerate, then it defaults 1.1. The burst ratio is only valid as part of a rate limit and can not be specified by itself.
- default:
1.1
- dynamic: yes
The nosqlbench rate limiter provides a sliding scale between strict rate limiting and average rate limiting. The difference between them is controlled by a burst ratio parameter. When the burst ratio is 1.0 (burst up to 100% relative rate), the rate limiter acts as a strict rate limiter, disallowing faster operations from using time that was previously forfeited by prior slower operations. This is a "use it or lose it" mode that means things like GC events can steal throughput from a running client as a necessary effect of losing time in a strict timing sense.
When the burst ratio is set to higher than 1.0, faster operations may recover lost time from previously slower operations. For example, a burst ratio of 1.3 means that the rate limiter will allow bursting up to 130% of the base rate, but only until the average rate is back to 100% relative speed. This means that any valleys created in the actual op rate of the client can be converted into plateaus of throughput above the strict rate, but only at a speed that fits within (op rate * burst ratio). This allows for workloads to approximate the average target rate over time, with controllable bursting rates. This ability allows for near-strict behavior while allowing clients to still track truer to rate limit expectations, so long as the overall workload is not saturating resources.
NOTE: The default burst ratio of 1.1 makes testing results slightly more stable on average, but can also hide some short-term slow-downs in system throughput. It is set at the default to fit most tester's expectations for averaging results, but it may not be strict enough for your testing purposes. However, a strict setting of 1.0 nearly always adds cold/startup time to the result, so if you are testing for steady state, be sure to account for this across test runs.
striderate
striderate=<strides per second>
striderate=<strides per second>,<burst_ratio>
- default: unset
- required: no
- dynamic: yes
The striderate
parameter allows you to limit the start of a stride
according to some rate. This works almost exactly like the cyclerate
parameter, except that it blocks a whole group of operations from starting
instead of a single operation. The striderate can use a burst ratio just
as the cyclerate.
This sets the target rate for strides. In nosqlbench, a stride is a group of operations that are dispatched and executed together within the same thread. This is useful, for example, to emulate application behaviors in which some outside request translates to multiple internal requests. It is also a way to optimize a client runtime for more efficiency and throughput. The stride rate limiter applies to the whole activity irrespective of how many threads it has.
WARNING: When using the cyclerate an striderate options together, operations are delayed based on both rate limiters. If the relative rates are not synchronised with the side of a stride, then one rate limiter will artificially throttle the other. Thus, it usually doesn't make sense to use both of these settings in the same activity.
seq
seq=<bucket|concat|interval>
- default:
seq=bucket
- required: no
- dynamic: no
The seq=<bucket|concat|interval>
parameter determines the type of
sequencing that will be used to plan the op sequence. The op sequence is a
look-up-table that is used for each stride to pick statement forms
according to the cycle offset. It is simply the sequence of statements
from your YAML that will be executed, but in a pre-planned, and highly
efficient form.
An op sequence is planned for every activity. With the default ratio on every statement as 1, and the default bucket scheme, the basic result is that each active statement will occur once in the order specified. Once you start adding ratios to statements, the most obvious thing that you might expect wil happen: those statements will occur multiple times to meet their ratio in the op mix. You can customize the op mix further by changing the seq parameter to concat or interval.
NOTE: The op sequence is a look up table of statement templates, not individual statements or operations. Thus, the cycle still determines the uniqueness of an operation as you would expect. For example, if statement form ABC occurs 3x per sequence because you set its ratio to 3, then each of these would manifest as a distinct operation with fields determined by distinct cycle values.
There are three schemes to pick from:
bucket
This is a round robin planner which draws operations from buckets in circular fashion, removing each bucket as it is exhausted. For example, the ratios A:4, B:2, C:1 would yield the sequence A B C A B A A. The ratios A:1, B5 would yield the sequence A B B B B B.
concat
This simply takes each statement template as it occurs in order and duplicates it in place to achieve the ratio. The ratios above (A:4, B:2, C:1) would yield the sequence A A A A B B C for the concat sequencer.
interval
This is arguably the most complex sequencer. It takes each ratio as a
frequency over a unit interval of time, and apportions the associated
operation to occur evenly over that time. When two operations would be
assigned the same time, then the order of appearance establishes
precedence. In other words, statements appearing first win ties for the
same time slot. The ratios A:4 B:2 C:1 would yield the sequence A B C A A
B A. This occurs because, over the unit interval (0.0,1.0), A is assigned
the positions A: 0.0, 0.25, 0.5, 0.75
, B is assigned the
positions B: 0.0, 0.5
, and C is assigned position C: 0.0
. These
offsets are all sorted with a position-stable sort, and then the
associated ops are taken as the order.
In detail, the rendering appears
as 0.0(A), 0.0(B), 0.0(C), 0.25(A), 0.5(A), 0.5(B), 0.75(A)
, which
yields A B C A A B A
as the op sequence.
This sequencer is most useful when you want a stable ordering of operation from a rich mix of statement types, where each operations is spaced as evenly as possible over time, and where it is not important to control the cycle-by-cycle sequencing of statements.
hdr_digits
hdr_digits=3
- default:
4
- required: no
- dynamic: no
This parameter determines the number of significant digits used in all HDR histograms for metrics collected from this activity. The default of 4 allows 4 significant digits, which means up to 10000 distinct histogram buckets per named metric, per histogram interval. This does not mean that there will be 10000 distinct buckets, but it means there could be if there is significant volume and variety in the measurements.
If you are running a scenario that creates many activities, then you can
set hdr_digits=1
on some of them to save client resources.