LOCK_PROFILING(9) manual page

The LOCK_PROFILING kernel option adds support for measuring and reporting lock use and contention statistics. These statistics are collated by "acquisition point". Acquisition points are distinct places in the kernel source code (identified by source file name and line number) where a lock is acquired.

For each acquisition point, the following statistics are accumulated:

The longest time the lock was ever continuously held after being acquired at this point.
The total time the lock was held after being acquired at this point.
The total time that threads have spent waiting to acquire the lock.
The total number of non-recursive acquisitions.
The total number of times the lock was already held by another thread when this point was reached, requiring a spin or a sleep.
The total number of times another thread tried to acquire the lock while it was held after having been acquired at this point.

In addition, the average hold time and average wait time are derived from the total hold time and total wait time respectively and the number of acquisitions.

The LOCK_PROFILING kernel option also adds the following sysctl(8) variables to control and monitor the profiling code:

debug.lock.prof.enable

Enable or disable the lock profiling code. This defaults to 0 (off).

debug.lock.prof.reset

Reset the current lock profiling buffers.

debug.lock.prof.stats

The actual profiling statistics in plain text. The columns are as follows, from left to right:

max

The longest continuous hold time in microseconds.

wait_max

The longest continuous wait time in microseconds.

total

The total (accumulated) hold time in microseconds.

wait_total

The total (accumulated) wait time in microseconds.

count

The total number of acquisitions.

avg

The average hold time in microseconds, derived from the total hold time and the number of acquisitions.

wait_avg

The average wait time in microseconds, derived from the total wait time and the number of acquisitions.

cnt_hold

The number of times the lock was held and another thread attempted to acquire the lock.

cnt_lock

The number of times the lock was already held when this point was reached.

name

The name of the acquisition point, derived from the source file name and line number, followed by the name of the lock in parentheses.

debug.lock.prof.rejected

The number of acquisition points that were ignored after the table filled up.

debug.lock.prof.skipspin

Disable or enable the lock profiling code for the spin locks. This defaults to 0 (do profiling for the spin locks).

debug.lock.prof.skipcount

Do sampling approximately every N lock acquisitions.

The LOCK_PROFILING option increases the size of struct lock_object, so a kernel built with that option will not work with modules built without it.

The LOCK_PROFILING option also prevents inlining of the mutex code, which can result in a fairly severe performance penalty. This is, however, not always the case. LOCK_PROFILING can introduce a substantial performance overhead that is easily monitorable using other profiling tools, so combining profiling tools with LOCK_PROFILING is not recommended.

Measurements are made and stored in nanoseconds using nanotime(9), (on architectures without a synchronized TSC) but are presented in microseconds. This should still be sufficient for the locks one would be most interested in profiling (those that are held long and/or acquired often).

LOCK_PROFILING should generally not be used in combination with other debugging options, as the results may be strongly affected by interactions between the features. In particular, LOCK_PROFILING will report higher than normal uma(9) lock contention when run with INVARIANTS due to extra locking that occurs when INVARIANTS is present; likewise, using it in combination with WITNESS will lead to much higher lock hold times and contention in profiling output.

“	One of the advantages of using UNIX to teach an operating systems course is the sources and documentation will easily fit into a student's briefcase.	”
— John Lions

Manual Pages — LOCK_PROFILING

NAME

CONTENTS

SYNOPSIS

DESCRIPTION

SEE ALSO

HISTORY

AUTHORS

NOTES

debug.lock.prof.enable
	Enable or disable the lock profiling code. This defaults to 0 (off).
debug.lock.prof.reset
	Reset the current lock profiling buffers.
debug.lock.prof.stats
	The actual profiling statistics in plain text. The columns are as follows, from left to right:

max
	The longest continuous hold time in microseconds.
wait_max
	The longest continuous wait time in microseconds.
total
	The total (accumulated) hold time in microseconds.
wait_total
	The total (accumulated) wait time in microseconds.
count
	The total number of acquisitions.
avg
	The average hold time in microseconds, derived from the total hold time and the number of acquisitions.
wait_avg
	The average wait time in microseconds, derived from the total wait time and the number of acquisitions.
cnt_hold
	The number of times the lock was held and another thread attempted to acquire the lock.
cnt_lock
	The number of times the lock was already held when this point was reached.
name
	The name of the acquisition point, derived from the source file name and line number, followed by the name of the lock in parentheses.

debug.lock.prof.rejected
	The number of acquisition points that were ignored after the table filled up.
debug.lock.prof.skipspin
	Disable or enable the lock profiling code for the spin locks. This defaults to 0 (do profiling for the spin locks).
debug.lock.prof.skipcount
	Do sampling approximately every N lock acquisitions.