5. On-demand monitoring

This example demonstrates on-demand monitoring. On-demand monitoring is where MonAMI will not trigger gathering information internally. Instead, something outside of MonAMI requests information. Only when metrics are requested will MonAMI acquire the data.

Another characteristic of on-demand monitoring is that one can present a large number of metrics. After MonAMI has started, the user can choose which metrics they are interested in. Typically, the user can change their selection (the user decides to start monitoring some metric, or stop monitoring it) or stop monitoring altogether. MonAMI will adjust accordingly; for example, if no users are requesting on-demand monitoring information and no internally triggered monitoring is configured, MonAMI will not gather any information. Using on-demand monitoring, one can make provision for monitoring a vast number of metrics that are normally uninteresting without burdening the monitored systems.

The example used in this section is KSysGuard. It is an excellent demonstration of the benefits of on-demand monitoring: we do not specify in the MonAMI configuration which of the available metrics is to be monitored. Rather, the user (through the KSysGuard GUI) chooses what to monitor. The user can change their mind by adding or removing metrics. When the user closes the KSysGuard application, no further monitoring is triggered.

KSysGuard is not unique in using on-demand monitoring: Nagios uses it for it's NRPE monitoring. However, KSysGuard is both easy to use and easy to configure, making it ideal for demonstration purposes.

Configuration file

The following example will include both internally triggered monitoring (as in the previous examples) and on-demand monitoring. As before, a log file is maintained that records the current status of two metrics. In addition, users are allowed to query any of the available metrics through the KSysGuard interface. KSysGuard was chosen as it is readily available and simple to understand, but the on-demand concept applies to some other monitoring systems.

As before, copy the following configuration file as /etc/monami.d/example.conf.

##  MonAMI by Example, Section 5

# Our root filesystem
 name = root-fs
 location = /
 cache = 2  ❶

# Our /home filesystem
 name = home-fs
 location = /home
 cache = 2

# Bring together all information we want KSysGuard to see
[sample]  ❷
 name = ksysguard-sample
 read = root-fs, home-fs
 cache = 10  ❸

# Also record the root and /home available space
 read = root-fs.capacity.available, \
 write = root-fs-log
 interval = 1m
 name = root-fs-log
 filename = /tmp/monami-root-fs-log

# Allow KSysGuard to request information
 read = ksysguard-sample 

Some points to note:

Each of the filesystem targets has a two second cache. This is to catch when periodic and on-demand monitoring requests happen to coincide.

The named sample aggregates information for the ksysguard target. We don't specify interval or write attributes as monitoring will be “on demand”: ksysguard target will request information as KSysGuard requires it.

The ten-second cache is included for safety: it prevents a KSysGuard user from overloading the services.

The read attribute declares from which target the datatree presented to KSysGuard is obtained.

Running the example with telnet

This section is included for those who do not have the KSysGuard GUI installed or for those who want to see some of the gory detail! If you have access to KSysGuard and don't want to type in commands in a command-line environment, feel free to move onto the next section.

To emphasise the point: people do not normally use the telnet client program for monitoring; we use it here but the program is included on almost every Unix-like platform. You are not expected to use this routinely. It is here to illustrate the idea of on-demand monitoring in environments that do not have KSysGuard installed. This hands-on approach also demonstrates that you (the user) are in control of the monitoring schedule, not MonAMI.

Start MonAMI in the usual non-detaching verbose mode: /usr/bin/monamid -fv. After starting MonAMI, and in a separate terminal, type telnet localhost 3112. This starts the telnet client so it connects to the MonAMI running on the local machine. You will see something like:

paul@donachain:~$ telnet localhost 3112
Connected to localhost.
Escape character is '^]'.
MonAMI 0.9

    -- Welcome to the ksysguard plugin for MonAMI --
This aims to be compatible with ksysguard's own ksysguardd daemon
and provide information for ksysguard to display.


The text ksysguardd> is a prompt, indicating you can enter a command. To start with, type the command MONITORS, which lists all available metrics. You will see a list of metrics, followed by the prompt.

ksysguardd> MONITORS
root-fs/fragment size   integer
root-fs/blocks/size     integer
root-fs/capacity/free   float
More metrics are reported.
home-fs/files/available integer
home-fs/flag    integer
home-fs/namemax integer

Different separator character

As with MonAMI, KSysGuard groups metrics within a tree structure. However, KSysGuard expects path elements to be seperated using the forward slash (/). The ksysguard plugin presents the data correctly for the KSysGuard GUI, so the metrics have a different separator character.

To obtain the metadata about a metric, type in the metric's full name immediately followed by a question mark. This will return a single line, containing the metric name, the minimum and maximum values for that metric (both zero if they are unknown) followed by the units. For example:

ksysguardd> home-fs/capacity/total?
home-fs/capacity/total  0       0       MiB

To obtain a metric's current value, type in the metric's full name. You can repeat this as often as you like; if correctly configured, MonAMI's caching policy will prevent queries from impacting heavily on any monitored system.

ksysguardd> home-fs/capacity/available
ksysguardd> root-fs/capacity/available

To terminate the connection, either close the telnet connection (Ctrl+] then type close and Enter), or instruct the MonAMI ksysguard plugin to disconnect (Ctrl+D).

You can run multiple connections concurrently, requesting the same or different metrics simultaneously.

Running the example with KSysGuard

Start MonAMI as previously (monamid -fv). Whilst MonAMI is running, it will listen on port 3112 for incoming connections. This is the default port to which KSysGuard will attempt to connect.

With MonAMI running, start the KSysGuard GUI: ksysguard. From the menu bar, select FileConnect host... The resulting dialogue box will request information about how to connect to MonAMI. Enter the name of the host MonAMI is running on in Host and ensure that, under Connection Type, Daemon is selected and that Port is set to 3112.

The KSysGuard connection dialogue box for connecting to a MonAMI instance.

Figure 1. KSysGuard about to connect to MonAMI.

KSysGuard and host names

The GUI will not accept localhost as the host name for MonAMI. This is because the GUI starts its own data-gathering daemon, ksysguardd, automatically. It considers localhost always to refer to this data-gathering daemon. To connect to MonAMI, use either machine's node name (run uname -n) if this is different to localhost, or its fully-qualified domain name.

Clicking on the Ok button should result in the host name appearing in the left Sensor Browser pane. If this doesn't work, check that MonAMI is running, that you entered the host name correctly and there is no firewall blocking connections. If connecting to MonAMI remotely, you must authorise the remote connection using the allow attribute (see the MonAMI User Guide or monami.conf(5) manual page for further details).

Once the hostname appears on the sensor browser, click on the + symbol (next to the hostname) to see the available sensors. If MonAMI is reporting many metrics, expanding the tree of available metrics can be a slow process: be patient.

To begin monitoring, drag and drop a metric to a display area. It is often easiest to create a new worksheet. Select FileNew worksheet... to generate a new worksheet in which you can drag-and-drop sensors.

The KSysGuard window showing the expanded list of sensors MonAMI is providing. Also shown is historic and current data for available capacity of the root and /home filesystems.

Figure 2. KSysGuard monitoring available capacity.

Wrong metric names

You may notice that some of the metrics are inappropriately labelled. For example, metrics that have a path that ends with “free” are displayed as “Free Memory”, those ending with “total” are displayed as “Total Accesses” and “used” as “Used Memory”. This is a bug in the current version of KSysGuard.

Mixed monitoring

The above configuration has MonAMI undertake interally triggered (or periodic) monitoring whilst allowing on-demand monitoring. MonAMI will record data to the root-fs-log reporting target every minute.

This file-logging of data is independent of and concurrent to any KSysGuard monitoring, except that MonAMI will honour the cache settings. Each of the two filesystem monitoring targets will cache results for two seconds, irrespective of whether the request came from KSysGuard or from the anonymous sample section.