Dynamic Update of Pages

Doing dyamic update of pages, especially when querying cluster resources, or non-standard MRTG targets, takes steps. The first step is to figure out which data you want to retrieve. Second, create the template to auto-generate your config-file. Third, set it into a batch-file so it can be generated as needed.

Creating the Template File

The process for creating the template file is lines out in the CfgMaker Manual found in the MRTG documentation. As with all such, it can be a bit confusing. Helpfully, they do include the code to re-create the standard MRTG cfgmaker output.

   $target_lines .= <<ECHO;
 
 Target[$target_name]: $if_ref:$router_connect
 SetEnv[$target_name]: MRTG_INT_IP="$if_ip" MRTG_INT_DESCR="$if_snmp_descr"
 ECHO
 
   if ($directory_name) {
       $target_lines .= "Directory[$target_name]: $directory_name\n";
   }
 
   $target_lines .= <<ECHO;
 MaxBytes[$target_name]: $if_speed
 Title[$target_name]: $html_desc_prefix$html_if_title_desc -- $sysname
 PageTop[$target_name]: <H1>$html_desc_prefix$html_if_title_desc -- $sysname</H1>
  <TABLE>
    <TR><TD>System:</TD>     <TD>$sysname in $html_syslocation</TD></TR>
    <TR><TD>Maintainer:</TD> <TD>$html_syscontact</TD></TR>
    <TR><TD>Description:</TD><TD>$html_if_description</TD></TR>
    <TR><TD>ifType:</TD>     <TD>$html_if_type_desc ($if_type_num)</TD></TR>
    <TR><TD>ifName:</TD>     <TD>$html_if_snmp_name</TD></TR>
 ECHO
 
   $target_lines .= <<ECHO if defined $if_port_name;
    <TR><TD>Port Name:</TD>  <TD>$if_port_name</TD></TR>
 ECHO
 
   $target_lines .= <<ECHO;
    <TR><TD>Max Speed:</TD>  <TD>$if_speed_str</TD></TR>
 ECHO
 
   $target_lines .= <<ECHO if $if_ip;
    <TR><TD>Ip:</TD>         <TD>$if_ip ($if_dns_name)</TD></TR>
 ECHO
 
   $target_lines .= <<ECHO;
  </TABLE>
 ECHO

This can be confusing, but here are a couple of tips:

If you use $router_name in place of $html_if_title_desc you can get a much more meaningful title on your pages
If you are doing a cluster, it would be a good idea to add a line like: <TR><TD>Service:</TD> <TD>$router_name</TD></TR> to better clarify what the resource actually is on the page.
For documentation reasons, changing the PageTop[x] to read: <H1>$html_desc_prefix$router_name, served by -- $sysname</H1> would also be a good idea for clusters.

If you are not doing a cluster, but are in fact tracking something else, such as CPU, similar methods can be used. The CfgMaker documentation also includes another sample of this, which I've also adopted. The problem here is that multi-CPU systems need to be handled separately. A template needs to be created for 1, 2, 3, and 4 CPU systems, and each config-file generated separately.

My Template for cluster resources

Template for 4-CPU systems:

ECHO

my $target_name = $router_name . "_cpu";
my $target_name_cpu1 = $target_name . "1";
my $target_name_cpu3 = $target_name . "3";

$target_lines .= <<ECHO;
 
YLegend[$target_name_cpu1]: CPU Usage (%)
ShortLegend[$target_name_cpu1]: Percent
Legend1[$target_name_cpu1]: CPU0 Usage (%)
Legend2[$target_name_cpu1]: CPU1 Usage (%)
LegendI[$target_name_cpu1]: CPU0Usage:&nbsp
LegendO[$target_name_cpu1]: CPU1Usage:&nbsp
MaxBytes[$target_name_cpu1]: 100
Unscaled[$target_name_cpu1]: dwmy
Options[$target_name_cpu1]: gauge,absolute,growright,nopercent,pngdate,unknaszero
Title[$target_name_cpu1]: $router_name CPU0-1 load
Target[$target_name_cpu1]: 1.3.6.1.2.1.25.3.3.1.2.1&1.3.6.1.2.1.25.3.3.1.2.2:$router_connect
PageTop[$target_name_cpu1]: <H1>CPU 0-1 usage (%) on $router_name </H1>
   <TABLE>
     <TR><TD><b>Notice</b>: CPU loads are point-in-time, not averaged</td></tr>
     <TR><TD>System:</TD><TD>$router_name in $html_syslocation</TD></TR>
     <TR><TD>Maintainer:</TD><TD>$html_syscontact</TD></TR>
     <TR><TD VALIGN="top">Description:</TD><TD>$html_sysdescr</TD></TR>
     <TR><TD>Resource:</TD><TD>CPU.</TD></TR>
   </TABLE>

YLegend[$target_name_cpu3]: CPU Usage (%)
ShortLegend[$target_name_cpu3]: Percent
Legend1[$target_name_cpu3]: CPU2 Usage (%)
Legend2[$target_name_cpu3]: CPU3 Usage (%)
LegendI[$target_name_cpu3]: CPU2Usage:&nbsp
LegendO[$target_name_cpu3]: CPU3Usage:&nbsp
MaxBytes[$target_name_cpu3]: 100
Unscaled[$target_name_cpu3]: dwmy
Options[$target_name_cpu3]: gauge,absolute,growright,nopercent,pngdate,unknaszero
Title[$target_name_cpu3]: $router_name CPU2-3 load
Target[$target_name_cpu3]: 1.3.6.1.2.1.25.3.3.1.2.3&1.3.6.1.2.1.25.3.3.1.2.4:$router_connect
PageTop[$target_name_cpu3]: <H1>CPU 2-3 usage (%) on $router_name </H1>
  <TABLE>
     <TR><TD><b>Notice</b>: CPU loads are point-in-time, not averaged</td></tr>
     <TR><TD>System:</TD><TD>$router_name in $html_syslocation</TD></TR>
     <TR><TD>Maintainer:</TD><TD>$html_syscontact</TD></TR>
     <TR><TD VALIGN="top">Description:</TD><TD>$html_sysdescr</TD></TR>
     <TR><TD>Resource:</TD><TD>CPU.</TD></TR>
   </TABLE>
ECHO

This template is special since it creates two Target entries per host queried. This is because you can only track two CPU's at a time with MRTG, and 4-CPU systems need to have two separate graphs. To do just 2-CPU's, just use the top of the two entries.

These templates use a customized Target[x] syntax in order to query the CPU-load OID's. Since you are using a template, you can also specify any Options[x] you like to use here and not on the cfgmaker command-line.

Generating the Config Files from a BatchFile

This is best done from a batchfile, as the command-line needed can get quite long.

perl cfgmaker --output=ndscpu.cfg --dns-domain=nov.youarehere.com --ifdesc name --global "workdir: d:/mrtg/data" --global "WithPeak[_]: " --if-template=cpu2-template.pl cronus mortis mars

--output	The config-file to output
--dns-domain	The DNS-domain of the targets, if they all have a common one. Can save space
--ifdesc	Which to use for the description
--global	Global Settings such as WorkDir (which needs to be defined on every file!), or other things you want to set for the whole file
--if-template	Which template to use

If you want all of your CPU queries in one file, a bit of DOS trickery can help you out here:

@type 1cpu.cfg >  cpumon.cfg
@type 2cpu.cfg >> cpumon.cfg
@type 3cpu.cfg >> cpumon.cfg
@type 4cpu.cfg >> cpumon.cfg

Which will create a single cpumon.cfg file that you can generate an index off of.

And if you are generating dynamic configs, you'll probably also have to generate a custom index using indexmaker

perl indexmaker --output=d:/mrtg/data/ndscpu.html --section=title --sectionhost --sidebyside --columns=1 --title="NDS-server CPU loads" ndscpu.cfg

--output	The name of the file to output, probably referenced by a static index page
--section	What to title the graphs
--sectionhost	Grab the target-name (remember $router_name?) and prepend to --section
--sidebyside	Place the graphs side-by-side, which I find nice to have for graphs with long names.
--columns	How many columns of graphs to have
--title	What to title the index

When plopped all together in one whonking big batch-file, you can get some really long lines. But it all works out! Set this batchfile to run at whatever frequency you need to keep it updated enough.

perl cfgmaker --output=wuf.cfg --dns-domain=wuf.someplace.com --ifdesc name --global "workdir: D:/mrtg/data/" --global "Title[^]: Served by " --global "Options[_]: growright,avgpeak,pngdate,unknaszero" --global "WithPeak[_]: wmy" --if-template=wufnic-template.pl wuf-lab1 wuf-lab2 wuf-lab3 wuf-user1 wuf-user2 wuf-user3 wuf-share1 wuf-share2 wuf-share3 wuf-class1 wuf-app1 wuf-labns wuf-labweb wuf-usrns wuf-usrweb
perl cfgmaker --output=utilnic.cfg --ifdesc name --global "workdir: D:/mrtg/data" --global "Options[_]: growright,avgpeak,pngdate,unknaszero" --global "WithPeak[_]: wmy" casper.itcs.someplace.com maint.itcm.someplace.com umbriel.itcs.someplace.com
perl cfgmaker --output=ndsnic.cfg --dns-domain=itcs.someplace.com --ifdesc name --global "workdir: D:/mrtg/data" --global "Options[_]: growright,avgpeak,pngdate,unknaszero" --global "WithPeak[_]: wmy" cronus mortis mars
perl cfgmaker --output=wufcpu2.cfg --dns-domain=wuf.someplace.com --ifdesc name --global "workdir: D:/mrtg/data" --global "WithPeak[_]: " --if-template=wufcpu2-template.pl usrsrv1 usrsrv2 usrsrv3
perl cfgmaker --output=wufcpu4.cfg --dns-domain=wuf.someplace.com --ifdesc name --global "workdir: D:/mrtg/data" --global "WithPeak[_]: " --if-template=wufcpu4-template.pl labsrv1 labsrv2 labsrv3
@type wufcpu2.cfg > wufcpu.cfg
@type wufcpu4.cfg >>wufcpu.cfg
perl cfgmaker --output=utilcpu.cfg --ifdesc name --global "workdir: D:/mrtg/data" --global "WithPeak[_]: " --if-template=wufcpu2-template.pl casper.itcs.someplace.com maint.itcm.someplace.com umbriel.itcs.someplace.com
perl cfgmaker --output=ndscpu.cfg --dns-domain=itcs.someplace.com --ifdesc name --global "workdir: D:/mrtg/data" --global "WithPeak[_]: " --if-template=wufcpu2-template.pl cronus mortis mars
perl cfgmaker --output=wufcon.cfg --dns-domain=wuf.someplace.com --ifdesc name --global "workdir: D:/mrtg/data/" --if-template=fs-connections-template.pl usrsrv1 usrsrv2 usrsrv3 labsrv1 labsrv2 labsrv3
perl cfgmaker --output=utilcon.cfg --ifdesc name --global "workdir: D:/mrtg/data/" --if-template=fs-connections-template.pl casper.itcs.someplace.com maint.itcm.someplace.com umbriel.itmcs.someplace.com
perl cfgmaker --output=ndscon.cfg --dns-domain=admcs.someplace.com --global "workdir: D:/mrtg/data/" --if-template=fs-connections-template.pl cronus mortis mars

perl indexmaker --output=D:/mrtg/data/wufnic.html  --section=title --sectionhost --sidebyside --columns=1 --title="WUF-Cluster network throughput" wuf.cfg
perl indexmaker --output=D:/mrtg/data/utilnic.html --section=title --sectionhost --sidebyside --columns=1 --title="Utility-Server network loads" utilnic.cfg
perl indexmaker --output=D:/mrtg/data/ndsnic.html  --section=title --sectionhost --sidebyside --columns=1 --title="NDS-Server network loads" ndsnic.cfg
perl indexmaker --output=D:/mrtg/data/wufcpu.html  --section=title --sectionhost --sidebyside --columns=1 --title="WUF-Cluster CPU loads" wufcpu.cfg
perl indexmaker --output=D:/mrtg/data/utilcpu.html --section=title --sectionhost --sidebyside --columns=1 --title="Utility-Server CPU loads" utilcpu.cfg
perl indexmaker --output=D:/mrtg/data/ndscpu.html  --section=title --sectionhost --sidebyside --columns=1 --title="NDS-server CPU loads" ndscpu.cfg
perl indexmaker --output=D:/mrtg/data/wufcon.html  --section=title --sectionhost --sidebyside --columns=1 --title="WUF-Server connection counts" wufcon.cfg
perl indexmaker --output=D:/mrtg/data/utilcon.html --section=title --sectionhost --sidebyside --columns=1 --title="Utility-Server connection counts" utilcon.cfg
perl indexmaker --output=D:/mrtg/data/ndscon.html  --section=title --sectionhost --sidebyside --columns=1 --title="NDS-Server connection counts" ndscon.cfg

perl indexmaker --output=D:/mrtg/data/allnic.html --section=title --sectionhost --columns=1 --title="All network traffic" wuf.cfg utilnic.cfg ndsnic.cfg
perl indexmaker --output=D:/mrtg/data/allcpu.html --section=title --sectionhost --columns=1 --title="All CPU loads" wufcpu.cfg utilcpu.cfg ndscpu.cfg
perl indexmaker --output=D:/mrtg/data/allcon.html --section=title --sectionhost --columns=1 --title="All connection counts" wufcon.cfg utilcon.cfg ndscon.cfg

The above batchfile will generate three separate types of graphs for the servers being monitored. It will then create an index for each file, and a global index for each type of resource being monitored. On a P4 3Ghz machine querying Netware servers from PII350's to P4 2.8GHz of various loads, it takes about 20 seconds to create all of the above. When MRTG queries the servers it takes under 5 seconds to get everything done.