Configuring the rrt.cfg File

The configuration file is used to specify one’s unique environment. One can setup email addresses, report formatting, search parameters, and environment info. Provided with the RRT tool is an example configuration file. It resides in the rrt/lib directory. One can copy this file for modification, or update the one provided directly. If a copy is made, be sure to use the -config file option when calling the tool. The following sections detail the options that can be controlled.

Email Notification

The following is a list of all the parameters available for configuring email notifications:

email_program

The mail program to use for email notifications. Valid entries are:

  • sendmail

  • qmail

  • smtp

If sendmail or qmail is desired, set the parameter email_program to just the name of the tool, e.g. sendmail. If smtp is desired then set the variable email_program to smtp. Also, for smtp to work the parameter email_smtp_server must be set to the domain name that the server lives on. The script test_email is provided to test the different options in order to find one that works. It has a help splash screen available with the -help option.

email_smtp_server

Domain where the smtp server lives on, e.g. mail.usgov.gov. This must be set if the selected email_program is smtp. This parameter is ignored if anything other than smtp is set.

email_domain_name

This parameter is provide as a way to append a domain name to addresses that do not have one. This typically occurs when one is using a username as the address to send to. The domain will only be appended to addresses that do not already have one.

email_addresses

When using the email notification feature, one can specify who will be sent a message by maintaining a list of addresses in the configuration file. The keyword email_addresses is used to designate that a list of addresses is to follow. Each entry should be separated by a “,”. The last entry in the list should end with a “;”. Spaces and line feeds are ignored. There should be only one instance of email_addresses. If there is more than one, the last one will be the list of addresses used.

Example 2.1. Specifying Email Addresses in Configuration File

email_addresses George.Washington@usgov.gov, Abe.Lincoln@usgov.gov,

John.Kennedy@usgov.gov, RichardNixon@usgov.gov;


Log File Extensions

A configuration file parameter is provided that allows one to specify what file extensions are being used to designate a test log file. This is a comma separated list where all the items must be contained on a single line. Note, extensions are not limited to just file types. One can specify a full and partial file names as well.

Example 2.2. Specifying Log File Extensions to Search and Parse

log_file_extensions  .log, .txt, sylver.log, _compile.log


Support for files that end ".gz" (gzip)

As of version 1.11.0 rrt is capable of parsing log files that have been compressed using the gzip tool. This is to support those verification environments that automatically compress their regression log files. Rrt will properly parse files that end in “.gz” and will store them in the rrt_logs directory. The files are stored uncompressed because web servers do not always automatically extract the files properly for viewing. When using the rrt_max_file_size parameter, it should be noted that the size of the file when it is uncompressed is the value that is used to determine if the file is too large.

RRT Environment Configuration Parameters

The following parameters in the configuration file are used to designate such things as where the web pages, test logs, and reports are stored:

master_web_dir

This is the top level directory where the web pages, reports, and test logs will be stored.

master_web_page

Name of the web home page file. This can be any web page. A template web page is provided (rrt_home.htm). The key is to have the following three lines placed in the HTML file. The results will be placed between these lines.

<!-- Regression Results Header --><PRE>
<!-- Regression Results Start -->
<!-- Regression Results End   --></PRE>

archive_web_page

Name of the archive web page. This is the page that regression results are moved to before deletion. The individual report pages are available. However, the links to the each of the log files is disabled. A template web page is provided (rrt_archived.htm).

master_web_link

This is the link string that one would enter into the browser to access the web pages minus the master_web_page name.

log_archive_path

Location where test log files will be stored. This is a path that will be placed under the master_web_dir location. Therefore, if the storage area needs to be located elsewhere, then use a symbolic link to the desired destination.

report_archive_path

Location where report files will be stored. The same restriction applies to this directory as specified in the log_archive_path parameter.

There are two methods that one can use to store and access the web pages. One is through a web server. The other is to directly point to the web home page file directly. In the first method every individual who wishes to access the information does not have to know or have access to the file system, just the web server. In direct file mode, each person who wants to access the results must be able to read from the directory location that the files are maintained.

If one is working on a system that has an Apache web server one can just copy the directory “rrt/public_html” to “~/”. Then, the web pages can be accessed via the link http://<host>/~<user>/<rrt_home_page>. If one is using a different web server, does not have a user account on the web server machine, or just does not wish to use their personal home directory, then one will have to talk to the web administrator to set up a link.

Performance Configuration Parameters

Rrt provides several parameters to reduce the time it takes to search for, parse, and store log files. The following can be defined in the configuration file.

rrt_fast_mode

Parameter that allows one to set the parsing mode that RRT uses when parsing a file. The default is to parse the entire log file. However, the more search keys one has the longer it takes for RRT to parse the file. Since in 95% (or higher) of the time the information that one is parsing for is at the beginning and end of the log file, parsing the middle of large files is needless. Therefore, the following fast mode allows one to tell RRT to only look at the top x bytes of the file as well as only the last x bytes of the file. If you need to parse the entire file then comment out the line, or set the value to 0.

rrt_max_file_size

Occasionally, in a verification environment a test will get into a mode where the simulation only ends after a watchdog timeout. This can result in an output log file that exceeds 1 Gigabyte or more. These files can be time consuming to parse and they can also use up much needed disk space because rrt will store them for access via the web interface. The rrt_max_file_size parameter will set a maximum limit that rrt will use to parse and store a file. If a file exceeds this limit, rrt will only parse and store the upper rrt_max_file_size / 2 bytes of the file. It will also parse and store the lower rrt_max_file_size / 2 portion of the file. The middle portion of the file will not be parsed and will be discarded. The parameter is specified in number of Megabytes. The default is 20 MBytes

Note

If the rrt_fast_mode parameter is defined then it determines the amount of the file that is parsed regardless if the file is too large. However, the storage rule will still apply.

include_dir_types

This is a comma separated list of directory names to include in the search for log files. The directory names do not need to be complete. The search will match partial names. For example, if all log files are stored in run_<seed> directories then set the include value to be run_. The result will be only those paths that have “run_” in their name will be searched. This parameter only applies when using the rrt –recursive command line option.

include_dir_types run_

exclude_dir_types

This parameter allows one to define a comma separated list of directory names to exclude from the search for log files. If the name appears anywhere in the path it is excluded. Note, Exclusion has precedence over inclusion using the include_dir_types parameter. This parameter only applies when using the rrt -recursive command line option.

exclude_dir_types work, src

rrt_num_not_to_archive

The rrt -archive and -delete_archive option commands are based on the date the regressions were parsed. As a project matures, regressions may stop or only be run occasionally. If a cron process has been setup and it has not been disabled the result would be that when someone comes back to review the regression results, everything is gone because the date thresholds were met.

The rrt_num_not_to_archive parameter allows one to specify a minimum number of regressions to be kept on the master web page regardless of their date-stamp. These reports will be maintained on the main page and all of the log files will still be accessible. Default: 10

rrt_reports_to_keep

This parameter specifies the number of reports to maintain on the archive web page regardless of their date-stamp. Only the reports themselves will be maintained. Any archived log files will still be removed. Default: 20

Search Strings and Web Report Fields

Special parameters are provided that can be used to format the output information that is displayed on each of the web pages. The following parameters are used in the configuration file to define the action that is to be performed on the variable that comes after.

search_key

Assigns to a defined or user specified variable the pearl search string to use to capture data. This is the method in which way one details what information one would like to retrieve from each of the test result files. The variables defined here are then used to populate the master & report fields. See Section : Predefined Variables for a list of predefined variable names.

master_field

Designates the title, content, width, and justification of a column field on the master/home web page. See the below report field example for the format.

report_field

Designates the title, content, width, and justification of a column field in the report files.

Example 2.3. Setting Up Search Keys

The following shows the assigning of Perl regular expressions to variable names. Note how these variables are used in the next example to set up the report fields.

search_key     test_name_string   "/testfn=(\w+)/"
search_key     passed_string      "/(PASSED)/"
search_key     failed_string      "/(FAILED)/"


Note

The above variables are rrt defined and are the minimum set that need search definitions in order for the tool to work properly. Any other search key variables are considered user defined and can be anything the end user wishes. Also, if the “test_name_string” is not matched in the log file then the log file is considered invalid and is removed from the report. This is to account for those environments that may have .log files that are generated by different tools (See Section : Predefined Variables for a list of predefined variables).

Example 2.4. Main Web Page Report Set-Up

Setting up the master fields. This is an example of how to set up the configuration file to display tabulated regression results data on the main web page. See Figure 1.1, “Main Report Web Page” for the subsequent output.

Field TypeKeywordColumn HeaderColWidthJustification (L/R/C)
master_fielddate_stamp"Time Stamp"116C
master_fieldlist_name"Regression List"224L
master_fieldpass_total"Passed"36R
master_fieldfail_total"Failed"46R
master_fielderror_total"Errors"56R
master_fieldtest_total"Total"65R

Example 2.5. Test Report Set-Up

Setting up the report fields. This is an example of how to set up the information that will be displayed on the individual test results page. See Figure 2.1: Regression Report Web Page for the resultant output.

Field TypeKeywordColColumn HeaderMin WidthJustification (L/R/C)
report_fieldtest_name_string1"Test name"16L
report_fielduser_defined_12"Configuration"16L
report_fieldpassed_string3"Result"6L
report_fieldfailed_string3"Result"6L
report_fielderror_string3"Result"6L
report_fieldtest_duration4"Duration"8R

Note

In the above example, the passed_string, failed_string, and error_string variables share a common column number. This overloading works because only one should be true for any given test.

Regression Report Web Page

Figure 2.1. Regression Report Web Page


Predefined Variables

RRT has a set of variables that are available for use in the output reports. The test_name, passed_string, and failed_string must be set in the configuration file by the user for the tool to work properly. The remaining variables are controlled by RRT and contain information relevant to the regression.

test_name (Required)

This variable is set for each log file that has a Perl regular expression match using the “search_key” mechanism. It is provided as a means of capturing the name of the test that results are being retrieved for. If no test name is matched then the log file is removed from the report. This is to account for those environments that may have .log files that are generated by different tools. For each match the test_total counter is incremented. This variable is typically used in the report_fields reports (Figure 2.1: Regression Report Web Page). It is suggested that the test name be located at the start of the log file. That way if the test errors out due to compile or system issues, RRT will not delete the log file because it could not find a test name

passed_string (Required)

This variable is set for each log file that has a Perl regular expression match using the “search_key” mechanism. It is provided as a way of determining whether a test ran to completion and “PASSED”. For each test that passes, the pass_total counter is incremented. This variable is typically used in the report_fields reports (Figure 2.1: Regression Report Web Page).

failed_string (Required)

This variable is set for each log file that has a Perl regular expression match using the “search_key” mechanism. It is provided as a way of determining whether a test ran to completion and “FAILED”. For each test failure the fail_total counter is incremented. This variable is typically used in the report_fields reports (Figure 2.1: Regression Report Web Page).

error_string

This string has the constant value of “ERROR”. It cannot be set via the search_key mechanism. It represents the failure case where a test_name string was found but no pass/fail string was matched. This provides a means of categorizing tests that did not run to completion, e.g. Compilation errors, segmentation faults, etc. Each time no pass/fail info is found in a log file the error_total counter is incremented. This variable is typically used in the report_fields reports (Figure 2.1: Regression Report Web Page).

date_stamp

If not set from the command line this variable represents the time that the RRT tool was called to perform its task. The default format is MM_DD_YYYY-hh.mm. It is typically used on the master report page (Figure 1.1: Main Report Web Page).

list_name

Represents the name of the list of tests that were run. It is set from the command line. The default is “Regression_list”. It is typically used on the master report page (Figure 1.1: Main Report Web Page).

pass_total

This is a counter that represents each of the passed_string matches, i.e. The tests that passed. It is typically used on the master report page (Figure 1.1: Main Report Web Page).

fail_total

This is a counter that represents each of the fail_string matches, i.e. The tests that failed. It is typically used on the master report page (Figure 1.1: Main Report Web Page).

error_total

This is a counter that represents the number of log files that did not have a pass or fail match. It is typically used on the master report page (Figure 1.1: Main Report Web Page).

test_total

This is a counter that represents each of the test_name matches, i.e. The number of tests that were run. It is typically used on the master report page (Figure 1.1: Main Report Web Page).