Node:Building, Next: Index, Prev: Collected Statistics, Up: Top, Contents, Top

Initialization, Building, and Scripting

Minor but useful additions have been made to two Proteus commands, install-proteus and makesim. A mechanism for run-time initialization of variables has been added (more flexible than the `.param' files but without the benefits of constant folding. An extensive scripting facility has been added which is very useful for conducting experiments on a range of Proteus configurations and programs. A small utility is also included, useful in previous and perhaps future Proteus versions.

Node:Installation, Next: Build Command, Prev: Building, Up: Building, Contents, Top

Installation Command


`install-proteus' [ ToDirectory [Version Name]]


Installs Proteus in ToDirectory, default, the current directory. Proteus is installed by copying many files to ToDirectory. (2)

The result of installing Proteus in a directory in which it is already installed is unpredictable. (Note: installing Proteus, as used here, is what users do before running simulations. It's not what system administrators do after they receive the distribution.)

Option Version Name refers to the version to install, by convention in the form `engineR'##, where ## is a version identifier. If omitted, a recent version is installed.

Node:Build Command, Next: Initialization, Prev: Installation, Up: Building, Contents, Top

Changes to the `makesim' Program

Additional options added to the `makesim' program, used to build Proteus, are described below. Refer to Proteus 3 documentation for more on `makesim'.

Include a file named `AuxMake' into the SimMake file. This file should contain additional rules for building the simulator. File `AuxMake' should be used for changes to the simulator itself, while `UserMake' should be used for programs running on the simulated system. Additional files used to build the simulator should be listed in the following variables: `auxSource' for the names of main files in a compilation unit (for example, `shmem.c', `main.c'), `auxMiddle' for the names of non-header files included by main files (for example, `shmem.calls.c'), `auxObj' for object files generated, and `auxIFiles' for header files. These file lists are used by the tags rule (for building a tags file), the depend rule (for making dependency rules), and in other places.
Compile Proteus to facilitate the debugging of engine files. In Version 3.6, this option turns off optimization (except for the file req_queue.c). The debug (`-gstabs+') compiler option is used when building Proteus whether or not the `-DdebugEngine' option is present. (So that line-number and other information is available to the snapshot routines and other code.)
Do not remove temporary files created when building the simulator. This might be used when one wants to examine cycle-counted assembler files (e.g., `aug-my_prog.s') to determine how many cycles a particular fragment will take.
`-DOPT' `-DOPT1' `-DOPT2' `-DOPT3'
Compile user code using optimization flag `-O', `-O1', `-O2', or `-O3'.

Node:Initialization, Next: Scripting, Prev: Build Command, Up: Building, Contents, Top

Run-Time Variable Initialization

Run-time initialization (RTI) files are used to initialize variables which can be used by engine and user code. Values are specified by editing the RTI file (which by default has a name with extension `.par'). The file is read at the beginning of a simulation run. Unlike Proteus' `.param' files, values in RTI files can be changed without having to re-build the simulator. RTI files can also contain comments and only need contain entries for non-default values.

Node:RTI Use, Next: RTI Declaration, Prev: Initialization, Up: Initialization, Contents, Top


Variables which are to be initialized using RTI files, called RTI variables, must be declared using the Param macro. These declarations are placed in an initialization header file. Declarations for Proteus engine code appear in initialization header file `paramProt.h'. User declarations can be placed in file `paramUser.h'. Additional header files can be added for user code by appropriately modifying `paramAll.h'.

The declaration specifies the variable type, range of valid values, and a default. Valid types are int, double, and char*. (Storage for character pointers is dynamically allocated when the value is read.) Numeric arrays are not supported. RTI variables have global scope. The contents of a typical initialization header file appears below:

Param(dataFileName, char*,"dc.ff",0,30, "Database file name.")
Param(logFileName,  char*,"dc.rpt",0,30,"Log file name.")
Param(piBins,       int,2,2,MAXINT4,    "Number of %ile bins.")
Param(rptWlist,     int,0,0,1,          "Interwindow distances in report.")
Param(rptPiList,    int,0,0,1,          "Percentiles in report.")
Param(wBins,        int,10,1,MAXINT4,   "Number of weight bins.")
Param(windowTime,   int,1000,1,MAXINT4, "Cycles per window.")

The header file is part of the Proteus source code read by the C compiler, so C syntax rules apply.

Initial values for variables are placed in the RTI file. A line of the file can be blank, have a comment (which starts with a #, or have an initialization. An initialization line starts with the name of the variable being initialized, followed by whitespace and the variable value. An unrecognized variable name will cause a fatal error. The contents of a sample RTI file appears below:

# Parameters for DC analysis.

wBins 10
piBins 100
windowTime 500000
dataFileName none

By default Proteus will look for an RTI file named `prot.par'. The -p command-line option can be used to specify an alternate file name. The file is read before usermain starts

Code that uses these variables can include the initialization header file in which they were declared (rather than declaring them separately). The file `param.h' must be included before the file containing the initializations. (The defining declarations are made with `main.c'.) For example, if the header file `paramDC.h' is used for user-code initializations, then the following might be placed in the user source-code file:

#include "param.h"
#include "paramDC.h"

int usermain(int argv,char **argc)
 int i=wBins; /* The default or value read from the RTI file. */

Node:RTI Declaration, Prev: RTI Use, Up: Initialization, Contents, Top

Declaring variables for run-time initialization.

Macro: Param( variable, type, default, min, max, doc )
Specifies that variable is to be automatically initialized. The variable will be declared as type type and will be assigned default value default. The variable is initialized with the default value. If an entry for variable is found in the RTI file it is tested against min and max, which are cast to double. For type int and double, min and max are interpreted as the inclusive boundaries of a valid range. For type char*, min and max are a range of lengths. If the entry is within the range it will replace the default value. If the entry is outside the valid range a fatal error is generated. Documentation for the variable is specified by doc which should point to a null-terminated character string.

Node:Scripting, Next: copyLink, Prev: Initialization, Up: Building, Contents, Top

Running Proteus Multiple Times

Using a testscript and the testProt (Multirun) program Proteus can be run multiple times without user intervention. The test script specifies parameters and variable initializations for each run, as well as how to save the build output, run output, and event files. A timeout can also be specified. A Perl package is provided for automatically generating test scripts, which can be used for specifying a run for each of several values of some parameter. Or all pairs of two parameters, or...

Node:Test Script, Next: Multirun Command, Prev: Scripting, Up: Scripting, Contents, Top

Test Script

The test script specifies how Proteus is to be run. In particular, parameters, variable initializations, command-line arguments and a timeout can be specified.

The test script is divided into two parts: a header and the body. The header consists of defaults which specify default settings. The body consists of one or more tests; each test consists of a number of entries. The scope of an entry is the test in which it appears.

The header starts at the beginning of the file and ends at a line containing `EndHeader'. The start of a test is indicated by a line containing `BeginTest' Test Name. The end of a test is indicated by a line containing `EndTest'. A typical test script appears below.

# Sample test script for testProt

Timeout 10
ParameterFileExt .paramT


BeginTest Should Pass
 ProteusArgs -v 
 si utilSwitch 0
 Timeout 1000
 sp Nk 2
 sp Ndim 1

Node:Test Script Commands, Next: Test Script Syntax, Prev: Test Script, Up: Test Script, Contents, Top

Test Script Commands

Test Script Default: TestMode mode
Specifies if testProt runs in test mode. If mode is 0, testProt runs normally, if mode is 1, testProt runs in test mode. This entry overrides the command-line option `-t'.

Test Script Default: ParameterFileExt ext
Specifies the parameter file extension to use. After testProt reads the header, all files matching `*.'ext will be loaded as parameter files. Parameter files for Proteus will be generated using these and the changes specified in the tests. If the entry is not present extension `.param' is used.

Test Script Default: Timeout limit
Test Script Entry: Timeout limit
Specifies the amount of time, limit seconds, that Proteus will be allowed to run. At limit seconds the Proteus process will be killed and the next test will be started.

Test Script Default: UserMake name
Test Script Entry: UserMake name
Specifies the name of the UserMake file; default `UserMake'.

Test Script Default: AutoVarInitSpecFiles files
The testProt program uses files for the names of initialization header files in which run-time initialized variables are defined. (See section Run-Time Variable Initialization.) The testProt program will read these files to determine the names, types, and valid ranges of RTI variables; this information is used to determine if the test script entries are valid. The testProt program has a limited ability to parse the initialization header files so avoid including elaborate code. (The files are passed through the C preprocessor, so that the use of defines and comments should not cause problems.

Test Script Default: InitFileName name
Test Script Entry: InitFileName name
Specifies the name of the RTI file. (The file containing variable initializations.) The name specified in a test replaces the name specified in the header; that is, the files are not merged. In contrast, individual `si' entries replace corresponding entries in the file. If no file is specified, `prot.par' is used.

Test Script Default: ProteusArgs args
Test Script Entry: ProteusArgs args
Specifies command-line arguments passed to makesim. The build is started with
makesim -S -f UserMake args

where UserMake is the specified or default usermake file.

Test Script Default: MakesimOptions args
Test Script Entry: MakesimOptions args
Specifies command-line arguments passed to Proteus. The command to start Proteus is:
proteus -abort args

Test Script Entry: si variable value
Specifies that the initialization file to be written is to contain a line of the form:
variable value

This entry overrides any similar line present in the active initialization file.

Test Script Entry: sp parameter value
Specifies that the appropriate parameter file to be written is to contain the corresponding parameter setting. For boolean parameters, value should be `true' or `false' or `on' or `off'. The name of the parameter file is automatically determined.

Node:Test Script Syntax, Prev: Test Script Commands, Up: Test Script, Contents, Top

Test Script Syntax

TestScript ==> Header Tests
Header ==> Defaults `EndHeader' RET
Defaults ==> Default Defaults
Defaults ==> 
Default ==> `ParameterFileExt' ext RET
ext ==> <Word>
Default ==> `TestMode' 0 | 1 RET
Default ==> `AutoVarInitSpecFiles' fileNames RET
Default ==> `Timeout' limit RET
Default ==>  `ProteusArgs' args RET
limit ==> <Number>
Default ==> `UserMake' fileName RET
Default ==> `InitFileName' fileName RET
Default ==> `MakesimOptions' args RET
fileNames ==> fileNames fileName RET
fileNames ==> RET
fileName ==> <Word>
Tests ==> `BeginTest' testName RET Entries `EndTest' RET
testName ==> <String_A>
Entries ==> Entry Entries
Entry ==>  `Timeout' limit RET
Entry ==>  `ProteusArgs' args RET
args ==> <String>
Entry ==>  `InitFileName' fileName RET
Entry ==> `MakesimOptions' args RET
Entry ==>  `UserMake' fileName RET
Entry ==>  `si' varName varValue RET
varName ==> <Word>
varValue ==> <String>
Entry ==> `sp' parName parValue 
parValue ==> <Word>

Item RET indicates the end of line. Item <Word> indicates a whitespace-delimited string. Item <String> specifies all characters delimited by RET, the end of line, (that is, everything up to the end of the line). Item <String_A> is the same as string, except the first character cannot be a -.

Node:Multirun Command, Next: Script Generators, Prev: Test Script, Up: Scripting, Contents, Top



`testProt' [ `-d0' | `-d1' | `-d2' ] [ `-f' script file ] [ `-bNone' | `-bBad' | `-bAll' ] [ `-B' build-output file ] [ `-rAll' | `-rBad' | `-rRun' | `-rPass' | `-rBomb' | `-rFail' | `-rNone' ] [ `-R' run-output file ] [ `-e0' | `-e1' | `-ez' ] [ `-E' event file ] [ `-t0' | `-t1' ] [ `-o00' | `-o01' | `-o10' | `-o11' [ `-D' test ... ]


Builds and runs Proteus based on a test script. The test script specifies parameters, variable initializations, timeout limits, and command-line arguments as well as how to save the build output, run output, and event files.

Before each run of Proteus changed parameters are written to `.param' files and variable initializations are written to `prot.par'. Variable initializations are written, even if they have not changed. Each parameter file is always written before the first test; only parameter files that would change are written before subsequent Proteus runs.

To avoid futile runs, testProt checks to see if the parameters and variable initializations specified in the test script are valid. If errors are encountered in entries describing a single run that run is skipped. A test script can optionally be checked but not executed.

After the files are written Proteus is built; if the build is successful Proteus is run. If Proteus runs longer than the timeout limit, it is killed. In test mode Proteus is to run a test program which writes test results to a file; testProt reads this file to determine the results of the tests. Build, run, and event output, if appropriate, is saved. The process is repeated for each run specified in the test script.

The testProt program sends one line to `stdout' for each run of Proteus, indicating a run name and the outcome. After the runs are complete a summary of the outcomes is displayed. When run in test mode the outcomes displayed are: successful builds, unsuccessful builds, partial runs (runs ending due to errors or the timeout limit), passed runs, and failed runs. When not in test mode the number of Proteus runs with a return code of zero is displayed as successful.

After testProt runs parameter and initialization files are reset as specified by the -o options. The time stamps of the restored files do not change. No attempt is made to clean up the files created during the build. (Beware, this could confuse make.)


`-f' script file
If `-f' is present, uses test script in file script file (default `TestScript').
`-d0' | `-d1' | `-d2'
Option `-d2' indicates that the test script should be read and checked for errors but that no files should be modified. Option `-d1' (as in dry run) indicates that the test script should be read, but that Proteus should not be built or run. Files will be modified as they would in a real run. Option `-d0', the default, indicates that Proteus should be run.
`-B' build-output file
`-bNone' | `-bBad' | `-bAll'
The output generated when building Proteus is saved in file build-output file (default `build.out'). Unless the `-bNone' option is specified, the file is emptied after the test script header is read. After each Proteus run, specified build outputs are appended. The self-explanatory `-b'X options specify which builds to save.
`-R' run-output file
`-rAll' | `-rBad' | `-rRun' | `-rPass' | `-rBomb' | `-rFail' | `-rNone'
The output generated when running Proteus is saved in file run-output file (default `run.out'). Unless the `-rNone' option is specified, the file is emptied after the test script header is read. After each Proteus run specified run outputs are appended. The `-r'X options specify which runs to save. The `-rBad' indicates all runs which did not finish successfully. The `-rBomb' option indicates runs which ended due to a run-time error, not including a timeout or a failed test. The other options are self-explanatory.
`-e0' | `-e1' | `-ez'
`-E' event file
If the `-e1' option is present, event files (`events.sim') are renamed to event file####, where #### is an integer. For each run of Proteus, the integer is incremented. With option `-ez' the files are renamed and compressed. If event file is not specified `events.sim' is used for event file. Event files are saved even if a run does not finish successfully.
`-t0' | `-t1'
If `-t1' is present then testProt runs in test mode; if `-t0' is present then testProt runs normally (the default). The option is overridden if a `TestMode' entry is in the test script header. In test mode the user code is expected to write the outcome of tests to a file; the file name is specified (by testProt) in the `proteus' command line. After Proteus runs, testProt reads the file to determine if the test was completed and passed. The test results are tallied and sent to `stdout' before testProt finishes.
`-o00' | `-o01' | `-o10' | `-o11'
The `-o'XY options specify whether to make backups of, and whether to restore, the parameter and initialization files. If X is 1 then backups of initialization and parameter files will be made; if X is 0 then backups will not be made. Note, backups are only made if there are no previous backups. If Y is 1 then backup copies will be restored; if Y is 0 then backup copies will not be restored. The `-o01' option is valid only when it is the only option. That is, `testProt -o01' should work but `testProt -o01' AnythingElse will not. When `testProt -o01' is issued backups made on previous testProt run will be restored, but Proteus will not be run.
`-D' test ...
If `-D' is present then only the tests named in test ... will be performed. This option can be used in conjunction with `-d1' (dry run) and `-o10' to set the parameter and initialization files for a particular test, without actually running the test. Proteus could then be run "by hand," perhaps using the debugger.

Node:Script Generators, Prev: Multirun Command, Up: Scripting, Contents, Top

Perl Functions for Generating Test Scripts

A Perl5 package, MakeScript, is provided for automatically generating test scripts. The MakeScript package is used by a user-written Perl program called a script generator which generates a test script to be read by testProt.

A script generator is useful when simulations are required for all combinations of values of multiple simulation variables. One simulation variable might specify cache size, another might specify the benchmark program, and a third might specify the type of network to use. A script generator can easily generate a script file which includes all combinations.

Node:MakeScript Concepts, Next: MakeScript Example, Prev: Script Generators, Up: Script Generators, Contents, Top

MakeScript Concepts

A script generator uses the functions provided by the MakeScript package to define one or more axes. Each axis specifies sets of values for targets. Axes are defined using the `axis_new' functions; the first argument is the axis name, the second argument describes the axis. The first column contains the targets; the other columns contain values for the targets. Each column following the first defines a plane.

In the example below, an axis named `TopologyM' is defined. The axis specifies values for four targets: a name, `<NameT>', the number of dimensions in the network, `<ndim>', the network radix, `<nk>', and the number of processors, `<procs>'.

&axis_new('TopologyM', [
      ['<NameT>','4 Proc Mesh', '16 Proc Mesh', '36 Proc Mesh'],
      ['<ndim>',  2,             2,              2,           ],
      ['<nk>',    2,             4,              6,           ],
      ['<procs>', 4,             16,             36,          ]]);

A script generator should define a template string. The template string contains the targets appearing in the axes. A typical axis appears below. (Text `EndTemplate;' is used as the string delimiter.)

BeginTest: <NameT>
 sp Nk <nk>
 sp Ndim <ndim>
 sp NO_OF_PROCESSORS <procs>
 sp NO_OF_MODULES <procs>


Tests themselves are generated using the function `&iterate'. The first argument is a list of axis names, the second argument specifies the template. The `&iterate' function iterates over all axes planes. At each iteration, the targets appearing in the template are replaced with values in the axes and the resulting string is written to stdout. See section Sample Script Generator for a complete example, including the use of `&iterate'. For a complete list of functions See section MakeScript Functions.

Node:MakeScript Example, Next: MakeScript Functions, Prev: MakeScript Concepts, Up: Script Generators, Contents, Top

Sample Script Generator

The Perl script below generates a test script containing six simulations. The script specifies simulations of systems having three different associativities and two different replacement methods.

The first part loads the necessary files.

#!/usr/local/bin/perl -w

push @INC,'/home2/koppel/protLocal/bin';  # Location of perl files. 

{ require Misc; import Misc; } # Function for formatting time.
{ require MakeScript; import MakeScript; } # Code for generating script.

The portion below defines two axes specifying cache associativity. The two axes are combined into one. (Alternately, a single axis specifying all six associativities could have been specified.) Each axis specifies the cache associativity, `<setAssoc>', number of sets, `<setSize>', and a label for the planes, `<NameC>'.

&axis_new('Cache Associativity Low', [
		['<NameC>',    '2-Way',   '4-Way',  '8-Way'],
		['<setAssoc>',	2,         4,	     8 ],
		['<setSize>',	12,        11,	     10 ]]);

&axis_new('Cache Associativity High', [
		['<NameC>',    '16-Way',   '32-Way', '64-Way'],
		['<setAssoc>',	16,         32,	      64 ],
		['<setSize>',	9,          8,	      7 ]]);

&axes_join('Cache Associativity','Cache Associativity Low'
	   ,'Cache Associativity High');

The code below associates a cache latency with each cache associativity. The latencies are specified in a new axis which is combined with the `Cache Associativity' axis to form `Cache Associativity Timing'. Perhaps because six simulations would take too long, a new axis, `Cache Associativity Timing B', containing three planes is created.

&axis_new('Assoc. Latencies', [
		['<cacheLat>', 1, 2, 3, 4, 5, 6]]);

&axes_combine('Cache Associativity Timing'
	      ,'Cache Associativity','Assoc. Latencies');

&axis_subset('Cache Associativity Timing B'
	     ,'Cache Associativity Timing','2-Way','8-Way','32-Way');

Define an axis specifying two cache replacement methods.

&axis_new('Replacement Method', [
		['<NameM>',	'Deterministic', 'Part-Random'],
		['<detR>',	'on',		 'off'],
		['<frR>',	'off',		 'on']]);

Define the header. Note that the header includes the time and the name of the script generator file (`$0'). The header is assigned to string `$header'; the string starts after `"EndH;";' and ends at the second `EndH;'.


$header= <<"EndH;";
# Automatically generated test script.
# Generated from $0 on $now.

Timeout 8000



Define a test template. Note how `<NameC>' and `<NameM>' are used to label the test.

BeginTest <NameC> <NameM> Cache Experiments Running Queens
 sp CACHE_BITS <setSize>
 sp SET_SIZE <setAssoc>


Finally, write out the test script. First the header is written. Next, the tests are generated using the `Cache Associativity Timing B' and `Replacement Method' axes.

print $header;

&iterate( ['Cache Associativity Timing B','Replacement Method'],$template);

print "\n# End of automatically generated test script.\n";

To syntax-check the script generator issue the following shell command:

[shell]% autoRun | testProt -d2 -f -

The start of the output generated by the script generator above appears below.

# Automatically generated test script.
# Generated from autoRun on 20 Nov 1995 18:21:28.

Timeout 8000


BeginTest 2-Way Deterministic Cache Experiments Running Queens
 sp SET_SIZE 2

BeginTest 2-Way Full-Random Cache Experiments Running Queens
 sp SET_SIZE 2

BeginTest 8-Way Deterministic Cache Experiments Running Queens
 sp SET_SIZE 8

The end of the test script:

BeginTest 32-Way Full-Random Cache Experiments Running Queens
 sp SET_SIZE 32

# End of automatically generated test script.

Node:MakeScript Functions, Prev: MakeScript Example, Up: Script Generators, Contents, Top

MakeScript Functions

The MakeScript functions are defined as Perl subroutines. The functions are listed below.

MakeScript: axes_combine newName oldNames
Returns the name of a new axis constructed by combining corresponding planes in each axis in oldNames. The number of targets in the resulting axis is the sum of the number of targets in each axis in oldNames. The number of planes in each axis in oldNames must be identical. If newName is null a name is automatically chosen for the new axis, otherwise the name used is newName.

MakeScript: axes_join newName oldNames
Returns the name of a new axis constructed by concatenating the planes of each axis in oldNames. The number of planes in the resulting axis will be the sum of the number of planes in each axis in oldNames. Each axis in oldNames must have the exact same set of targets. If newName is null a name is automatically chosen for the new axis, otherwise the name used is newName.

MakeScript: axis_new name axis
Defines axis to be a new axis named name, a string. If name is null a name is automatically chosen. The name of the new axis is returned. Argument axis is a reference to an array of references to arrays. (Effectively a two-dimensional array.) Each row of the array specifies a target, in the first column, and values for the target, in the remaining columns. The values in a column (except the first) are referred to as a plane. By convention, the target in the first row specifies the names of the planes.

MakeScript: axis_subset newName oldName planes
Returns the name of a new axis constructed using the planes named in planes taken from oldName. If newName is null a name is automatically chosen for the new axis, otherwise the name used is newName.

MakeScript: axis_to_string axis
Returns a string representation of axis. The string might be used for documentation or debugging.

MakeScript: iterate axes template
Argument axes, a reference to an array of strings, is used to specify the names of axes. String template is modified using values specified by each combination of planes defined by these axes; the modified string is written to stdout. The first element of axes is the outermost iterator, the last element is the innermost (most-frequently changed) iterator.

Node:copyLink, Prev: Scripting, Up: Building, Contents, Top

The `copyLink' Utility


`copyLink' file...


Each symbolic link, file, is deleted and a copy of the once-linked file is made in file if every file is a readable link. Otherwise, nothing is done.


A list of file which are not both readable and symbolic links.