Appendix B. Command Driver Manual

Table of contents

• Introduction
• Command Driver control window
• Execution from command line
• Window management
• Search keys
• Identifiers and expressions
• Command specifications
• Expressional specifications
• Repetitive task example
• Standard test scripts

Introduction

Around 2006, it became clear that OpendTect was growing so fast that extensive testing of every part was becoming a real big task. This is a well known issue in the Agile development literature. The usual approach is to implement automated testing.

When designing the implementation, the main issues we had to cope with were:

• A dynamic user interface that is defined by relations between fields. Therefore, the exact layout of windows can change considerably by small changes
• Small changes or fixes in algorithms can produce slightly different outputs for the same input
• A test system should be programmable, not just be a replay of previously run tasks

This resulted in a Command Driver based on the following principles:

• Fields are addressed by names, wildcards supported
• Operations are defined to select, fill, push, manipulate, ...
• Additional flow control operations can be used
• Scripts can call other scripts
• Variables can be set or used from the environment

ListClick "Calculated attri*" "F03-2" Double
Include "$SCRIPTSDIR$/EvalutionEnergyAttrib.odcmd"

Almost immediately we noticed that finding out the names of the fields was very hard without some tool. Therefore, we made the 'Tooltip name guide', which will put OpendTect in a state where the name of a field is shown as a tool tip (overriding any existing tool tip). When it became clear that our automated test facility would be used as a scripting system for power users, we implemented a tool to record actions.

A webinar held in March 2010 gave an extensive overview on the use of OpendTect's Command Driver plug-in. A forty-minutes flash video is available here: Command Driver webinar

The Command Driver control window

Utilities->Tools->Command Driver ...

Secondly, this so-called Command Controller also pops up when pressing Ctrl-R with the mouse pointer inside any OpendTect window or dialog. This is particularly useful if the Command Driver menu bar is greyed out or menu access is blocked after a modal dialog has been popped up. Only a few Qt-borrowed QDialogs like the QMessageBox, QFileDialog and QColorDialog will nevertheless prevent any user input to the Command Controller, and have to be clicked away first.

The Command Controller has a combobox to switch between two appearances. One for running and one for recording a command file. The TooltipNameGuide checkbox can be (un)checked at both appearances. It sets a tooltip mode that displays the hidden name of any uiObject pointed to. Tooltip names are displayed double-quoted on a cyan background. Go to the Survey->Select/Setup window for a small demo-example. Click the '(X,Y) <=> I/C' -button and unveil the hidden names of buttons and input fields with your mouse point er. OpendTect (plugin) developers might have a look in the file $WORK/src/uiIo/uiconvpos.cc to see how these names are annotated in the code.

Running the Command Driver requires selecting an input command file with the extension '.odcmd', or the obsolete '.cmd'. Filenames without a full path are defined relative to the 'Proc'-directory of the current survey. The 'Edit'-button launches a simple text editor to view and edit the input command file directly from OpendTect. One can also specify an output log file to have the Command Driver write its progress and error messages. The 'Examine'-button launches a scroll window to view this log file even while running. The Command Driver starts running when pushing the 'Go'-button, after which 'Pause' and 'Abort'-buttons allow a temporary stop or premature termination.

The Command Recorder is able to record a sequence of user interface actions performed by the user. One has to specify an output command file with the extension '.odcmd'. Filenames without a full path are defined relative to the 'Proc'-directory of the current survey. The 'Examine'-button launches a scroll window to view which commands are written during recording. Recording starts after pushing the 'Start'-button. Next perform a sequence of user actions and push the 'Finish'-button. The Command Controller will automatically switch to run-mode afterwards, so that one can play back the recorded script in order to verify its correct operation. View the log file for eventual errors. Finally, the recorded script can be edited to avoid these errors, improve robustness and general applicability, and insert auxiliary commands.

Executing scripts from the command line

One does not necessarily have to run a command script from OpendTect's user interface. One or more scripts can also run straight from the command line, and with that they might be called from shell scripts as well. An example of the command line syntax on Linux:

start_dtect cmd=/full_path_1/my_script_1.odcmd ... cmd=/full_path_N/my_script_N.odcmd

The user can also create command scripts named 'autoexec.odcmd'. If available, these scripts will be run at start-up before any script specified on the command line. The first autoexec-script to be run is searched in the settings directory. This is the '.od'-subdirectory of your personal home directory, unless the DTECT_SETTINGS or DTECT_WINSETTINGS environment variable specifies otherwise. The second autoexec-script to be run is searched in the 'Proc'-directory of the start-up survey. Next, the command line scripts will be run in order. Moreover, every time a new survey is selected in OpendTect, the 'Proc'-directory of this new survey will also be searched for an autoexec-script to be run at that moment.

Apart from the availability of autoexec-scripts, their execution can always be disabled by adding 'cmd=noautoexec', 'cmd=nosettingsautoexec' or 'cmd=nosurveyautoexec' as an extra argument on the command line. Command line argument 'cmdlog=/full_path/my_odcmdlog.txt' will override the location of the output log file. Its default name is 'odcmdlog.txt' and it is located in the 'Proc'-directory of the start-up survey, if writable, or in the personal user directory otherwise.

To run a command file other than 'autoexec.odcmd' on Windows, one has to create a DOS batch file. That file must contain a command line similar to this:

X:\full_path_0\bin\win64\od_main cmd="X:\full_path_1\my_script_1.odcmd" ...

Window management

Commands always apply to the current window. This is OpendTect Main Window when starting the driver. Any modal window popping up will automatically become the current window. Modeless windows that pop up can be appointed current window by using the Window-command. In case a modal current window closes, its parent will become the current window. If a modeless current window closes, the latest former current window still existing will be restored as current window. Subwindows in the workspace of another window cannot become current window. No commands can be addressed to the progress window popped up by a uiTaskRunner, since the driver will await its decease before executing a next command. Neither addressible are the Command Controller window and its descendants.

QDialog windows popping up are a special case. OpendTect is borrowing the QMessageBox, QFileDialog, QFontDialog, and QColorDialog from Qt. It is impossible for the command driver to manipulate the contents of these windows. The only QDialog parts that can be manipulated by the driver are those buttons resulting in closing the QDialog. One may use the standard commands Button, Ok, Cancel and Close for this aim. The special commands ColorOk, FileOk and FontOk will be available to specify the desired color, file, or font from the command script directly, meanwhile closing the QDialog.

Neither can the command driver manipulate the contents of windows popped up by batch programs that were launched from OpendTect. An example of this is the multi-machines processing window. Except for manual intervention, there is no workaround yet.

A higher form of window management is provided by the so-called window assertions. These are square-bracketed window names in between the command lines. All commands succeeding a window assertion are expected to be executed in the asserted window. If the current window does not match the latest window assertion, this results into the detection of an error. A window assertion is cancelled either by the next window assertion or by an empty assertion: [].

The benefit of window assertions is in error recovery. If OpendTect does not pop up a window that was expected by the command file or does pop up a window that was not expected by the command file, the Command Driver is able to proceed under guidance of these window assertions. This already solves a lot of cases in which the flow of OpendTect happens to be mildly data or environment dependent. For example, if an error message like "File already exist! Overwrite?" appears but is not handled in the command file (or the other way around), the Command Driver can continue as it was most probably intended by the user: close the window and write the file. Another example is any command to switch on/off a button or menu item. If it happens to be switched on/off already, window assertions enable the Command Driver to skip all commands following from that switch as well.

Although not compulsory, it is advised to make systematical use of window assertions. It also improves the readability of your command file. See any file generated by the Command Recorder to get an idea.

When using window assertions in combination with control flow commands, the most easy-going and efficacious method is having each individual command (If, ElseIf, Then, Fi, For, Rof, DoWhile, Od, Do, OdUntil) both preceded and succeeded by the window assertion in force. Wrongly omitting a window assertion around a control flow command will not result in an assertion violation error, but it does severely reduce the possibilities of the Command Driver to recover in case of other errors.

Search keys

Many commands supported by the Command Driver use search keys to address window names, names of buttons, input fields and other GUI-elements, item names of menus, lists, trees and tables, etc. The matching of these search keys is strict where necessary and accommodating where practical. The matching process honors the following principles:

The asterisk '*' is used as a wildcard character. It may appear any number of times in the search key to match an arbitrary substring.

Any sequence of white space characters like spaces, tabs, newlines, etc. will be removed at the beginning and at the end of both the match name and search key, and compressed into one single space when found in between.

The ellipses found at the end of menu item names and button texts are ignored. Any dot after the last non-white space character of the match name and search key is removed.

Underscores denoting keyboard shortcuts in menu item names, tab item names and button texts are ignored. The ampersands denoting these shortcuts in Qt-interface programming code should not appear in your search key, unless you want to match an actual ampersand of course.

If the name to match contains a symbol interpreted by the Command Driver as something special, the symbol must be preceded by the escape character '@' in your search key. Symbols for which this may apply include '*', '#', '$', '`', '"', '[' and ']', depending on the context in which the search key occurs. Escaping of symbols that do not have a special meaning is unnecessary but harmless. In the rare case that the match name happens to contain '@' characters preceding a symbol, these themselves have to be escaped in the search key. Alpha-numerical and white space characters are not considered symbols, so a single '@' character in a search key like "jaap.glas@dgbes.com" does not have to be escaped.

One example to demonstrate the use of escape characters. The table column header "* [All lines]" will be matched exactly by the search key "@* [All lines]". The asterisk must be escaped to prevent it from being interpreted as a wildcard for postfix matching, while the brackets do only need escapes when occurring inside window assertions.

Any pair of bracket-like characters enclosing a name to match are ignored, unless the search key does explicitly specify them. Outer character pairs that may be stripped include "...", [...], <...>, and >...<. This matching protocol is recursive. If the match name and search key do not match, eventual outer brackets of the match name are removed, and the whole matching process is repeated.

The Case-command allows the user to determine whether upper-case and lower-case alphabetical characters will match each other. The default is case-insensitive.

Identifiers and expressions

Command scripts may contain identifiers to store data and (re)use its values. Every identifier consists of a letter followed by any sequence of letters, digits and underscores ('_'). Identifiers are treated case insensitive. Identifiers can represent constants, script variables, environment variables, built-in function names, or user-defined procedure names.

The Command Driver has only one internal data type. All identifier values are represented as character strings. The string can be a boolean value, an integer value (even octal or hexadecimal), a fixed or floating point value, one single character or indeed a whole character string. The operator or built-in function using the identifier will define into which data type the string has to be converted, and will produce an error if the conversion fails. User-defined procedures should do the same. Boolean values are actually mapped onto numbers, where 0 is false and 1 is true, and any other number is also interpreted as true.

Predefined identifier constants include FALSE==0, TRUE==1, PI==3.14159265..., and UNDEF==1e30. Command actions may assign either SUCCESS==1, FAILURE==0, or WARNING==-1 as return value. Any 'Is...On'-question command returns ON==1, OFF==0, or UNSWITCHABLE==-1 as an answer.

Expressions are assembled from identifiers, numbers, string constants ( "..." ), parentheses, built-in functions, and about twenty mathematical and logical operators (see Expressional specifications). The assign command ( <ident> = <expr> ) stores the evaluated result of an expression into an identifier.

The value of an identifier can be substituted anywhere in the command script by putting the identifier between dollar-signs ( $<ident>$ ). Identifier substitution is the only and most flexible method to give command actions a variable meaning. Multiple substitutions in one command are allowed, but nested substitutions will be misinterpreted. Assign the result of the inner substitution to an auxiliary identifier, and use that one in the outer substitution.

In order to keep the parsing of command actions tractable, few commands will directly accept expressions as argument(s). Apart from the assign command, the only ones are control flow commands: If, ElseIf, For, DoWhile, OdUntil, Return, and any user-defined procedure call with value parameters. If you want to use an expression in an arbitrary command action, assign the expression result to an identifier and substitute its value.

Identifier substitution can also be used to simulate array variables (e.g. array[index]). The Command Driver does not explicitly support them, but index substitution can be applied for that purpose: array_$index$.

The predefined directory identifiers BASEDIR, DATADIR, PROCDIR, APPLDIR, USERDIR, SCRIPTSDIR, SNAPSHOTDIR, IMPORTDIR and EXPORTDIR are environment variables that can be substituted in file-path specifications. Their purpose is defining a file-path relatively to either OpendTect, the current survey, user directories, etc. The Command Driver will automatically change Unix-style file-paths into Windows-style file-paths on Windows-platforms and vice versa. The special identifier IDX (to be called FILEIDX after OpendTect 4.2) represents an integer variable that is automatically incremented after every occurrence in the command stream. It may be substituted in file-paths to generate unique filenames, although meanwhile more sophisticated methods exist as well (see Repetitive task example).

The scope of any identifier used in the body of a command script is global. The scope of any identifier used inside a user-defined procedure is local by default, and may shadow a global identifier with the same name. Attach the scope operator '@' in front of an identifier to force access to its global namesake. That is all for (re)assigning an identifier or (re)defining a procedure. Two extra scope rules apply for reading an identifier or calling a procedure. If an unscoped identifier value is locally undefined, the search will continue at the global scope level directly. If an unscoped procedure call is locally undefined, the search will continue at the previous scope level, and recursively descend to the global level as long as no definition is found.

Command specifications

dTect V4.0
OpendTect commands
Mon Apr 20 09:20:09 2009
!

One can make a quick start by having the first command file generated by the Command Recorder, and start editing from there.

Expressional specifications

Repetitive task example

One of the recurring user questions about OpendTect is whether there is a quick way to do some kind of repetitive task. Loading a huge amount of wells, importing multiple 2D line SEG-Y data, etc. Such a service can only be offered by OpendTect itself if the task is very simple, common and straightforward. For example, one does have the possibility to select multiple horizons with the mouse in order to load them in one go. However, if the repetitive task is more complex, unique, or variant, the workflow can be automatized by means of a command script.

The current Command Driver capabilities are demonstrated by automizing a case raised by Magnus Lidgren. His scene contained a big number of 2D lines, and a number of attributes on each line. He needed to generate a picture for every attribute of every 2D line, and wanted to avoid doing that manually.

Automizing a repetitive task consists of three stages. Firstly, the Command Recorder is applied to record the mouse and keyboard actions needed to perform the task once. Secondly, a text editor is used to modify the recorded script. Some of the recorded actions have to be generalized, and a few new commands have to be added to make the script iterative. These commands are listed in Table III. Thirdly, the Command Driver is applied to run the modified script, initially to debug it and finally to perform the repetitive task.

Listed below is the recorded command script that makes a snapshot of one attribute on one 2D line. The passages that need to be generalized have been printed in red.

[OpendTect Main Window]
TreeMenu "2D Seismics`LS 5k`i5007`Seis" "Display in a 2D Viewer as`VD"

[2D Viewer - Line: i5007]
Button "Save image"

[Create snapshot]
Button "Screen" On
Input "Select filename" "/d43/jaap/surveys/Demo2D/Snapshots/dump.png" Hold
Button "Ok"

[2D Viewer - Line: i5007]
Close

[OpendTect Main Window]

Listed below is the modified command script after it has been generalized and made iterative. All changes with regard to the originally recorded script have been printed in green.

Def res ? dumpAttribute( setnr, namenr, attrnr )
    [OpendTect Main Window]
    res ~ TreeMenu "2D Seismics`*#$setnr$`*#$namenr$`*#$attrnr$" \
                   "Display in a 2D Viewer as`VD"
    If res==FAILURE ; Return ; Fi

    pic_name = strCat( wildcard(1), "_", wildcard(2), "_" )
    pic_name = strCat( pic_name, wildcard(3), ".png" )

    [2D Viewer - Line: *]
    Button "Save image"

    [Create snapshot]
    Button "Screen" On
    Input "Select filename" "$SNAPSHOTSDIR$/$pic_name$" Hold
    Button "Ok"

    [2D Viewer - Line: *]
    Close

    [OpendTect Main Window]
Fed


For setidx = 1
    For nameidx = 1
        For attridx = 1
            res ? dumpAttribute( setidx, nameidx, attridx ) 
            If !res ; Break ; Fi
        Rof
        If attridx==1 ; Break ; Fi
    Rof
    If nameidx==1 ; Break ; Fi
Rof

• Def-inition of procedure(s) to modularize, reuse and/or hide command sequences.
• "Try"-command ( res ~ ... ) to check whether a command executes.
• Wildcards ('*') to match arbitrary substrings.
• Disambiguators ('#') to select the n-th match out of multiple matches.
• Substitution ( $...$ ) of identifier values into command actions.
• Backslash ('\') to spread a long command over multiple lines.
• If-command to execute command actions conditionally.
• Use of predefined identifiers: constant FAILURE and environment variable SNAPSHOTSDIR.
• Semicolons (';') to separate multiple commands on one line.
• Return-command to terminate a procedure immediately.
• Use of assignment and built-in function calls ( pic_name = strCat(...,...) ).
• Wildcard(.)-function to get matching strings from the latest successful wildcarded command action. Note that these strings must be secured before the next use of a wildcard ( i.e. in window assertion [2D Viewer - Line: *] ).


• (For)-loops to make a script iterative.
• Call to a user-defined procedure ( res ? dumpAttribute(...,...) )
• Break-command to escape from loops immediately.

The introduction of question commands allows the body of the previous script to be written in a different style. An identifier can take the answer to one of the many questions (Nr..., Cur..., Is..., Get...) from Table IV to VII about some property of a user-interface element. Instead of applying the Break-command to escape from loops when the procedure call to dumpAttribute(...,...) fails, now the number of items to iterate over can be asked and set before entering a loop ( nritems ? NrTreeItems ... ). Note that a procedure defined with a return parameter can also be called without. Listed below is the restyled body of the command script above.

nrsets ? NrTreeItems "2D Seismics"
For setidx = 1 To nrsets

    nrnames ? NrTreeItems "2D Seismics`*#$setidx$"
    For nameidx = 1 To nrnames

        nrattrs ? NrTreeItems "2D Seismics`*#$setidx$`*#$nameidx$"
        For attridx = 1 To nrattrs

            dumpAttribute( setidx, nameidx, attridx )
        Rof
    Rof
Rof

Standard test scripts

In the 'doc' directory of the release, you can find a 'Scripts' subdirectory. It contains the standard test scripts for OpendTect. These test scripts all work on a survey 'F3_Demo', the demo data set for OpendTect.

The directory contains several scripts, many of which can be run stand-alone, but certainly not all. There is also an execute-all script: 'AllScripts.cmd'. Another composite script is 'AllAttributes.cmd', which will make snapshots in the Snapshots directory. This Snapshots directory is created automatically; the location is your_surveys/F3_Demo/Snapshots. The snapshots are created as an index followed by the file name.

Another script is 'ExportData.cmd'. While running this script an 'Export' directory is automatically created as a subdirectory of 'F3_Demo'. All the exported data will be stored in the above directory.

Some scripts are dependent on PlugIns like SSIS, VMB etc. These scripts are located in dgb/doc/Scripts. To run these scripts you should make sure the related plugins are loaded.