OpendTect users need a lot of processing power. Therefore, chances are that they want lots of processing access to other machines. The OpendTect multi-machine batch utility provides this. It relies on remote shell execution and NFS mounting or Windows shares to do its work.
Therefore, Unix (i.e. Linux and Mac) users should have a valid $HOME/.rhosts file writeable by the user only (i.e. file permissions must be: -rw-r--r--). ; The Unix systems that need to be available for processing need to have rsh or ssh enabled.
NFS or Windows network shares are needed to make the application directory and the data directory available on all hosts that are used for processing. On Unix, in case rsh access is not possible you can use ssh, which must then be set up with public key authentication without passwords.
If on Unix machines no 'BatchHosts' file is present, OpendTect calls the system utilities of the 'hostent' (sethostent, gethostent, etc.) type to get a table of hosts for the user to select from. How the Operating System builds the lists is dependent on the particular system setup; most likely /etc/hosts and/or the NIS tables are consulted. The 'hostent' functions are old and not very good up to the task. We recommend to always create a BatchHosts file.
The program will try to use the machines that are selected when necessary. So when a job is finished on a machine, it will see which machine from the 'used machines' list is free (most likely the same machine) and start the next job on that machine. Sometimes, the machine selected has problems, is down or the network is overloaded. In any case, if the machine does not react within 5 minutes, the program considers the machine dead. It will remove the machine from the 'used machines' list, and allocates the job to the next machine that is selected or becomes free. Note: the 5 minutes timeout can be overridden using the environment variable DTECT_MM_CL_FAIL_TO, which can be used to set the desired timeout in seconds.
This leads us to the thing that needs to be checked, on Unix-based systems. NFS has an option to cache file attributes. This can cause NFS to report that a file is not there when it in fact does exist. We recommend this attribute caching is turned off, which is in fact the default on Solaris systems. On Linux, the default caching time can be as high as 30 seconds. For mounting, we therefore recommend 'noac' (Not available on Linux) and 'actimeo=0' as extra mount options for remote disks.
In terms of licensing, we encourage to tie any licensing to actual users working with it. Although we have no control on plugin builders, we advocate to never license per-cpu, per-machine and so forth. No extra licenses should be needed for processing on remote hosts. The licensing should be per actual user and therefore on the GUI level.
We assume that the survey folder(DataRoot) is on a central server and the user works from a workstation(local host) along with some other machines connected to the network which can be used as remote hosts for processing. The survey folder has to be shared and mapped on each of the host machine. Then install OpendTect on all the hosts and start the daemon service(which comes with the installation) in each of them. In case of local host machine the daemon is not needed. Create the "BatchHosts" file on the local host which contains information about remote host's IP-address/host names and the mapped survey folder corresponding to each of the machines. All these sections are explained below in details.
It is assumed that the survey folder is in a server which has to be shared and mapped on all host machines to be used for multi-machine processing. So if the survey folder is on the C drive on the server e.g. C:/Surveys then map the C drive from this machine to other host machines by choosing an available drive letter ( like C:, D: S: etc). So the survey folder now becomes S:/Surveys or F:/Surveys on each of the machines depending on drive letter that is used.
Start the daemon on every remote machine by selecting Start-menu -> programs -> OpendTect-4.2.0 -> Remote Processing Service. This tool will add a notification icon to the system tray. Once the service starts, the remote machines are ready to be used as multi-machine processing nodes. The screen-shots below shows the user interface of the Remote Processing Service and its notification icon in the system tray. The service can be controlled by right-clicking on the tray icon and selecting appropriate menu item like start, stop or exit.
As mentioned earlier, the BatchHosts file has to be created and configured so that OpendTect can communicate with the remote hosts. A BatchHosts_example file comes with OpendTect installation and could be found in the data folder. The file consists of a header section followed by information about the remote host machines. The header section contains details about the remote shell, nice-level and the port number, for Windows the port number is only used but nice-level and remote shell are Unix specific. The structure of the remote host section is like this.
IP-Address The IP address of the machine, or the name of the machine in the network.
Description more informative description which users see in the list of available hosts in Multi-Machine processing window.
Platform The platform on which the local host is running, i.e "win" if the remote host is a Windows machine.
Survey folder The full path of the survey folder along with the mapped drive. Here ';' is used in place of ':' for drives, as ':' is already used as the field separator.
Please note that the local host is automatically added to the list, so there is no need to add the IP address of the local host here.
Multi-Machine processing feature could be used from several utilities in OpendTect like creating seismic/attribute output or creating a steering cube. All these utilities have a "Multiple machines" options which will launch the Multi-machine Processing window. Now on the left of this window a list box will contain all the available hosts based on the BatchHosts file, select any of the machines from this list and start processing. The annotated screen-shot describes the fields and their purpose.
