Output File Description (e)
An output file named output is generated with every execution of the simulator. Examples of post-processing of the output file are provided on the Output File Post-processing page. The output file has the following structure:
Input Record
The output file begins with the input record, which documents the simulation input.
An input record is included on the output file to document the simulation in the event that the input file is destroyed or becomes separated from the output file. The input record is formatted differently than an input file, but contains all of the information listed in an input file. Optional cards which are not included in the input file are noted in the input record. If the simulator identifies an input error, an error message will appear in the input record at the point the error was noted. Input cards are read by the simulator in a specific order and will appear in the input record in that order. Because input cards can be organized randomly in the input file, the card order on the input record may not match that in the input file.
Reference Node Output
The reference node output record follows the input record.
The reference node output record is a table of simulation data and selected reference node variables, which are printed according to the frequency requested in the Output Control Card with the input item Reference Node Output File Frequency. A table column header is printed to delineate the columns every 10 print records (1). A link to the list of definitions of the header abbreviations for the reference node output are provided for each operational mode on the Output Control Card options page. Each print record will show simulation data and reference node variable data for each reference node requested. Simulation data comprises the time step (2), the reference node number (3), the simulation time (4), the simulation time step (5), and the number of Newton-Raphson iterations required to reach convergence followed by the number of iterations in the sub-cycled transport step (6). Reference node variable data comprise values of the variables specified in the Output Control Card through the Reference Node Variable inputs (7).
Reference nodes are indexed by node number as shown as (7) in the following graphic.
Column headers for reference node variables are delineated with an abbreviated title and associated units. Units are enclosed in brackets below the variable abbreviation (8). Definitions of the reference node header abbreviations for the output file are given here. A portion of the reference node output record for an unsaturated flow and transport problem involving 1000 nodes for time steps 10 and 11 appears as:
where the reference nodes are nodes 1, 155, 255, 260, 261, 262, 300 and 500, and the reference node variable data includes the aqueous courant number, the solute volumetric concentration, aqueous concentration, mole fraction, inventory, integrated mass, integrated solute on an aqueous basis, and the source integral (8).
Output Differences on Multiple Processors
It is possible that when eSTOMP is executed on the same number of processors, that a slightly different answer results. This may occur because inner products and norms in PETSc are computed using the MPI_Allreduce() command. In different runs the order at which values arrive at a given process (via MPI) can be in a different order, thus the order in which some floating point arithmetic operations are performed will be different. Since floating point arithmetic arithmetic is not associative, the computed quantity may be (slightly) different. Over a run the many slight differences in the inner products and norms will effect all the computed results. It is important to realize that none of the computed answers are any less right or wrong (in fact the sequential computation is no more right then the parallel ones), they are all equally valid.
It is also possible that when eSTOMP is executed on a different number of processors, that a slight different answer results. The convergence of many of the preconditioners in PETSc including the the default parallel preconditioner block Jacobi depends on the number of processes. The more processes the (slightly) slower convergence it has. This is the nature of iterative solvers, the more parallelism means the more "older" information is used in the solution process hence slower convergence.
Newton-Raphson Convergence Failures
Convergence failures and subsequent time step reductions are also noted within the reference node output record. Three types of convergence error messages may appear within the output. The first type indicates that a converged solution was not obtained within the maximum number of Newton-Raphson iterations. An example of this type of convergence error for eSTOMP-W is shown below:
Here the convergence failure and the maximum residual for the water equation are reported (7.0184E-02) at Node 214, which exceeded the convergence criterion. After this convergence failure, the time step is reduced from 3.1688E-08 yr to 6.3376E-09 yr and then repeated. Convergence errors result in a reduction in the time step with a repeated attempt to solve the system of governing equations. Four successive convergence failures without an intermediate successfully converged time step results in termination of the execution.
ECKEChem Convergence Errors
A second type of convergence failure can occur when using the ECKEChem (reactive geochemistry) module. The module will report the failure, the node at which the failure occurred, and then create a smaller sub-time step to achieve convergence. A maximum of 16 sub-time steps is attempted before the simulation will abort due to non-convergence.
PETSc Solver Errors
A third type of convergence errors are solver errors, which are used in many different places in the PETSc source code. The PETSc error handler takes a generic error code. The generic error codes are defined in include/petscerror. The same generic error code would be used many times in the libraries. For example the generic error code PETSC_ERR_MEM is used whenever requested memory allocation is not available. All indicate that the linear system solver has failed to reach a solution. An example of a KSP solver error is shown below:
Here the PETSc solver terminated execution after several successive attempts at solving the system of equations failed. The user is referred to the PETSc manual for additional information on solver error codes (http://www.mcs.anl.gov/research/projects/petsc/documentation/index.html).
Output File Termination
