43  File Formats

OPALX consumes statement-oriented .in files plus model-specific field maps and particle distributions. It produces statistics, particle, field, loss, and diagnostic data through several writers.

The audited formats currently documented are:

The output inventory below separates current availability from format-documentation coverage. Complete format references remain to be written for several implemented outputs. A legacy OPAL variant is not an OPALX compatibility guarantee.

For the main .stat files, the bunch moments and RMS emittance conventions define the position, momentum-spread, correlation, and emittance columns. Beams with POLARIZATION add the spin columns listed in Spin Tracking. The full statistics schema remains to be documented.

Each completed format page will specify:

43.1 Output inventory

Default names use <base> for the input basename. Main statistics and phase-space files add _c0, _c1, and so on when multiple particle containers are present. Auxiliary files normally use the data/ directory; explicit filename controls are described in the relevant row.

Frequencies below count global tracking steps unless another event or counter is named. Available means implemented; file creation still depends on the listed controls, the selected model, and the presence of data. The final column identifies both existing documentation and remaining format-reference work.

Scroll horizontally on narrow screens to read all columns.

Output and default filename Availability Controls, defaults and timing Contents, units and format details
Bunch statistics: <base>.stat or <base>_cN.stat Available OPTION, STATDUMPFREQ=N
Default 10; 0 disables. Enabled runs also write a final record if needed.
Particle counts, bunch moments, energy, emittances and rest-frame temperatures [K]. Column headers specify units.
Statistics conventions, rest-frame temperatures and spin columns. Complete column reference pending.
Particle phase space: <base>.h5 or <base>_cN.h5 Available PSDUMPFREQ, ENABLEHDF5; optional EBDUMP and RANKDUMP
Requested interval defaults to 10; 0 disables. HDF5 is enabled by default.
Positions [m], momentum normalized by mc, IDs and step time [s]. Optional spin, rank and particle E/B samples.
Output options and spin datasets. Complete dataset/metadata reference pending.
Restart state: <base>_checkpoint.h5 Available OPTION, CHECKPOINTFREQ=N
Default 0 disables; positive N writes every N completed steps.
Versioned particle, tracker and RF state for continuation.
Checkpoint and restart.
Monitor particles: <OUTFN>.h5 or <OUTFN>.loss Available Included MONITOR; OUTFN, TYPE, ASCIIDUMP, ENABLEHDF5
Event-driven. Default output stem is the element name; default collection is spatial and output is HDF5.
Sampled positions [m], normalized momentum, IDs and times [s]; optional spin.
Monitor files and timestamps.
External-field samples: data/<FILE_NAME> for a relative name, or the supplied absolute path Available DUMPEMFIELDS
Explicit request; the configured space/time grid is written once before tracking. Existing files are overwritten.
Cartesian or cylindrical coordinates and time; B [T], E [MV/m].
Field-output command and columns.
Binning snapshots: DUMPBINSFILE (.json appended when missing) Available BINNING, DUMPBINSFILE=..., DUMPBINSFREQ=N
Filename defaults to NONE (disabled); frequency defaults to 1 when enabled.
Step, time [s], merge stage, bin counts and widths in the selected binning variable.
Binning. Complete JSON reference pending.
Dirichlet-plane potential: data/dirichletplanedumps/phi_dirichlet_legacy_step-NNNNNNNN.dat Available for unbinned explicit-image mode; single-rank output only EMISSIONSOURCE.ZEROFACEPLANEDUMP=N
Default 0 disables; positive N samples every N steps while the plane is active.
Grid indices, plane x/y [m] and potential [V]; header includes time [s] and plane position [m].
Explicit image-charge diagnostics.
Spectral tunes: <base>-tunes.csv, <base>-tune-<row>-samples.csv, <base>-tune-<row>-spectrum.csv Available RUN, SPECTRALTUNES=TRUE
Disabled by default; one analysis per TUNEINITIAL row.
Tune summary, sampled transverse signals and spectra. Units are identified by the column names.
Spectral tune files.
Closed-orbit result: exact OUTPUT filename Available COF, OUTPUT=...
No file by default; one JSON object after successful convergence when requested.
Converged section coordinates, return matrix, tunes and convergence metadata.
COF result JSON.
Reference trajectory: data/<base>_DesignPath.dat Available; rank zero outside optimization Automatic design-reference threading output
During the OrbitThreader pass, with a sampling stride derived from the reference timestep and 10 ps.
Reference path, position, momentum, fields, energy and time.
OrbitThreader. Complete output schema pending.
Element support map: data/<base>_ElementPositions.sdds Available Automatic design-reference support-map export
After threading.
Element names, types and occupied reference-path intervals.
OrbitThreader. SDDS column reference pending.
Lattice geometry: data/<base>_ElementPositions.txt, data/<base>_ElementPositions.py, data/<base>_3D.opal Available; rank zero outside optimization Automatic lattice preparation
Before particle tracking; no ENABLEVTK gate on these files.
Element placement data, a geometry helper and a spatially placed input representation.
Geometry exports. Detailed export schemas pending.
Geometry visualizations: <base>_ElementPositions.vtk, <base>_ElementPositions.html, <base>_ElementPositions.gpl Post-processing output Run the generated _ElementPositions.py with its export options
Created on request by the helper.
Visualization and plotting exports from the generated geometry helper.
Geometry exports.
Autophasing trace: data/<element-name>_AP.dat Available; rank-zero output A non-vetoed automatic RF phase search
Per search, outside optimization; vetoed and DC cases produce no trace.
Cavity-local z [m] and kinetic energy [MeV].
Autophasing outputs.
Timings: timing.dat Available Runtime information level (--info)
At normal termination; standalone COF-only inputs omit it. Information level zero suppresses its content.
Runtime and OrbitThreader timing measurements.
Performance diagnostics.
Runtime messages: stdout/stderr; conditional errormsg.txt Available Command-line verbosity and message options
During the run. The warning file is cleared at startup.
Status, warnings and diagnostic reports. errormsg.txt records certain RF-frequency warnings.
Output and logging. Redirect stdout/stderr to retain a run log.
Poisson debug fields: data/<base>-rho_scalar-NNNNNN.dat, ...-phi_scalar-..., ...-ef_vector-... Debug output; single-rank only Build with OPALX_FIELD_DEBUG=ON
Default build setting is OFF; files use a Poisson-solve counter, not a tracking-step frequency.
Grid charge density, potential and electric-field diagnostics. Complete format/unit reference pending.
Build feature flags.
Passive ring observations: configured CSV filename Internal test diagnostic for eligible bare rings Internal OPALX_TEST_PASSIVE_RING_PROBES configuration
Disabled unless configured; observations follow accepted steps.
Selected-particle observations and section-crossing diagnostics.
Passive observations. Configuration/CSV reference pending.
Charge-density HDF5: requested by RHODUMP Not available OPTION, RHODUMP=TRUE
Option defaults to FALSE; enabling it does not add a rho dataset.
No charge-density dataset is currently written.
Output options.
Particle PROBE: <OUTFN>.h5 or .loss; cyclotron-only .hist and .peaks Available for fresh cyclotron and supported RING particle tracking Included PROBE; ASCIIDUMP, ENABLEHDF5
Collective flush every 100 steps and at completion. Fresh basenames required.
Crossing records [m, normalized momentum, s] in cyclotrons and FFAs; radial summaries [mm] only for cyclotrons.
Probe interface and crossing model.
Retained statistics names: .lbal, _Monitors.stat Not available No active output control
No current data-row cadence.
No active data-row output in the current tracker.
Legacy/restart references to these names do not establish an active statistics output.

RHODUMP is not available in the current OPALX version. Enabling it does not add a charge-density dataset to the phase-space HDF5 files.

Main statistics and phase-space files are written during tracking, without an initial snapshot. A positive dump frequency also requests final output when the last step did not already produce it. Phase-space scheduling currently also depends on the statistics/status-output calls; keep those enabled when using periodic phase-space dumps. Monitor sampling is independent of these periodic frequencies. HDF5 monitor mode requires ENABLEHDF5=TRUE; select ASCIIDUMP=TRUE for ASCII monitor records when HDF5 is disabled.

A monitor .loss file contains sampled particles. The current tracker reports particle-deletion counts without writing a general file of every removed particle.

The simulation writes the _ElementPositions.py geometry helper. Running that helper separately can create the VTK, HTML and Gnuplot exports listed above in the directory where the helper is run. The .txt, .py, _3D.opal and support-map SDDS exports do not require ENABLEVTK=TRUE.

43.1.1 Lattice visualization

The geometry helper includes SCALINGFFAMAGNET bodies using their nominal arc length, radial extents, height and spiral geometry. Zero-field FFA gap arcs appear as lines in VTK exports. Passive PROBE segments appear as grey planes with a 1 m display height. That height does not limit vertical acceptance in tracking. These display representations do not alter fields, placement or tracking apertures.

For an input named case.in and the default auxiliary directory, run the generated helper from the simulation working directory:

python3 data/case_ElementPositions.py --export-vtk
python3 data/case_ElementPositions.py --export-web
python3 data/case_ElementPositions.py --show

The first two commands create case_ElementPositions.vtk and case_ElementPositions.html in the directory where the helper is invoked. Use separate commands for separate operations. VTK and web export use Python’s standard library. The interactive viewer needs NumPy and PyVista and creates or refreshes the VTK file when needed. Open the HTML export in a browser; geometry is embedded, while renderer scripts load from a public CDN.

For projected outlines:

python3 data/case_ElementPositions.py --project-to-plane --normal 0 1 0

This creates case_ElementPositions.gpl. The normal defaults to {0,1,0}; --normal nx ny nz selects another plane. The source geometry coordinates are laboratory Cartesian metres. These exports do not require ENABLEVTK=TRUE.

43.2 Spectral tune files

RUN,SPECTRALTUNES=TRUE writes one summary CSV and two detailed CSV files for each TUNEINITIAL row. The basename is the input basename used by the run. Row indices start at zero.

File Rows Columns
<base>-tunes.csv One row per TUNEINITIAL triple. energy_MeV, radius_m, pr, nu_r, nu_y, grid_spacing, legacy_turns, samples, max_relative_energy_drift
<base>-tune-<row>-samples.csv One row per retained sample for that tune row. time_s, delta_r_m, y_m, reference_angle_rad
<base>-tune-<row>-spectrum.csv One row per tune grid point. tune, radial_power, vertical_power

The summary energy is written in MeV even though BEAM.TUNEINITIAL uses GeV. pr is radial momentum in units of mc. legacy_turns is the nominal turn count used by the spectral analysis, and samples is the number of sampled signal values. Sample rows contain pre-step signal values and the accumulated reference angle used for turn-context diagnostics.

These files are rank-zero text diagnostics for the spectral tune calculation. They are independent of normal particle phase-space output.

43.3 COF result JSON

The closed-orbit command writes one JSON object when OUTPUT="orbit.json" is specified. OUTPUT is the exact filename: OPALX adds no extension and rejects an existing target or an explicitly empty filename. Omitting OUTPUT produces no COF result file. Only rank zero writes, after successful convergence; there are no text or orbit-CSV sidecars.

Let \(u=(x,p_x,y,p_y)\) be the section coordinates and \(T(u)\) the directed fixed-energy return map. Positions use metres and momenta use mechanical \(p/(mc)\), rather than slopes.

Key Shape or value Meaning
converged Boolean true for a successfully written result.
dt_s Number Configured ray timestep in seconds.
energy_MeV Number Reference kinetic energy in MeV.
coordinates Four numbers Solved section coordinates \(u\), ordered \((x,p_x,y,p_y)\).
residual Four numbers \(T(u)-u\), in the same order and units.
matrix Four rows of four numbers \(M=\partial T/\partial u\) at the solution, not the Newton matrix \(M-I\). Entry \((i,j)\) has units \(u_i/u_j\).
eigenvalues Four [real, imaginary] pairs Eigenvalues of the full coupled matrix.
stability String STABLE, UNSTABLE, NON_UNIT_CIRCLE, or MARGINAL.
near_integer Boolean Whether the eigenanalysis flags a phase near an integer.
fractional_modes Array of objects Each object contains tune and complement; both are null for an off-unit-circle mode. No integer branch or transverse-plane label is assigned.
full_tunes Object or null horizontal (horizontal/radial) and vertical tunes in oscillations per turn, including the integer branch; null when unavailable.
full_tunes_diagnostic String Explanation of the full-tune result, on success or failure.
phase_samples Nonnegative integer Sample count of the successful phase-winding traversal, including launch and final samples; 0 when unavailable.
maximum_phase_step_rad Nonnegative number Largest absolute wrapped phase increment between successive samples across both planes, in radians; 0 when unavailable.
relative_energy_drift Number Signed relative kinetic-energy change from launch to return.
time_s Number Laboratory launch time in seconds, not the return time or revolution period.
position_m Three numbers Full laboratory launch position \((X,Y,Z)\).
momentum_mc Three numbers Full laboratory launch momentum \((p_X,p_Y,p_Z)/(mc)\).
line String Selected RING name.
species String Particle species name.
mass_eV Number Rest-mass energy in eV.
charge_e Number Charge in units of the elementary charge.
section_origin_m Three numbers Laboratory position of the section origin.
section_rotation_wxyz Four numbers Unit quaternion (w,x,y,z) rotating laboratory vectors into the section frame.

A null full-tune result leaves the converged orbit and its fractional-mode diagnostics available. Zero phase_samples and maximum_phase_step_rad in that case indicate unavailable measurements. See the full-tune interface for the applicability restrictions.

The section frame is distinct from the orbit-local bunch frame used by TRACK.INITIALORBIT: the latter is centred at the solved launch and aligned with its momentum. The JSON format is experimental and has no schema-version field. It is a diagnostic export, not a checkpoint or an import interface for INITIALORBIT; handover uses the named result held in memory. Iteration details and return timing are available in the solver’s console report.

43.4 Checkpoint and restart

Enable periodic checkpoints in the input file:

OPTION, CHECKPOINTFREQ=1000;

After each 1000 completed global tracking steps, OPALX writes <input-basename>_checkpoint.h5. It first writes a same-directory .tmp file and replaces the checkpoint only after every MPI rank closes it. Each new checkpoint replaces the previous one.

Restart with the input file that defines the continuation and the saved checkpoint:

opalx --restart simulation_checkpoint.h5 simulation.in

The checkpoint file may be given either before or after the input file. For example, the following forms are equivalent:

opalx simulation.in --restart simulation_checkpoint.h5
opalx --restart simulation_checkpoint.h5 simulation.in
opalx --restart simulation_checkpoint.h5 simulation.in --info 1

If --restart is given without a checkpoint filename, OPALX derives the checkpoint name from the input filename by removing the extension and appending _checkpoint.h5. For example, opalx simulation.in --restart reads simulation_checkpoint.h5.

The input file still defines the beamline, beams, field solver, emission-source lists, and complete TRACK step-size schedule. OPALX restores the saved particle and tracker state instead of sampling the initial distributions. A restart may use a different number of MPI ranks; saved global particle tables are repartitioned over the ranks in the new run.

The checkpointed step-size segment and completed-step offset within that segment are the authoritative restart position. OPALX selects that segment directly and executes only the part of its MAXSTEPS budget that remains. This is distinct from the completed global step: an earlier segment may reach its ZSTOP without consuming all of its MAXSTEPS, so the global step alone cannot identify the current segment. The global step remains the cumulative counter used for diagnostics and checkpoint frequency.

The restart input must provide a compatible TRACK schedule. The saved segment must exist, its completed-step offset must be valid for that segment’s MAXSTEPS, and its DT must match the time step stored in the checkpoint. Changing MAXSTEPS in an earlier segment does not remap the checkpoint to a different schedule position.

The checkpoint contains:

  • Format version, completed global step, simulation time, and time step.
  • Current step-size segment and the completed-step offset within it. Restart positioning uses these values directly.
  • Every particle container’s position, momentum, per-particle time step, ID, bin, charge, mass, species, and optional spin. On restart, POLARIZATION must be set on the same beams as in the checkpointed run; each particle’s saved spin is restored, not the BEAM value.
  • Per-container path position, reference position and momentum, reference-to-lab transform, and current segment-stop state.
  • Autophased RF-cavity phases.

Checkpoint writing does not depend on phase-space output: ENABLEHDF5=FALSE and PSDUMPFREQ=0 do not disable it. On restart, OPALX rewinds the original per-container .stat files to the saved path positions and appends the continuation. Existing .lbal files also retain restart rewind support. The current tracker does not write new load-balancing rows. Existing phase-space .h5 files are likewise rewound to the saved simulation time and appended under their original names. If an original output file is unavailable, OPALX creates it and writes the restarted portion. Restart also selects the process-wide APPEND output mode, which is used by monitor, loss, design-path, and other output writers that support continued output.

Current restrictions:

  • A restart cannot execute COF, use TRACK.INITIALORBIT, or enable explicit turn-count stopping or EKINSTOP. See Tracking.
  • Time-dependent emission, delayed one-shot sources, and EMITTEDFROMFILE cannot yet be restarted because sampler progress and random state are not stored.
  • Stochastic global processes such as DECAY cannot yet be restarted because their random-pool state is not stored.
  • Iterative field-solver history is rebuilt after restart. Results remain a valid continuation, but the last bits can differ from those of an uninterrupted run or a run with a different MPI rank count because reduction order and solver initial guesses can change.
  • Checkpoint schema compatibility is explicit: an unsupported format version is rejected instead of being interpreted as a normal diagnostic HDF5 file.

OPALX reports the saved global step and time after a successful restore. A checkpoint already at the end of the configured step schedule is accepted as a completed run with no remaining integration steps.