flowchart LR Input[Input file] --> Parser[OPAL parser] Parser --> Registry[OpalData registry] Registry --> Track[TrackCmd and TrackRun] Registry --> Cof[Named CofCmd] Cof --> Launch[ClosedOrbitInitialState] Launch --> Track Track --> Tracker[ParallelTracker] Tracker --> Orbit[OrbitThreader and elements] Tracker --> Bunch[PartBunch and particle containers] Bunch --> IPPL[IPPL fields and particles] IPPL --> Kokkos[Kokkos execution backend] Tracker --> Output[Statistics and diagnostics]
48 Architecture
OPALX keeps the statement-driven OPAL object model at its boundary while using IPPL and Kokkos for distributed, performance-portable particle and field work. The Space-Charge Solver Architecture chapter expands the solver-specific ownership and execution flow.
The configurator creates the command and element exemplars available to the parser. ParallelTracker coordinates the time step, while local kernels such as the Boris pusher operate on particle state. OrbitThreader determines active elements and their local transforms. IPPL provides distributed data structures and solvers; Kokkos maps kernels to CPU and GPU backends.
GPU-callable code must avoid host-only state. In particular, enclosing methods used by CUDA device lambdas need public accessibility and all data captured by device kernels must have a valid device representation.
48.1 Linear transfer-map calculation
Linear-map responsibilities are separated as follows:
| Component | Responsibility |
|---|---|
Structure/LinearTransferMap.h |
Matrix result, boundary reference states, frames and diagnostics; no tracking logic. |
Algorithms/ExternalFieldRayTracker |
Typed dispatch for Boris/LF2, fixed-step RK4 and fixed-step DOP853; shared host-side field evaluation and stage-aware support subdivision for map reference and private rays. |
Algorithms/LinearTransferMapBuilder |
Explicit Settings, frame transport, reference segmentation, private-ray launches, finite-difference/Richardson solve and ordered composition. Returns segment owners without attaching results. |
Algorithms/OrbitThreader |
Reference pass, overlap bookkeeping, invoking the builder, attaching results to runtime elements and printing the combined map. |
Algorithms/LinearTransferMap.h remains a forwarding include for compatibility. The ray calculator uses external-field interfaces, not analytic element maps or production particle kernels. The calculation is host-side and replicated on MPI ranks; stdout remains rank-zero only. See the physics description for coordinates, boundary tolerance and limitations.
The builder does not read global Options. OrbitThreader copies the option values into LinearTransferMapBuilder::Settings at construction and passes the same integration selection to its reference tracker and the builder’s private tracker. Differentiation refinement and time integration are independent settings. A future method must be explicitly added to the ray integrator’s dispatch and tested with both kinds of rays; unknown methods fail early. The RK selection is gated to design-reference map computation; disabled-map and secondary-species threading remain Boris. RungeKuttaTableau.h contains the RK coefficients, without changing TRACK’s production steppers. DOP853 currently uses only its eighth-order integration formula, not an adaptive solver or dense output.
LinearTransferMapBuilder::Result::combined includes every unique segment, including field-free gaps, and maps between transported reference frames in (x,x',y,y',zeta,delta) coordinates. COF instead uses OneTurnMap for a four-dimensional, fixed-energy return in section-frame mechanical coordinates (x,px,y,py). The two calculations share ExternalFieldRayTracker, but have different boundaries and coordinates. ClosedOrbitSolver differentiates the return map and solves its closure residual; it does not extract a transverse block from the six-dimensional combined map.
48.2 Linear Maps
flowchart TD Options[Map options] --> Threader[OrbitThreader] Threader --> Ref[Reference samples] Ref --> Builder[LinearTransferMapBuilder] Builder --> Rays[ExternalFieldRayTracker] Rays --> Fields[Element field evaluation] Builder --> Segments[Segment maps] Segments --> Product[Combined map] Product --> Elements[Attach maps to ordered element occurrences]
OrbitThreader owns the production reference pass and hands sampled reference states to LinearTransferMapBuilder. The builder constructs finite-difference segment maps with private rays and returns ordered maps plus their combined product.
48.3 Closed Orbit Finder
flowchart TD Cof[Named CofCmd] --> Setup[RING and explicit-energy BEAM validation] Setup --> Solver[ClosedOrbitSolver] Solver --> Return[OneTurnMap] Return --> Rays[ExternalFieldRayTracker] Rays --> Fields[Native element fields and apertures] Return --> Residual[Closure residual and central differences] Residual --> Solver Solver --> Eigen[LinearMapEigenAnalysis] Eigen --> State[ClosedOrbitInitialState retained by CofCmd] State --> JSON[Optional exact JSON file] State --> Track[TrackCmd and TrackRun compatibility checks] Track --> Tracker[ParallelTracker launch frame and reference]
CofCmd derives from Action. A named statement executes at its semicolon and retains an independent ClosedOrbitInitialState by value. There is no nested tracking parser or ENDCOF. findResult requires a successfully completed named command. A fresh execution clears any previous result before starting.
The command validates static magnetic elements, nominal end-to-start geometry, beam energy and charge, and the absence of global processes. It initializes one empty PartBunch collectively for element reference data, without loading distributions or solving self-fields. Monitors are skipped before their output initialization. The scalar solve runs on rank zero; result diagnostics, launch coordinates and failure status are broadcast before the command returns.
OneTurnMap localizes the directed return to the fixed section. The Newton solver uses central differences and verifies convergence with a fresh return; LinearMapEigenAnalysis then classifies the full four-dimensional derivative. No production bunch is created for trial rays. Optional JSON output is written only on rank zero.
TrackCmd copies a compatible named launch into its tracking state. TrackRun rechecks compatibility and requires one fresh, fully emitted beam with a generated distribution. ParallelTracker uses the solved laboratory reference and rigid orbit-local frame, independently of the sampled centroid. Species, mass, charge, momentum and ring name are checked, but no lattice or field fingerprint is stored. The handover rules describe the user-visible restrictions.
COF uses the selected section normal and the host ray integrator; TRACK uses its production Boris integration and a return plane normal to the launch momentum. Their timestep convergence must be established separately. For explicit TURNS, DirectedTurnCounter and track_reference::advanceInBeamline localize the final reference return by reintegration. The terminal timestep is shared collectively and used for the whole bunch. The current implementation requires one fully emitted container, no self-fields, and the supported static lattice; it does not project particle coordinates onto the return plane.
48.4 Tune Computation
flowchart TD Beam[BEAM.TUNEINITIAL triples] --> TuneRun[RUN SPECTRALTUNES] TuneRun --> Sector[TUNESECTOR launch frame] TuneRun --> Rays[Reference and displaced rays] Rays --> Tracker[ExternalFieldRayTracker] Tracker --> Samples[Radial and vertical signals] Samples --> Lomb[Lomb spectral analysis] Lomb --> Files[Summary, samples, and spectrum CSVs]
Spectral tune computation is a diagnostic mode. It tracks two serial rays per launch row in static cyclotron sector fields and writes CSV diagnostics; it does not build the normal linear-map matrices.
48.5 Ring
flowchart TD Input[RING syntax] --> LineParser[LINE list parser] LineParser --> Checks[Reject repetition, reflection, and nested RING] Checks --> Expanded[Expanded ordered members] Expanded --> Tags[Ring membership tags] Tags --> Track[TRACK directed returns] Tags --> Cof[COF fixed-energy returns]
RING is a sequence declaration. It tags expanded member occurrences with the enclosing ring name while leaving physical placement and geometric closure to the element definitions and tracking diagnostics.
48.6 CYCLOTRONSECTOR, RFCAVITY, and Trim Coils
flowchart TD Map[PSI sector map] --> Sector[CYCLOTRONSECTOR] CoilDef[TRIMCOIL definitions] --> Sector Sector --> Ring[RING] GapProfile[Single-gap profile] --> Cavity[RFCAVITY TYPE=SINGLEGAP] Cavity --> Ring Ring --> Coasting[Coasting tracking] Ring --> Accel[Accelerated tracking with EKINSTOP] Ring --> Tunes[Spectral tunes]
Cyclotron sectors provide the static mapped magnetic field. Named trim coils add radial correction fields to sectors. RFCAVITY,TYPE="SINGLEGAP" supplies the experimental radial RF kick implementation used by accelerated cyclotron rings.
48.7 OrbitThreader
flowchart TD
State[Reference state] --> Query[IndexMap active-element query]
Query --> Local[Local element frames]
Local --> Step[External field integration]
Step --> Sample[Reference sample record]
Sample --> Return{RING return?}
Return -- no --> Query
Return -- yes --> Metrics[Return length and closure residuals]
Sample --> MapBuilder[Optional linear-map builder]
OrbitThreader follows the reference through the ordered beamline, records active element intervals, detects ring return sections, and supplies the sampled reference path used by linear-map construction.