1. Core

sed.core module easy access APIs

class sed.core.SedProcessor(metadata=None, config=None, dataframe=None, files=None, folder=None, runs=None, collect_metadata=False, **kwds)

Bases: object

Processor class of sed. Contains wrapper functions defining a work flow for data correction, calibration and binning.

Parameters:

• metadata (dict, optional) – Dict of external Metadata. Defaults to None.

• config (Union[dict, str], optional) – Config dictionary or config file name. Defaults to None.

• dataframe (Union[pd.DataFrame, ddf.DataFrame], optional) – dataframe to load into the class. Defaults to None.

• files (List[str], optional) – List of files to pass to the loader defined in the config. Defaults to None.

• folder (str, optional) – Folder containing files to pass to the loader defined in the config. Defaults to None.

• collect_metadata (bool) – Option to collect metadata from files. Defaults to False.

• **kwds – Keyword arguments passed to the reader.

• runs (typing.Optional[typing.Sequence[str]], default: None) –

property dataframe: DataFrame | DataFrame

Accessor to the underlying dataframe.

Returns:

Dataframe object.

Return type:

Union[pd.DataFrame, ddf.DataFrame]

property timed_dataframe: DataFrame | DataFrame

Accessor to the underlying timed_dataframe.

Returns:

Timed Dataframe object.

Return type:

Union[pd.DataFrame, ddf.DataFrame]

property attributes: dict

Accessor to the metadata dict.

Returns:

The metadata dict.

Return type:

dict

add_attribute(attributes, name, **kwds)

Function to add element to the attributes dict.

Parameters:

• attributes (dict) – The attributes dictionary object to add.

• name (str) – Key under which to add the dictionary to the attributes.

property config: Dict[Any, Any]

Getter attribute for the config dictionary

Returns:

The config dictionary.

Return type:

Dict

property files: List[str]

Getter attribute for the list of files

Returns:

The list of loaded files

Return type:

List[str]

property binned: DataArray

Getter attribute for the binned data array

Returns:

The binned data array

Return type:

xr.DataArray

property normalized: DataArray

Getter attribute for the normalized data array

Returns:

The normalized data array

Return type:

xr.DataArray

property normalization_histogram: DataArray

Getter attribute for the normalization histogram

Returns:

The normalizazion histogram

Return type:

xr.DataArray

cpy(path)

Function to mirror a list of files or a folder from a network drive to a local storage. Returns either the original or the copied path to the given path. The option to use this functionality is set by config[“core”][“use_copy_tool”].

Parameters:

path (Union[str, List[str]]) – Source path or path list.

Returns:

Source or destination path or path list.

Return type:

Union[str, List[str]]

load(dataframe=None, metadata=None, files=None, folder=None, runs=None, collect_metadata=False, **kwds)

Load tabular data of single events into the dataframe object in the class.

Parameters:

• dataframe (Union[pd.DataFrame, ddf.DataFrame], optional) – data in tabular format. Accepts anything which can be interpreted by pd.DataFrame as an input. Defaults to None.

• metadata (dict, optional) – Dict of external Metadata. Defaults to None.

• files (List[str], optional) – List of file paths to pass to the loader. Defaults to None.

• runs (Sequence[str], optional) – List of run identifiers to pass to the loader. Defaults to None.

• folder (str, optional) – Folder path to pass to the loader. Defaults to None.

• collect_metadata (bool, default: False) –

Raises:

ValueError – Raised if no valid input is provided.

bin_and_load_momentum_calibration(df_partitions=100, axes=None, bins=None, ranges=None, plane=0, width=5, apply=False, **kwds)

1st step of momentum correction work flow. Function to do an initial binning of the dataframe loaded to the class, slice a plane from it using an interactive view, and load it into the momentum corrector class.

Parameters:

• df_partitions (int, optional) – Number of dataframe partitions to use for the initial binning. Defaults to 100.

• axes (List[str], optional) – Axes to bin. Defaults to config[“momentum”][“axes”].

• bins (List[int], optional) – Bin numbers to use for binning. Defaults to config[“momentum”][“bins”].

• ranges (List[Tuple], optional) – Ranges to use for binning. Defaults to config[“momentum”][“ranges”].

• plane (int, optional) – Initial value for the plane slider. Defaults to 0.

• width (int, optional) – Initial value for the width slider. Defaults to 5.

• apply (bool, optional) – Option to directly apply the values and select the slice. Defaults to False.

• **kwds – Keyword argument passed to the pre_binning function.

define_features(features=None, rotation_symmetry=6, auto_detect=False, include_center=True, apply=False, **kwds)

2. Step of the distortion correction workflow: Define feature points in momentum space. They can be either manually selected using a GUI tool, be ptovided as list of feature points, or auto-generated using a feature-detection algorithm.

Parameters:

• features (np.ndarray, optional) – np.ndarray of features. Defaults to None.

• rotation_symmetry (int, optional) – Number of rotational symmetry axes. Defaults to 6.

• auto_detect (bool, optional) – Whether to auto-detect the features. Defaults to False.

• include_center (bool, optional) – Option to include a point at the center in the feature list. Defaults to True.

• ***kwds – Keyword arguments for MomentumCorrector.feature_extract() and MomentumCorrector.feature_select()

• apply (bool, default: False) –

generate_splinewarp(use_center=None, **kwds)

3. Step of the distortion correction workflow: Generate the correction function restoring the symmetry in the image using a splinewarp algortihm.

Parameters:

• use_center (bool, optional) – Option to use the position of the center point in the correction. Default is read from config, or set to True.

• **kwds – Keyword arguments for MomentumCorrector.spline_warp_estimate().

save_splinewarp(filename=None, overwrite=False)

Save the generated spline-warp parameters to the folder config file.

Parameters:

• filename (str, optional) – Filename of the config dictionary to save to. Defaults to “sed_config.yaml” in the current folder.

• overwrite (bool, optional) – Option to overwrite the present dictionary. Defaults to False.

pose_adjustment(scale=1, xtrans=0, ytrans=0, angle=0, apply=False, use_correction=True, reset=True)

3. step of the distortion correction workflow: Generate an interactive panel to adjust affine transformations that are applied to the image. Applies first a scaling, next an x/y translation, and last a rotation around the center of the image.

Parameters:

• scale (float, optional) – Initial value of the scaling slider. Defaults to 1.

• xtrans (float, optional) – Initial value of the xtrans slider. Defaults to 0.

• ytrans (float, optional) – Initial value of the ytrans slider. Defaults to 0.

• angle (float, optional) – Initial value of the angle slider. Defaults to 0.

• apply (bool, optional) – Option to directly apply the provided transformations. Defaults to False.

• use_correction (bool, option) – Whether to use the spline warp correction or not. Defaults to True.

• reset (bool, optional) – Option to reset the correction before transformation. Defaults to True.

apply_momentum_correction(preview=False)

Applies the distortion correction and pose adjustment (optional) to the dataframe.

Parameters:

• rdeform_field (np.ndarray, optional) – Row deformation field. Defaults to None.

• cdeform_field (np.ndarray, optional) – Column deformation field. Defaults to None.

• inv_dfield (np.ndarray, optional) – Inverse deformation field. Defaults to None.

• preview (bool) – Option to preview the first elements of the data frame.

calibrate_momentum_axes(point_a=None, point_b=None, k_distance=None, k_coord_a=None, k_coord_b=array([0., 0.]), equiscale=True, apply=False)

1. step of the momentum calibration workflow. Calibrate momentum axes using either provided pixel coordinates of a high-symmetry point and its distance to the BZ center, or the k-coordinates of two points in the BZ (depending on the equiscale option). Opens an interactive panel for selecting the points.

Parameters:

• point_a (Union[np.ndarray, List[int]]) – Pixel coordinates of the first point used for momentum calibration.

• point_b (Union[np.ndarray, List[int]], optional) – Pixel coordinates of the second point used for momentum calibration. Defaults to config[“momentum”][“center_pixel”].

• k_distance (float, optional) – Momentum distance between point a and b. Needs to be provided if no specific k-koordinates for the two points are given. Defaults to None.

• k_coord_a (Union[np.ndarray, List[float]], optional) – Momentum coordinate of the first point used for calibration. Used if equiscale is False. Defaults to None.

• k_coord_b (Union[np.ndarray, List[float]], optional) – Momentum coordinate of the second point used for calibration. Defaults to [0.0, 0.0].

• equiscale (bool, optional) – Option to apply different scales to kx and ky. If True, the distance between points a and b, and the absolute position of point a are used for defining the scale. If False, the scale is calculated from the k-positions of both points a and b. Defaults to True.

• apply (bool, optional) – Option to directly store the momentum calibration in the class. Defaults to False.

save_momentum_calibration(filename=None, overwrite=False)

Save the generated momentum calibration parameters to the folder config file.

Parameters:

• filename (str, optional) – Filename of the config dictionary to save to. Defaults to “sed_config.yaml” in the current folder.

• overwrite (bool, optional) – Option to overwrite the present dictionary. Defaults to False.

apply_momentum_calibration(calibration=None, preview=False)

2. step of the momentum calibration work flow: Apply the momentum calibration stored in the class to the dataframe. If corrected X/Y axis exist, these are used.

Parameters:

• calibration (dict, optional) – Optional dictionary with calibration data to use. Defaults to None.

• preview (bool) – Option to preview the first elements of the data frame.

adjust_energy_correction(correction_type=None, amplitude=None, center=None, apply=False, **kwds)

1. step of the energy crrection workflow: Opens an interactive plot to adjust the parameters for the TOF/energy correction. Also pre-bins the data if they are not present yet.

Parameters:

• correction_type (str, optional) –

  Type of correction to apply to the TOF axis. Valid values are:

  • ’spherical’

  • ’Lorentzian’

  • ’Gaussian’

  • ’Lorentzian_asymmetric’

  Defaults to config[“energy”][“correction_type”].

• amplitude (float, optional) – Amplitude of the correction. Defaults to config[“energy”][“correction”][“amplitude”].

• center (Tuple[float, float], optional) – Center X/Y coordinates for the correction. Defaults to config[“energy”][“correction”][“center”].

• apply (bool, optional) – Option to directly apply the provided or default correction parameters. Defaults to False.

save_energy_correction(filename=None, overwrite=False)

Save the generated energy correction parameters to the folder config file.

Parameters:

• filename (str, optional) – Filename of the config dictionary to save to. Defaults to “sed_config.yaml” in the current folder.

• overwrite (bool, optional) – Option to overwrite the present dictionary. Defaults to False.

apply_energy_correction(correction=None, preview=False, **kwds)

2. step of the energy correction workflow: Apply the enery correction parameters stored in the class to the dataframe.

Parameters:

• correction (dict, optional) – Dictionary containing the correction parameters. Defaults to config[“energy”][“calibration”].

• preview (bool) – Option to preview the first elements of the data frame.

• **kwds – Keyword args passed to EnergyCalibrator.apply_energy_correction.

• preview – Option to preview the first elements of the data frame.

• **kwds – Keyword args passed to EnergyCalibrator.apply_energy_correction.

load_bias_series(binned_data=None, data_files=None, axes=None, bins=None, ranges=None, biases=None, bias_key=None, normalize=None, span=None, order=None)

1. step of the energy calibration workflow: Load and bin data from single-event files, or load binned bias/TOF traces.

Parameters:

• binned_data (Union[xr.DataArray, Tuple[np.ndarray, np.ndarray, np.ndarray]], optional) – Binned data If provided as DataArray, Needs to contain dimensions config[“dataframe”][“tof_column”] and config[“dataframe”][“bias_column”]. If provided as tuple, needs to contain elements tof, biases, traces.

• data_files (List[str], optional) – list of file paths to bin

• axes (List[str], optional) – bin axes. Defaults to config[“dataframe”][“tof_column”].

• bins (List, optional) – number of bins. Defaults to config[“energy”][“bins”].

• ranges (Sequence[Tuple[float, float]], optional) – bin ranges. Defaults to config[“energy”][“ranges”].

• biases (np.ndarray, optional) – Bias voltages used. If missing, bias voltages are extracted from the data files.

• bias_key (str, optional) – hdf5 path where bias values are stored. Defaults to config[“energy”][“bias_key”].

• normalize (bool, optional) – Option to normalize traces. Defaults to config[“energy”][“normalize”].

• span (int, optional) – span smoothing parameters of the LOESS method (see scipy.signal.savgol_filter()). Defaults to config[“energy”][“normalize_span”].

• order (int, optional) – order smoothing parameters of the LOESS method (see scipy.signal.savgol_filter()). Defaults to config[“energy”][“normalize_order”].

find_bias_peaks(ranges, ref_id=0, infer_others=True, mode='replace', radius=None, peak_window=None, apply=False)

2. step of the energy calibration workflow: Find a peak within a given range for the indicated reference trace, and tries to find the same peak for all other traces. Uses fast_dtw to align curves, which might not be too good if the shape of curves changes qualitatively. Ideally, choose a reference trace in the middle of the set, and don’t choose the range too narrow around the peak. Alternatively, a list of ranges for all traces can be provided.

Parameters:

• ranges (Union[List[Tuple], Tuple]) – Tuple of TOF values indicating a range. Alternatively, a list of ranges for all traces can be given.

• refid (int, optional) – The id of the trace the range refers to. Defaults to 0.

• infer_others (bool, optional) – Whether to determine the range for the other traces. Defaults to True.

• mode (str, optional) – Whether to “add” or “replace” existing ranges. Defaults to “replace”.

• radius (int, optional) – Radius parameter for fast_dtw. Defaults to config[“energy”][“fastdtw_radius”].

• peak_window (int, optional) – Peak_window parameter for the peak detection algorthm. amount of points that have to have to behave monotoneously around a peak. Defaults to config[“energy”][“peak_window”].

• apply (bool, optional) – Option to directly apply the provided parameters. Defaults to False.

• ref_id (int, default: 0) –

calibrate_energy_axis(ref_id, ref_energy, method=None, energy_scale=None, **kwds)

3. Step of the energy calibration workflow: Calculate the calibration function for the energy axis, and apply it to the dataframe. Two approximations are implemented, a (normally 3rd order) polynomial approximation, and a d^2/(t-t0)^2 relation.

Parameters:

• ref_id (int) – id of the trace at the bias where the reference energy is given.

• ref_energy (float) – Absolute energy of the detected feature at the bias of ref_id

• method (str, optional) –

  Method for determining the energy calibration.

  • ’lmfit’: Energy calibration using lmfit and 1/t^2 form.

  • ’lstsq’, ‘lsqr’: Energy calibration using polynomial form.

  Defaults to config[“energy”][“calibration_method”]

• energy_scale (str, optional) –

  Direction of increasing energy scale.

  • ’kinetic’: increasing energy with decreasing TOF.

  • ’binding’: increasing energy with increasing TOF.

  Defaults to config[“energy”][“energy_scale”]

save_energy_calibration(filename=None, overwrite=False)

Save the generated energy calibration parameters to the folder config file.

Parameters:

• filename (str, optional) – Filename of the config dictionary to save to. Defaults to “sed_config.yaml” in the current folder.

• overwrite (bool, optional) – Option to overwrite the present dictionary. Defaults to False.

append_energy_axis(calibration=None, preview=False, **kwds)

4. step of the energy calibration workflow: Apply the calibration function to to the dataframe. Two approximations are implemented, a (normally 3rd order) polynomial approximation, and a d^2/(t-t0)^2 relation. a calibration dictionary can be provided.

Parameters:

• calibration (dict, optional) – Calibration dict containing calibration parameters. Overrides calibration from class or config. Defaults to None.

• preview (bool) – Option to preview the first elements of the data frame.

• **kwds – Keyword args passed to EnergyCalibrator.append_energy_axis.

add_energy_offset(constant=None, columns=None, signs=None, reductions=None, preserve_mean=None)

Shift the energy axis of the dataframe by a given amount.

Parameters:

• constant (float, optional) – The constant to shift the energy axis by.

• columns (Union[str, Sequence[str]]) – Name of the column(s) to apply the shift from.

• signs (Union[int, Sequence[int]]) – Sign of the shift to apply. (+1 or -1) A positive sign shifts the energy axis to higher kinetic energies. Defaults to +1.

• preserve_mean (bool) – Whether to subtract the mean of the column before applying the shift. Defaults to False.

• reductions (str) – The reduction to apply to the column. Should be an available method of dask.dataframe.Series. For example “mean”. In this case the function is applied to the column to generate a single value for the whole dataset. If None, the shift is applied per-dataframe-row. Defaults to None. Currently only “mean” is supported.

Raises:

ValueError – If the energy column is not in the dataframe.

Return type:

None

append_tof_ns_axis(**kwargs)

Convert time-of-flight channel steps to nanoseconds.

Parameters:

• tof_ns_column (str, optional) – Name of the generated column containing the time-of-flight in nanosecond. Defaults to config[“dataframe”][“tof_ns_column”].

• kwargs – additional arguments are passed to energy.tof_step_to_ns.

align_dld_sectors(sector_delays=None, **kwargs)

Align the 8s sectors of the HEXTOF endstation.

Parameters:

sector_delays (np.ndarray, optional) – Array containing the sector delays. Defaults to config[“dataframe”][“sector_delays”].

calibrate_delay_axis(delay_range=None, datafile=None, preview=False, **kwds)

Append delay column to dataframe. Either provide delay ranges, or read them from a file.

Parameters:

• delay_range (Tuple[float, float], optional) – The scanned delay range in picoseconds. Defaults to None.

• datafile (str, optional) – The file from which to read the delay ranges. Defaults to None.

• preview (bool) – Option to preview the first elements of the data frame.

• **kwds – Keyword args passed to DelayCalibrator.append_delay_axis.

add_jitter(cols=None, amps=None, **kwds)

Add jitter to the selected dataframe columns.

Parameters:

• cols (List[str], optional) – The colums onto which to apply jitter. Defaults to config[“dataframe”][“jitter_cols”].

• amps (Union[float, Sequence[float]], optional) – Amplitude scalings for the jittering noise. If one number is given, the same is used for all axes. For uniform noise (default) it will cover the interval [-amp, +amp]. Defaults to config[“dataframe”][“jitter_amps”].

• **kwds – additional keyword arguments passed to apply_jitter

pre_binning(df_partitions=100, axes=None, bins=None, ranges=None, **kwds)

Function to do an initial binning of the dataframe loaded to the class.

Parameters:

• df_partitions (int, optional) – Number of dataframe partitions to use for the initial binning. Defaults to 100.

• axes (List[str], optional) – Axes to bin. Defaults to config[“momentum”][“axes”].

• bins (List[int], optional) – Bin numbers to use for binning. Defaults to config[“momentum”][“bins”].

• ranges (List[Tuple], optional) – Ranges to use for binning. Defaults to config[“momentum”][“ranges”].

• **kwds – Keyword argument passed to compute.

Returns:

pre-binned data-array.

Return type:

xr.DataArray

compute(bins=100, axes=None, ranges=None, normalize_to_acquisition_time=False, **kwds)

Compute the histogram along the given dimensions.

Parameters:

• bins (int, dict, tuple, List[int], List[np.ndarray], List[tuple], optional) –

  Definition of the bins. Can be any of the following cases:

  • an integer describing the number of bins in on all dimensions

  • a tuple of 3 numbers describing start, end and step of the binning range

  • a np.arrays defining the binning edges

  • a list (NOT a tuple) of any of the above (int, tuple or np.ndarray)

  • a dictionary made of the axes as keys and any of the above as values.

  This takes priority over the axes and range arguments. Defaults to 100.

• axes (Union[str, Sequence[str]], optional) – The names of the axes (columns) on which to calculate the histogram. The order will be the order of the dimensions in the resulting array. Defaults to None.

• ranges (Sequence[Tuple[float, float]], optional) – list of tuples containing the start and end point of the binning range. Defaults to None.

• normalize_to_acquisition_time (Union[bool, str]) – Option to normalize the result to the acquistion time. If a “slow” axis was scanned, providing the name of the scanned axis will compute and apply the corresponding normalization histogram. Defaults to False.

• **kwds –

  Keyword arguments:

  • hist_mode: Histogram calculation method. “numpy” or “numba”. See bin_dataframe for details. Defaults to config[“binning”][“hist_mode”].

  • mode: Defines how the results from each partition are combined. “fast”, “lean” or “legacy”. See bin_dataframe for details. Defaults to config[“binning”][“mode”].

  • pbar: Option to show the tqdm progress bar. Defaults to config[“binning”][“pbar”].

  • n_cores: Number of CPU cores to use for parallelization. Defaults to config[“binning”][“num_cores”] or N_CPU-1.

  • threads_per_worker: Limit the number of threads that multiprocessing can spawn per binning thread. Defaults to config[“binning”][“threads_per_worker”].

  • threadpool_api: The API to use for multiprocessing. “blas”, “openmp” or None. See threadpool_limit for details. Defaults to config[“binning”][“threadpool_API”].

  • df_partitions: A range or list of dataframe partitions, or the number of the dataframe partitions to use. Defaults to all partitions.

  Additional kwds are passed to bin_dataframe.

Raises:

AssertError – Rises when no dataframe has been loaded.

Returns:

The result of the n-dimensional binning represented in an xarray object, combining the data with the axes.

Return type:

xr.DataArray

get_normalization_histogram(axis='delay', use_time_stamps=False, **kwds)

Generates a normalization histogram from the timed dataframe. Optionally, use the TimeStamps column instead.

Parameters:

• axis (str, optional) – The axis for which to compute histogram. Defaults to “delay”.

• use_time_stamps (bool, optional) – Use the TimeStamps column of the dataframe, rather than the timed dataframe. Defaults to False.

• **kwds –

  Keyword arguments:

  -df_partitions (int, optional): Number of dataframe partitions to use.

  Defaults to all.

Raises:

• ValueError – Raised if no data are binned.

• ValueError – Raised if ‘axis’ not in binned coordinates.

• ValueError – Raised if config[“dataframe”][“time_stamp_alias”] not found in Dataframe.

Returns:

The computed normalization histogram (in TimeStamp units per bin).

Return type:

xr.DataArray

view_event_histogram(dfpid, ncol=2, bins=None, axes=None, ranges=None, backend='bokeh', legend=True, histkwds=None, legkwds=None, **kwds)

Plot individual histograms of specified dimensions (axes) from a substituent dataframe partition.

Parameters:

• dfpid (int) – Number of the data frame partition to look at.

• ncol (int, optional) – Number of columns in the plot grid. Defaults to 2.

• bins (Sequence[int], optional) – Number of bins to use for the speicified axes. Defaults to config[“histogram”][“bins”].

• axes (Sequence[str], optional) – Names of the axes to display. Defaults to config[“histogram”][“axes”].

• ranges (Sequence[Tuple[float, float]], optional) – Value ranges of all specified axes. Defaults toconfig[“histogram”][“ranges”].

• backend (str, optional) – Backend of the plotting library (‘matplotlib’ or ‘bokeh’). Defaults to “bokeh”.

• legend (bool, optional) – Option to include a legend in the histogram plots. Defaults to True.

• histkwds (dict, optional) – Keyword arguments for histograms (see matplotlib.pyplot.hist()). Defaults to {}.

• legkwds (dict, optional) – Keyword arguments for legend (see matplotlib.pyplot.legend()). Defaults to {}.

• **kwds – Extra keyword arguments passed to sed.diagnostics.grid_histogram().

Raises:

TypeError – Raises when the input values are not of the correct type.

save(faddr, **kwds)

Saves the binned data to the provided path and filename.

Parameters:

• faddr (str) –

  Path and name of the file to write. Its extension determines the file type to write. Valid file types are:

  • ”.tiff”, “.tif”: Saves a TIFF stack.

  • ”.h5”, “.hdf5”: Saves an HDF5 file.

  • ”.nxs”, “.nexus”: Saves a NeXus file.

• **kwds –

  Keyword argumens, which are passed to the writer functions: For TIFF writing:

  • alias_dict: Dictionary of dimension aliases to use.

  For HDF5 writing:

  • mode: hdf5 read/write mode. Defaults to “w”.

  For NeXus:

  • reader: Name of the nexustools reader to use. Defaults to config[“nexus”][“reader”]

  • definiton: NeXus application definition to use for saving. Must be supported by the used reader. Defaults to config[“nexus”][“definition”]

  • input_files: A list of input files to pass to the reader. Defaults to config[“nexus”][“input_files”]

  • eln_data: An electronic-lab-notebook file in ‘.yaml’ format to add to the list of files to pass to the reader.