Package xal.extension.wirescan.profile

Class ProfileDataProcessor

java.lang.Object
xal.extension.wirescan.profile.ProfileDataProcessor
public class ProfileDataProcessor extends Object

Class for processing profile data contained in ProfileData objects.

The objective of this class is to interpolate any missing data (represented by a value Double.NaN), remove the noise baseline, decouple the cross-talk between the signals, then gate the noise in the signal. Thus, it is expected that the data is processed by the following sequence of method calls:

   interpolateMissingData()
   removeBaselineNoise(double)
   decoupleSignals(int, double)
   thresholdFilter(double)

Once this sequence is completed the processed signals will be approximately zero base-lined, decoupled, and normalized to have unity first integral.

One may apply the above sequence of calls using default values for all the arguments with the single call to ProfileDataProcessor.processData()

There are three numeric tuning parameters that must be provided in the processing.

ProfileDataProcessor.removeBaselineNoise(double dblSigThreshold)
dblSigThreshold = amplitude of indicator function deciding signal presence
This value is in the interval [0,1]. Larger will use more signal samples to determine noise properties, but may also include actual signals. Smaller values are more likely to exclude signals in the noise calculations, but will use few samples to characterize the noise.

ProfileDataProcessor.decoupleSignals(int cntSmpMax, double dblFracSigMin)
cntSmpMax = number of samples to average when computing coefficients
This value must be an integer greater than 1. This value determined the number of samples around the maximum value used when computing coupling coefficients. Small numbers will use samples with the most integrity. Larger numbers will include samples more affected by noise, but may provide a better statistical approximation if the data set contains suspect data.
dblFracSigMin fractional signal strength necessary for inclusion in coupling calculation
This value ranges in the interval [0,1]. Large values ensure that only the best samples are used in computing the coupling coefficients, however, it reduces the available sample size. Small values increase the sample size but may include samples corrupted by noise.

ProfileDataProcessor.thresholdFilter(double dblFracSigMax)
dblFracSigMax = fraction of maximum signal where threshold is applied
The argument is the fraction of the maximum signal strength where the threshold is activated. Thus, this value is in the interval (-1,1) where a value of 1 would cause a filter output of zero for all inputs. Although sign is not enforced per se, we are expecting a positive signal. Thus, a value for dblFracSigMax in the interval (0,1) is appropriate. Then if the projection data were strictly non-negative, a value 0 would indicate no action by the filter.


Default values are suggested as constants with this class.

Author:
Christopher K. Allen

  • Field Details

    • DBL_SIG_INDICATOR_THRESHOLD

      public static final double DBL_SIG_INDICATOR_THRESHOLD
      Amplitude of signal indicator used when determining noise floor
      See Also:

    • INT_CPL_MAX_SAMPLE_COUNT

      public static final int INT_CPL_MAX_SAMPLE_COUNT
      Maximum number of samples to use when computing coupling coefficients
      See Also:

    • DBL_CPL_MIN_FRAC_SIGNAL_AMPL

      public static final double DBL_CPL_MIN_FRAC_SIGNAL_AMPL
      Minimum fractional signal amplitude necessary to include sample for coupling coefficient calc.
      See Also:

    • DBL_THR_MAX_FRAC_SIGNAL_AMPL

      public static final double DBL_THR_MAX_FRAC_SIGNAL_AMPL
      Maximum fractional signal amplitude for threshold filtering
      See Also:

    • INT_DSP_LOWPASS_ORDER

      public static final int INT_DSP_LOWPASS_ORDER
      Order of the filter
      See Also:

    • ARR_DBL_LOWPASS_INP_COEFFS

      public static final double[] ARR_DBL_LOWPASS_INP_COEFFS
      Coefficients of the filter input

    • ARR_DBL_LOWPASS_OUT_COEFFS

      public static final double[] ARR_DBL_LOWPASS_OUT_COEFFS
      Coefficients of the filter output

    • INT_DSP_HIGHPASS_ORDER

      public static final int INT_DSP_HIGHPASS_ORDER
      Order of the filter
      See Also:

    • ARR_DBL_HIGHPASS_INP_COEFFS

      public static final double[] ARR_DBL_HIGHPASS_INP_COEFFS
      Coefficients of the filter input

    • ARR_DBL_HIGHPASS_OUT_COEFFS

      public static final double[] ARR_DBL_HIGHPASS_OUT_COEFFS
      Coefficients of the filter output

  • Constructor Details

    • ProfileDataProcessor

      public ProfileDataProcessor(ProfileData dataOrg)
      Create a new ProfileDataProcessor object attached to the given profile data object.
      Parameters:
      dataOrg - profile data under process
      See Also:

  • Method Details

    • getDataSize

      public int getDataSize()
      Convenience method for returning the (vector) size of the associated ProfileData data object.
      Returns:
      size of the associated projection data vectors
      See Also:
      • gov.sns.tools.data.profile.ProfileData#getDataSize()

    • getRawData

      public ProfileData getRawData()
      Return the raw profile data object.
      Returns:
      raw profile data object

    • getProcessedData

      public ProfileData getProcessedData()
      Return the processed profile data object.
      Returns:
      processed profile data object

    • getProcessedData

      public double[] getProcessedData(ProfileData.Angle view)
      Return the processed data for the given profile view.
      Parameters:
      view - desired profile view
      Returns:
      processed profile data

    • indicatorFunction

      public double[] indicatorFunction(ProfileData.Angle view) throws IllegalArgumentException
      Compute and return the signal presence indicator function for the given projection view. The indicator function is computed by subtracting the (running) average from the signal then computing the total variation.
      Parameters:
      view - projection data
      Returns:
      signal presence indicator function
      Throws:
      IllegalArgumentException - this should not occur (serious internal error)
      See Also:

    • processData

      public void processData() throws DataProcessingException

      Perform the default processing on the associated projection data. The following operations are performed:

         interpolateMissingData()
         removeBaselineNoise(double)
         decoupleSignals(int, double)
         thresholdFilter(double)

      Default values for the arguments are taken from the global constants of this class.

      Throws:
      DataProcessingException - data set exceeds maximum bad data points

    • interpolateMissingData

      public void interpolateMissingData() throws DataProcessingException
      Interpolate between missing data points. Missing data is indicated with a value Double.NaN. If we encounter more than two contiguous bad data points the operation punts by throwing an exception.
      Throws:
      DataProcessingException - data set exceeds maximum bad data points

    • lowPassFilter

      public void lowPassFilter()

    • highPassFilter

      public void highPassFilter()

    • removeBaselineNoise

      public void removeBaselineNoise(double dblSigThreshold)

      Remove the baseline noise signal from the data set. We assume that each profile signal f^(n) is given by the following:

         f^(n) = f(n) + W(n)

      where f(n) is the true signal at index n and W(n) is a noise process with a (possibly) nonzero mean given by E[W(n)] = w (independent of n). The value of w represents the baseline noise signal.

      To remove this content we compute the signal indicator function for each signal and find the largest index such that all the indicator functions are less than the given value. All data before this index is assumed to be noise. We use that data to compute w for each signal assuming that the original signal is filtered such that E[(W - w)²] is small. The value w is subtracted from each respective signal.

      Note, however, that only the mean component of the noise is removed. If the noise process has a non-zero standard deviation σW = E[(W(n) - w)2]½ then spurious noise is still present. The spurious component can be attenuated by further filtering, but at the cost of degrading the true signal f(n). We originally assumed that this value was small enough to get an accurate determination from the data before signal presence.

      Parameters:
      dblSigThreshold - amplitude of indicator function deciding signal presence
      Throws:
      DataProcessingException - currently this should not happen

    • decoupleSignals

      public void decoupleSignals(int cntSmpMax, double dblFracSigMin)

      Decouple the project data signals. That is, remove the cross-talk signals from each projection data set.

      It is assumed that the profile signals may be represented by the equation

         f*(n) = GCf(n)

      where f*(n) is the vector-valued function of measured profile data (one element for each projection) at time n, G is the gain matrix, C is the cross-talk matrix, and f(n) is the vector-valued function of profile data. The gain matrix is chosen so that the integral of element in f(n) is unity. That is, ∫f(n)dn = (1,1,…,1).

      NOTE:

      It is best to use this method after the baseline noise have been removed from the profile data.

      Parameters:
      cntSmpMax - maximum number of samples to average when computing coefficients
      dblFracSigMin - minimum fractional signal strength for use in calculation

    • thresholdFilter

      public void thresholdFilter(double dblFracSigMax)

      Amplitude threshold filter for use as a noise gate.

      Apply a threshold filter to each projection data signal using the given fractional amplitude. Specifically, the projection data signal strength must be strictly greater than dblFracSigMax of the maximum signal strength to achieve a nonzero value. If the value dblFracSigMax is chosen properly the filter will perform as a noise gate where we assume a positive signal in the projection data.

      NOTE:
      - This is a zero-order filter (no memory), but it is highly nonlinear. Thus, you need to be careful when using this method, especially for computations with the resulting data.
      - The argument is the fraction of the maximum signal strength where the threshold is activated. Thus, this value is in the interval (-1,1) where a value of 1 would cause a filter output of zero for all inputs.
      - Although sign is not enforced per se, we are expecting a positive signal. Thus, a value for dblFracSigMax in the interval (0,1) is appropriate. Then if the projection data were strictly non-negative, a value 0 would indicate no action by the filter.

      Parameters:
      dblFracSigMax - fraction of maximum signal where threshold is applied

    • toString

      public String toString()
      Write out contents both original data and processed data to a string for inspection. The returned string is a formatted table of values as produced by DigitalFunctionUtility#buildValueTable().
      Overrides:
      toString in class Object
      Returns:
      string of tabulated projection data
      See Also: