brine.brine module#
The brine.brine module contains the class BRINE (Blind Resolvent-based Iterative Noise
Equalizer), the iterative, direction-adaptive row/column scaling procedure described in
“Blind Variance Equalization for Large Data Matrices under General Heteroskedasticity”.
- class brine.brine.BRINE(Y: ndarray)[source]#
Bases:
DysonEqualizerImplements BRINE (Blind Resolvent-based Iterative Noise Equalizer) [1], a data-driven procedure that iteratively rescales the rows and columns of the observed matrix to equalize the average noise variance across both dimensions, restoring the Marchenko-Pastur law for the noise spectral distribution.
BRINE subclasses DysonEqualizer to reuse its plotting and Kolmogorov-Smirnov diagnostics, which only depend on Y, Y_hat, S, S_hat and r_hat. Its own iterative scaling procedure is implemented in run(). The inherited compute()/compute_iteratively() methods still run the original, non-iterative Dyson Equalizer scaling, e.g. for comparison.
- Attributes:
- x_scale: (m) numpy.ndarray
The row scaling factors, such that (up to the global normalizations undone at the end of run())
Y_hat = D{sqrt(x_scale)} @ Y @ D{sqrt(y_scale)}. Note this is the reciprocal convention of DysonEqualizer.x_hat, whereY_hat = Y / D{sqrt(x_hat)} / D{sqrt(y_hat)}.- y_scale: (n) numpy.ndarray
The column scaling factors (see x_scale)
Methods
compute([use_Y_hat])Computes the Dyson Equalizer and stores the results.
compute_iteratively([Y_hat_delta_threshold, ...])Computes the Dyson Equalizer iteratively and stores the results.
ks_Y()Computes the Kolmogorov–Smirnov test between the density of eigenvalues of ¹⁄ₙYYᵀ
ks_Y_hat()Computes the Kolmogorov–Smirnov test between the density of eigenvalues of ¹⁄ₙŶŶᵀ
ks_pvalue_Y()Computes the p-value of the Kolmogorov–Smirnov test between the density of eigenvalues of ¹⁄ₙYYᵀ
ks_pvalue_Y_hat()Computes the p-value of the Kolmogorov–Smirnov test between the density of eigenvalues of ¹⁄ₙŶŶᵀ
plot_mp_cdf_Y_hat([sigma_hat, ...])Plots the cdf of eigenvalues of ¹⁄ₙŶŶᵀ and compares to the Marchenko-Pastur distribution
plot_mp_density_Y([sigma, ...])Plots the density of eigenvalues of ¹⁄ₙYYᵀ and compares to the Marchenko-Pastur distribution
plot_mp_density_Y_hat([sigma_hat, ...])Plots the density of eigenvalues of ¹⁄ₙŶŶᵀ and compares to the Marchenko-Pastur distribution
plot_mp_density_convergence([...])Plots the density of eigenvalues of Y_hat across BRINE's iterations and compares them to the Marchenko-Pastur distribution
plot_mp_eigenvalues_Y([sigma, ...])Plots the eigenvalues of ¹⁄ₙYYᵀ and compares to the Marchenko-Pastur threshold
plot_mp_eigenvalues_Y_hat([sigma_hat, ...])Plots the eigenvalues of ¹⁄ₙŶŶᵀ and compares to the Marchenko-Pastur threshold
plot_mp_eigenvalues_and_densities([...])Plots the eigenvalues of ¹⁄ₙYYᵀ of ¹⁄ₙŶŶᵀ and their densities.
run([Y_hat_delta_threshold, ...])Runs BRINE and stores the results.
See also
References
[1]“Blind Variance Equalization for Large Data Matrices under General Heteroskedasticity”
- iteration_statistics: list[BrineIterationStatistics][source]#
Statistics on each iteration
- plot_mp_density_convergence(show_only_significant: int = 0, show_only_significant_right_margin: float = 0.3, matrix_labels: list[str] = None, ax=None) None[source]#
Plots the density of eigenvalues of Y_hat across BRINE’s iterations and compares them to the Marchenko-Pastur distribution
- Parameters:
- show_only_significant: int, optional
Set this value to show only a small number of significant eigenvalues (defaults to 0 here to show the convergence of the noise) This option is useful is some of the signal eigenvalues are much bigger than the noise. Set to zero to show only significant eigenvalues within the margin indicated by show_only_significant_right_margin
- show_only_significant_right_margin: float, optional
Specifies the size of the right margin (defaults to 0.3) from the largest eigenvalue selected by the show_only_significant option
- matrix_labels: list[str], optional
The labels for each eigenvalue in the list. If not provided, numbers from 0 to n-1 will be used
- ax: plt.Axes, optional
A matplotlib Axes object. If none is provided, a new figure is created.
- run(Y_hat_delta_threshold: float = 1e-06, ks_pvalue_unchanged: bool = True, j_unchanged: bool = True, start_from_dyson: bool = True, max_iterations: int = 10)[source]#
Runs BRINE and stores the results.
- Parameters:
- Y_hat_delta_threshold: float, optional
Terminates if the average difference between Y_hat entries is below this value
- ks_pvalue_unchanged: bool, optional
Terminates if the KS-pvalue has not changed in the last iteration
- j_unchanged: bool, optional
Terminates and reverts to the previous iterate if the estimated potential J has not decreased in the last iteration
- start_from_dyson: bool, optional
If True (default), performs a Dyson Equalizer scaling as the initial point, which makes the method converges faster since it starts closer to the global solution. If False, the method starts from a vector of ones
- max_iterations: int, optional
The maximum number of iterations
- Returns:
- selfBRINE
A reference to this instance
- class brine.brine.BrineIterationStatistics(delta_Y_hat: float = nan, x_hat_mean: float = nan, y_hat_mean: float = nan, kstest_result: KstestResult = None, J: float = nan, S: ndarray = None)[source]#
Bases:
IterationStatisticsThis class stores the results of each iteration of BRINE. It adds the estimated potential and the principal values of Y_hat at that iteration to the base IterationStatistics.
- Attributes:
- J: float
The estimated potential function
- S: (m) numpy.ndarray
The principal values of Y_hat at this iteration