search

mlpack / ensmallen

A header-only C++ library for numerical optimization --
http://ensmallen.org
Other
740 stars 119 forks source link

[aarch64] One test failure in Johnson844LovaszThetaFMatSDP #390

Closed barracuda156 closed 7 months ago

barracuda156 commented 8 months ago

Issue description

Johnson844LovaszThetaFMatSDP test fails on Sonoma / aarch64. The rest pass.

Your environment

Steps to reproduce

./ensmallen_tests --durations yes

Expected behavior

All tests should pass, especially on the current platforms.

Actual behavior

0.691 s: Johnson844LovaszThetaSDP

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ensmallen_tests is a Catch v2.13.10 host application.
Run with -? for options

-------------------------------------------------------------------------------
Johnson844LovaszThetaFMatSDP
-------------------------------------------------------------------------------
/opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_math_ensmallen/ensmallen/work/ensmallen-2.21.0/tests/lrsdp_test.cpp:136
...............................................................................

/opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_math_ensmallen/ensmallen/work/ensmallen-2.21.0/tests/lrsdp_test.cpp:161: FAILED:
  REQUIRE( finalValue == Approx(-14.0).epsilon(0.1) )
with expansion:
  -0.22222f == Approx( -14.0 )
rcurtin commented 8 months ago

This test in particular has been flaky for a long time. I think the LRSDP implementation is just fine, it's just that maybe the test needs a couple trials or something like this. Ideally any fix should see it succeeding with 1000 different random seeds (or more, depending on one's patience level). I'll get around to solving this eventually, but if someone else comes along and wants to handle it please do feel free!

In some other ensmallen packaging scripts I maintain, I specifically run and skip this test in particular: ./ensmallen_tests ~Johnson844LovaszThetaFMatSDP.

mlpack-bot[bot] commented 7 months ago

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! :+1:

barracuda156 commented 7 months ago

Maybe just skip it then on arm64 for now?

rcurtin commented 7 months ago

I just dug into it this afternoon and opened #392 as a bugfix. Thank you for the report @barracuda156! We had seen this one for a while but I had no way to reproduce it---but I do happen to have an M1 Macbook sitting around with which I can reproduce your setup exactly... and so then the test failed, and I had something to dig into. Fun numerical precision issue, detailed more in #392.

barracuda156 commented 7 months ago

@rcurtin Thank you for dealing with this!

rcurtin commented 7 months ago

Fixed in #392. :+1:

barracuda156 commented 7 months ago

Just in case, ran tests on PowerPC, all good too:

--->  Testing ensmallen
Executing:  cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_math_ensmallen/ensmallen/work/build" && ./ensmallen_tests --durations yes 
ensmallen version: 2.21.0 (Bent Antenna)
armadillo version: 12.2.0 (Cortisol Profusion Deluxe)
random seed: 0
0.082 s: ActiveCMAESRosenbrockFunctionTest
1.228 s: ApproxActiveCMAESRosenbrockFunctionTest
0.038 s: ActiveCMAESRosenbrockFunctionFMatTest
1.016 s: ApproxActiveCMAESRosenbrockFunctionFMatTest
0.238 s: ApproxActiveCMAESEmptyTransformationLogisticRegressionFMatTest
0.001 s: AdaBeliefSphereFunctionTest
0.001 s: AdaBeliefSphereFunctionTestFMat
0.002 s: AdaBeliefMcCormickFunctionTest
0.170 s: AdaBeliefLogisticRegressionTest
0.231 s: AdaBeliefLogisticRegressionFMatTest
0.003 s: AdaBoundSphereFunctionTest
0.003 s: AdaBoundSphereFunctionTestFMat
0.002 s: AMSBoundSphereFunctionTest
0.002 s: AMSBoundphereFunctionTestFMat
0.028 s: AdaBoundSphereFunctionTestSpMat
0.004 s: AdaBoundSphereFunctionTestSpMatDenseGradient
0.019 s: AMSBoundSphereFunctionTestSpMat
0.002 s: AMSBoundSphereFunctionTestSpMatDenseGradient
0.150 s: AdaDeltaLogisticRegressionTest
0.135 s: AdaDeltaLogisticRegressionTestFMat
3.237 s: AdaGradLogisticRegressionTest
0.563 s: AdaGradLogisticRegressionTestFMat
2.534 s: AdaSqrtLogisticRegressionTest
2.625 s: AdaSqrtLogisticRegressionTestFMat
0.001 s: AdamSphereFunctionTest
0.001 s: AdamSphereFunctionTestFMat
0.006 s: AdamSphereFunctionTestSpMat
0.001 s: AdamSphereFunctionTestSpMatDenseGradient
0.001 s: AdamStyblinskiTangFunctionTest
0.001 s: AdamMcCormickFunctionTest
0.001 s: AdamMatyasFunctionTest
0.001 s: AdamEasomFunctionTest
0.002 s: AdamBoothFunctionTest
0.076 s: AMSGradSphereFunctionTestFMat
2.459 s: AMSGradSphereFunctionTestSpMat
0.203 s: AMSGradSphereFunctionTestSpMatDenseGradient
0.097 s: AdamLogisticRegressionTest
16.600 s: AdaMaxLogisticRegressionTest
1.847 s: AMSGradLogisticRegressionTest
0.098 s: NadamLogisticRegressionTest
18.975 s: NadaMaxLogisticRegressionTest
0.103 s: OptimisticAdamLogisticRegressionTest
0.098 s: PadamLogisticRegressionTest
0.103 s: QHAdamLogisticRegressionTest
0.105 s: QHAdamLogisticRegressionFMatTest
9.175 s: QHAdamLogisticRegressionSpMatTest
0.476 s: AdamAckleyFunctionTest
0.003 s: AdamBealeFunctionTest
0.024 s: AdamGoldsteinPriceFunctionTest
0.002 s: AdamLevyFunctionTest
0.001 s: AdamHimmelblauFunctionTest
0.006 s: AdamThreeHumpCamelFunctionTest
0.000 s: AdamResetPolicyTest
0.006 s: AugLagrangianTestFunctionTest
0.005 s: GockenbachFunctionTest
0.007 s: GockenbachFunctionFMatTest
0.057 s: GockenbachFunctionSpMatTest
0.264 s: BBSBBLogisticRegressionTest
0.797 s: BBSArmijoLogisticRegressionTest
0.292 s: BBSBBLogisticRegressionFMatTest
0.987 s: BBSArmijoLogisticRegressionFMatTest
5.928 s: BBSBBLogisticRegressionSpMatTest
18.996 s: BBSArmijoLogisticRegressionSpMatTest
0.052 s: EarlyStopAtMinLossLambdaCallbackTest
0.068 s: EarlyStopAtMinLossCustomLambdaTest
0.078 s: ActiveCMAESCallbacksFullFunctionTest
0.044 s: AdaBoundCallbacksFullFunctionTest
0.050 s: AdaDeltaCallbacksFullFunctionTest
0.049 s: AdaGradCallbacksFullFunctionTest
0.045 s: AdamCallbacksFullFunctionTest
0.082 s: BigBatchSGDCallbacksFullFunctionTest
0.078 s: CMAESCallbacksFullFunctionTest
0.193 s: CNECallbacksFullFunctionTest
0.306 s: DECallbacksFullFunctionTest
0.052 s: EveCallbacksFullFunctionTest
0.059 s: FTMLCallbacksFullFunctionTest
0.048 s: GradientDescentCallbacksFullFunctionTest
0.125 s: IQNCallbacksFullFunctionTest
0.072 s: KatyushaCallbacksFullFunctionTest
4.909 s: NSGA2CallbacksFullFunctionTest
1.039 s: MOEADCallbacksFullFunctionTest
0.085 s: LookaheadCallbacksFullFunctionTest
0.048 s: PadamCallbacksFullFunctionTest
0.045 s: QHAdamCallbacksFullFunctionTest
0.047 s: RMSPropCallbacksFullFunctionTest
0.059 s: SARAHCallbacksFullFunctionTest
0.046 s: SCDCallbacksFullFunctionTest
0.052 s: SGDCallbacksFullFunctionTest
0.055 s: SGDRCallbacksFullFunctionTest
0.050 s: SMORMS3CallbacksFullFunctionTest
0.056 s: SPALeRASGDCallbacksFullFunctionTest
0.046 s: SPSACallbacksFullFunctionTest
0.067 s: SVRGCallbacksFullFunctionTest
0.051 s: SWATSCallbacksFullFunctionTest
0.052 s: WNGradCallbacksFullFunctionTest
0.050 s: ParallelSGDCallbacksFullFunctionTest
5.811 s: LBestPSOCallbacksFullFunctionTest
0.047 s: L_BFGSCallbacksFullFunctionTest
0.143 s: SACallbacksFullFunctionTest
3.463 s: EarlyStopAtMinLossCallbackTest
0.002 s: PrintLossCallbackTest
0.003 s: ProgressBarCallbackTest
4.474 s: StoreBestCoordinatesCallbackTest
0.500 s: TimerStopCallbackTest
0.002 s: ProgressBarCallbackNoMaxIterationsTest
0.001 s: ProgressBarCallbackNoMaxIterationsEpochTest
0.000 s: ProgressBarCallbackEpochTest
0.088 s: ReportCallbackTest
0.003 s: GradClipByNormCallbackTest
0.003 s: GradClipByValueCallbackTest
0.274 s: PreCalcCDTest
0.007 s: DisjointFeatureTest
0.012 s: DisjointFeatureFMatTest
0.021 s: DisjointFeatureSpMatTest
0.000 s: GreedyDescentTest
0.000 s: CyclicDescentTest
0.001 s: RandomDescentTest
0.001 s: LogisticRegressionFunctionPartialGradientTest
0.015 s: SoftmaxRegressionFunctionPartialGradientTest
0.363 s: CMAESLogisticRegressionTest
0.402 s: ApproxCMAESLogisticRegressionTest
0.341 s: CMAESLogisticRegressionFMatTest
0.155 s: ApproxCMAESLogisticRegressionFMatTest
0.124 s: ApproxCMAESEmptyTransformationLogisticRegressionFMatTest
5.254 s: CNELogisticRegressionTest
5.749 s: CNELogisticRegressionFMatTest
11.226 s: CNECrossInTrayFunctionTest
10.829 s: CNEAckleyFunctionTest
10.816 s: CNEBealeFunctionTest
10.955 s: CNEGoldsteinPriceFunctionTest
11.186 s: CNELevyFunctionN13Test
0.047 s: CNEHimmelblauFunctionTest
10.953 s: CNEThreeHumpCamelFunctionTest
12.916 s: CNESchafferFunctionN4Test
13.111 s: CNESchafferFunctionN2Test
0.216 s: DELogisticRegressionTest
0.239 s: DELogisticRegressionFMatTest
0.049 s: DemonAdamLogisticRegressionTest
0.001 s: DemonAdamSphereFunctionTest
0.001 s: DemonAdamMatyasFunctionTest
0.004 s: DemonAdamSphereFunctionTestFloat
0.001 s: DemonAdamMatyasFunctionTestFloat
0.050 s: DemonAdaMaxLogisticRegressionTest
0.167 s: DemonSGDLogisticRegressionTest
0.003 s: DemonSGDSimpleTestFunctionFloat
1.644 s: EveLogisticRegressionTest
0.005 s: EveSphereFunctionTest
0.006 s: EveStyblinskiTangFunctionTest
0.005 s: EveStyblinskiTangFunctionFMatTest
0.133 s: EveStyblinskiTangFunctionSpMatTest
0.002 s: FWOMPTest
0.002 s: FWRegularizedOMP
0.001 s: FWPruneSupportOMP
0.002 s: FWAtomNormConstraint
0.125 s: ClassicFW
0.106 s: ClassicFWFMat
0.001 s: FWLineSearch
0.216 s: FWLineSearchFMat
0.422 s: FTMLLogisticRegressionTest
0.087 s: FTMLSphereFunctionTest
0.057 s: FTMLStyblinskiTangFunctionTest
0.056 s: FTMLStyblinskiTangFunctionFMatTest
0.000 s: AddEvaluateWithGradientEmptyTest
0.000 s: AddEvaluateWithGradientEvaluateOnlyTest
0.000 s: AddEvaluateWithGradientGradientOnlyTest
0.000 s: AddEvaluateWithGradientBothTest
0.000 s: AddEvaluateWithGradientEvaluateWithGradientTest
0.000 s: AddEvaluateWithGradientAllThreeTest
0.000 s: LogisticRegressionEvaluateWithGradientTest
0.000 s: SDPTest
0.000 s: AddSeparableEvaluateWithGradientEmptyTest
0.000 s: AddSeparableEvaluateWithGradientEvaluateOnlyTest
0.000 s: AddSeparableEvaluateWithGradientGradientOnlyTest
0.000 s: AddSeparableEvaluateWithGradientBothTest
0.000 s: AddSeparableEvaluateWGradientEvaluateWithGradientTest
0.000 s: AddSeparableEvaluateWithGradientAllThreeTest
0.000 s: AddEvaluateWithGradientMixedTypesTest
0.000 s: AddEvaluateWithGradientMixedTypesStaticTest
0.000 s: SeparableFunctionTypeCheckTest
0.000 s: LagrangianFunctionTypeCheckTest
0.000 s: SparseFunctionTypeCheckTest
0.000 s: ResolvableFunctionTypeCheckTest
0.001 s: SimpleGDTestFunction
0.007 s: GDRosenbrockTest
0.008 s: GDRosenbrockFMatTest
0.001 s: GridSearchTest
0.001 s: GridSearchFMatTest
0.001 s: GridSearchIMatTest
0.260 s: IQNLogisticRegressionTest
0.420 s: IQNLogisticRegressionFMatTest
0.445 s: KatyushaLogisticRegressionTest
0.480 s: KatyushaProximalLogisticRegressionTest
0.512 s: KatyushaLogisticRegressionFMatTest
0.463 s: KatyushaProximalLogisticRegressionFMatTest
27.135 s: KatyushaLogisticRegressionSpMatTest
27.087 s: KatyushaProximalLogisticRegressionSpMatTest
0.003 s: RosenbrockFunctionTest
0.002 s: RosenbrockFunctionFloatTest
0.004 s: RosenbrockFunctionSpGradTest
0.004 s: RosenbrockFunctionSpMatTest
0.002 s: ColvilleFunctionTest
0.002 s: WoodFunctionTest
1.134 s: GeneralizedRosenbrockFunctionTest
0.007 s: RosenbrockWoodFunctionTest
0.000 s: FuncFWTest
0.000 s: FuncFWFMatTest
0.002 s: LookaheadAdamSphereFunctionTest
0.002 s: LookaheadAdaGradSphereFunction
0.371 s: LookaheadAdamLogisticRegressionTest
0.018 s: LookaheadAdamSimpleSphereFunctionFloat
12.641 s: Johnson844LovaszThetaSDP
6.455 s: Johnson844LovaszThetaFMatSDP
22.573 s: ErdosRenyiRandomGraphMaxCutSDP
64.130 s: GaussianMatrixSensingSDP
2.438 s: MOEADSchafferN1DoubleTest
4.821 s: MOEADSchafferN1TestVectorDoubleBounds
2.593 s: MOEADFonsecaFlemingDoubleTest
2.586 s: MOEADFonsecaFlemingTestVectorDoubleBounds
2.480 s: MOEADSchafferN1FloatTest
2.461 s: MOEADSchafferN1TestVectorFloatBounds
2.592 s: MOEADFonsecaFlemingFloatTest
2.585 s: MOEADFonsecaFlemingTestVectorFloatBounds
3.145 s: MOEADZDTONETest
3.168 s: MOEADDIRICHLETZDT3Test
4.094 s: MomentumSGDSpeedUpTestFunction
3.904 s: MomentumSGDGeneralizedRosenbrockTest
13.598 s: MomentumSGDGeneralizedRosenbrockFMatTest
127.971 s: MomentumSGDGeneralizedRosenbrockSpMatTest
2.627 s: NesterovMomentumSGDSpeedUpTestFunction
5.219 s: NesterovMomentumSGDGeneralizedRosenbrockTest
2.083 s: NesterovMomentumSGDGeneralizedRosenbrockFMatTest
216.801 s: NesterovMomentumSGDGeneralizedRosenbrockSpMatTest
0.315 s: NSGA2SchafferN1DoubleTest
0.305 s: NSGA2SchafferN1TestVectorDoubleBounds
0.440 s: NSGA2FonsecaFlemingDoubleTest
0.428 s: NSGA2FonsecaFlemingTestVectorDoubleBounds
0.364 s: NSGA2SchafferN1FloatTest
0.361 s: NSGA2SchafferN1TestVectorFloatBounds
0.508 s: NSGA2FonsecaFlemingFloatTest
0.539 s: NSGA2FonsecaFlemingTestVectorFloatBounds
15.433 s: NSGA2ZDTONETest
0.302 s: NSGA2FrontTest
0.000 s: ExponentialBackoffDecayTest
0.018 s: ProjectToL1
0.018 s: ProjectToL0
0.559 s: LBestPSOSphereFunctionTest
0.570 s: LBestPSOSphereFunctionFMatTest
34.982 s: LBestPSORosenbrockTest
63.098 s: LBestPSORosenbrockFMatTest
43.227 s: LBestPSORosenbrockDoubleTest
70.307 s: LBestPSOCrossInTrayFunctionTest
0.613 s: LBestPSOAckleyFunctionTest
0.488 s: LBestPSOBealeFunctionTest
5.379 s: LBestPSOGoldsteinPriceFunctionTest
0.543 s: LBestPSOLevyFunctionN13Test
0.466 s: LBestPSOHimmelblauFunctionTest
0.433 s: LBestPSOThreeHumpCamelFunctionTest
4.213 s: LBestPSOSchafferFunctionN2Test
0.004 s: QHSphereFunction
0.005 s: QHSphereFunctionFMat
0.270 s: QHSpMatTestSphereFunction
9.019 s: QHSGDSGDGeneralizedRosenbrockTest
0.098 s: RMSPropLogisticRegressionTest
0.100 s: RMSPropLogisticRegressionFMatTest
8.909 s: RMSPropLogisticRegressionSpMatTest
0.228 s: SAGeneralizedRosenbrockTest
0.035 s: SARosenbrockTest
0.020 s: SARosenbrockFMatTest
0.013 s: RastrigrinFunctionTest
0.621 s: SARAHLogisticRegressionTest
0.629 s: SARAHPlusLogisticRegressionTest
0.613 s: SARAHLogisticRegressionFMatTest
0.634 s: SARAHPlusLogisticRegressionFMatTest
45.417 s: SARAHLogisticRegressionSpMatTest
44.038 s: SARAHPlusLogisticRegressionSpMatTest
0.158 s: SmallMaxCutSdp
129.106 s: DeprecatedSmallLovaszThetaSdp
128.452 s: SmallLovaszThetaSdp
0.177 s: LogChebychevApproxSdp
0.062 s: CorrelationCoeffToySdp
0.016 s: SGDRCyclicalResetTest
0.445 s: SGDRLogisticRegressionTest
0.456 s: SGDRLogisticRegressionFMatTest
9.849 s: SGDRLogisticRegressionSpMatTest
2.544 s: GeneralizedRosenbrockTest
2.010 s: GeneralizedRosenbrockTestFloat
0.103 s: SMORMS3LogisticRegressionTest
0.109 s: SMORMS3LogisticRegressionFMatTest
0.018 s: SnapshotEnsemblesResetTest
0.451 s: SnapshotEnsemblesLogisticRegressionTest
0.455 s: SnapshotEnsemblesLogisticRegressionFMatTest
10.065 s: SnapshotEnsemblesLogisticRegressionSpMatTest
0.202 s: LogisticRegressionTest
0.201 s: LogisticRegressionFMatTest
0.858 s: SPSASphereFunctionTest
0.805 s: SPSASphereFunctionFMatTest
2.272 s: SPSASphereFunctionSpMatTest
0.795 s: SPSAMatyasFunctionTest
1.489 s: SPSALogisticRegressionTest
1.230 s: SVRGLogisticRegressionTest
1.877 s: SVRGBBLogisticRegressionTest
1.712 s: SVRGLogisticRegressionFMatTest
1.754 s: SVRGBBLogisticRegressionFMatTest
107.526 s: SVRGLogisticRegressionSpMatTest
123.295 s: SVRGBBLogisticRegressionSpMatTest
0.404 s: SWATSLogisticRegressionTestFunction
0.047 s: SWATSSphereFunctionTest
0.076 s: SWATSStyblinskiTangFunctionTest
0.031 s: SWATSStyblinskiTangFunctionFMatTest
0.934 s: SWATSStyblinskiTangFunctionSpMatTest
1.497 s: WNGradLogisticRegressionTest
0.002 s: WNGradSphereFunctionTest
0.016 s: WNGradStyblinskiTangFunctionTest
0.007 s: WNGradStyblinskiTangFunctionFMatTest
0.160 s: WNGradStyblinskiTangFunctionSpMatTest
0.001 s: YogiSphereFunctionTest
0.001 s: YogiSphereFunctionTestFMat
0.001 s: YogiMcCormickFunctionTest
0.099 s: YogiLogisticRegressionTest
0.104 s: YogiLogisticRegressionFMatTest
===============================================================================
All tests passed (12589 assertions in 320 test cases)
rcurtin commented 7 months ago

Awesome!