Mylin/#15 add new histogram test modify old

[acdo2002]

Description

Finish the Issue #15 , adding two new tests to test the new setHistogramRequirement protobuf: SET_HISTOGRAMCONFIG.test.t SET_HISTOGRAMCONFIG_QUEUE.test.t

The detail documentation is here

So far, I have tested all files are passed with the current dev backend on MacOS 11.

Checklist

For the pull request:

• [x] The Document unchange / Need update the Document

[ajm-ska]

@acdo2002 What size is your h_m51_b_s05_drz_sci.fits file? The copy that I have doesn't seem to work. The one I have is 419823360 bytes. Perhaps you have a different version of it? (I have put it in /images/set_QA/h_m51_b_s05_drz_sci.fits on almac).

[acdo2002]

@ajm-asiaa -rwxrwxrwx 1 ming-yi staff 419823360 Dec 9 2021 h_m51_b_s05_drz_sci.fits Yes, I have the same size for h_m51_b_s05_drz_sci.fits, any problem on the different platforms?

[ajm-ska]

I hope it doesn't need special customisation for different platforms. This is the error on RHEL7/RHEL8/RHEL9 and macOS12:

FAIL src/test/SET_HISTOGRAMCONFIG_QUEUE.test.ts (18.228s) Testing the large image with multi-polygon region and set_histogram_requirement, let the region_histogram_data in queue: Register a session ✓ check connection (2ms) ✓ Get basepath (394ms) Go to "set_QA" folder and open image "h_m51_b_s05_drz_sci.fits" ✓ Preparation: Open image (3302ms) ✓ (Step 1) Set a many points polygon region: (2ms) ✕ (Step 2) Send sequent setHistogramRequirements with different numBins, to simulate the user drag the Number of bins in the setting of Histogram (10002ms) ✕ (Step 3) Check the receiveing RegionHistogramData * 12 (2ms)

● Testing the large image with multi-polygon region and set_histogram_requirement, let the region_histogram_data in queue: › Register a session › Go to "set_QA" folder and open image "h_m51_b_s05_drz_sci.fits" › (Step 2) Send sequent setHistogramRequirements with different numBins, to simulate the user drag the Number of bins in the setting of Histogram

: Timeout - Async callback was not invoked within the 10000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 10000ms timeout specified by jest.setTimeout.Error:

  151 |             let RegionHistogramDataArray: CARTA.RegionHistogramData[] = [];
  152 |             let regionHistogramDataResponse: any = []
> 153 |             test(`(Step 2) Send sequent setHistogramRequirements with different numBins, to simulate the user drag the Number of bins in the setting of Histogram`, async () => {
      |             ^
  154 |                 let count = 0;
  155 |                 let regionHistogramDataPromise = new Promise((resolve)=>{
  156 |                     msgController.histogramStream.subscribe({

  at new Spec (node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
  at Suite.<anonymous> (src/test/SET_HISTOGRAMCONFIG_QUEUE.test.ts:153:13)

● Testing the large image with multi-polygon region and set_histogram_requirement, let the region_histogram_data in queue: › Register a session › Go to "set_QA" folder and open image "h_m51_b_s05_drz_sci.fits" › (Step 3) Check the receiveing RegionHistogramData * 12

expect(received).toEqual(expected) // deep equality

Expected: 12
Received: 0

  175 |
  176 |             test(`(Step 3) Check the receiveing RegionHistogramData * 12 `, () => {
> 177 |                 expect(regionHistogramDataResponse.length).toEqual(assertItem.numBinsArray.length);
      |                                                            ^
  178 |                 regionHistogramDataResponse.map((RegionHistogramData,index) => {
  179 |                     expect(RegionHistogramData.progress).toEqual(assertItem.ResponseRegionHistogramData.progress);
  180 |                     expect(RegionHistogramData.regionId).toEqual(assertItem.ResponseRegionHistogramData.regionId);
  at Object.<anonymous> (src/test/SET_HISTOGRAMCONFIG_QUEUE.test.ts:177:60)

What could be wrong?

[acdo2002]

@ajm-asiaa I suppose that it is due to queue timeout for the backend performance. the initial setting for the frontend is 200ms (minimum), but the ICD test can change this time interval. I change to 500ms, could you manually test them again? to see whether the backend can return 12 HistogramData ICD messages.

[acdo2002]

@ajm-asiaa I have used almac to test the set time. I decide to set config.repeat.setHistogram as 200ms and config.timeout.dragManyNumberBins as 100000ms. Please check it again.

[ajm-ska]

@acdo2002 It turns out we need to increase dragManyNumberBins. First I increased it from 10000ms to 30000ms. That worked for some platforms. Here were the times: macOS12 18144ms macOS13 26746ms RHEL7 19644ms RHEL8 19483ms RHEL9 24780ms

So 30000ms may be enough.

The other three platforms appear to have other problems. For some reason the PER_CUBE_HISTOGRAM_CANCELLATION.test.ts test fails on Ubuntu 20.04 and 22.04 so it won't reach the new tests.

Set histogram requirements:

        ✕ (Step1) "SDC335.579-0.292.spw0-channel-cutted.line.fits" REGION_HISTOGRAM_DATA should arrive completely within 10000 ms: (10002ms)

It looks like it will need more than 10000 ms. I see the 10000 ms is hard coded in the test. Perhaps this would be a good time to modify it to be a timeout setting that we can tune in config.json?

As for macOS11, in the SET_HISTOGRAMCONFIG_QUEUE.test.ts test, it looks like it needs more time than 6000ms to open the file.

      Go to "set_QA" folder and open image "h_m51_b_s05_drz_sci.fits"
        ✕ Preparation: Open image (6011ms)
        ✓ (Step 1) Set a many points polygon region: (3ms)
        ✕ (Step 2) Send sequent setHistogramRequirements with different numBins, to simulate the user drag the Number of bins in the setting of Histogram (33152ms)
        ✓ (Step 3) Check the receiveing RegionHistogramData * 12  (24ms)

I am currently testing it with openFile increased to 10000.

[ajm-ska]

@acdo2002 OK. I confirm that macOS11 requires more than 6000ms to open h_m51_b_s05_drz_sci.fits. In the last test, it needed 6468ms. So perhaps we could set openFile to 7500ms or 8000ms to be safe.

[acdo2002]

@ajm-asiaa I put the timeout of step 1 in PER_CUBE_HISTOGRAM_CANCELLATION.test.ts from 10000ms to 30000ms. About SET_HISTOGRAMCONFIG_QUEUE.test.ts, I think the new set timeout should be fine.

Please let me know if there is any other timeout should be adjusted to fit the Jenkins.

[ajm-ska]

@acdo2002 We got a failure on RHEL8. It failed once out of 5 runs, so it seems it might not be able to pass consistently. Would it be possible to do something to make it more robust?

FAIL src/test/SET_HISTOGRAMCONFIG_QUEUE.test.ts (27.342s)

  Testing the large image with multi-polygon region and set_histogram_requirement, let the region_histogram_data in queue:

    Register a session
      ✓ check connection (2ms)
      ✓ Get basepath (441ms)
      Go to "set_QA" folder and open image "h_m51_b_s05_drz_sci.fits"
        ✓ Preparation: Open image (3143ms)
        ✓ (Step 1) Set a many points polygon region: (2ms)
        ✓ (Step 2) Send sequent setHistogramRequirements with different numBins, to simulate the user drag the Number of bins in the setting of Histogram (19597ms)
        ✕ (Step 3) Check the receiveing RegionHistogramData * 12  (4ms)

  ● Testing the large image with multi-polygon region and set_histogram_requirement, let the region_histogram_data in queue: › Register a session › Go to "set_QA" folder and open image "h_m51_b_s05_drz_sci.fits" › (Step 3) Check the receiveing RegionHistogramData * 12 

    expect(received).toEqual(expected) // deep equality

    Expected: 4779
    Received: 4853

      183 |                     expect(RegionHistogramData.config.fixedBounds).toEqual(assertItem.ResponseRegionHistogramData.config.fixedBounds);
      184 |                     expect(RegionHistogramData.config.fixedNumBins).toEqual(assertItem.ResponseRegionHistogramData.config.fixedNumBins);
    > 185 |                     expect(RegionHistogramData.config.numBins).toEqual(assertItem.numBinsArray[index]);
          |                                                                ^
      186 |                     expect(RegionHistogramData.histograms.numBins).toEqual(assertItem.numBinsArray[index]);
      187 |                     expect(RegionHistogramData.histograms.bins[0]).toEqual(assertItem.firstBinsArray[index]);
      188 |                 })
      at map (src/test/SET_HISTOGRAMCONFIG_QUEUE.test.ts:185:64)
          at Array.map (<anonymous>)
      at Object.<anonymous> (src/test/SET_HISTOGRAMCONFIG_QUEUE.test.ts:178:45)

Plus there is a small type in the Step 3 text; receiveing > receiving

Also, thePER_CUBE_HISTOGRAM_CANCELLATION problem on only Ubuntu 20.04 and 22.04 continues. It is strange that it only happens on Ubuntu now. Also, it happened on the resume contour branch so seems it is not related to any modifications to this branch. It might be something that has changed in the carta-backend? (It is not showing up on the daily ICD tests yet because those are still running on an older copy of the 'dev' branch). Therefore perhaps we should post it as a new ICD-RxJS issue?

Before that, regarding the timeout, the output log text still says "10000ms" even though the timeout variable was increased to 30000ms, so maybe you could fix that in this branch first?

✕ (Step1) "SDC335.579-0.292.spw0-channel-cutted.line.fits" REGION_HISTOGRAM_DATA should arrive completely within 10000 ms: (30002ms)

[acdo2002]

@ajm-asiaa About SET_HISTOGRAMCONFIG_QUEUE.test.ts, I changed the expect method (to be more flexible, no sequent is required, the receive number is match the given value inside the array). please check whether it can pass on the RHEL8. Also fixed the typo in step 3

About PER_CUBE_HISTOGRAM_CANCELLATION, I fixed the step1 description. But I cannot produce the failure on my Ubuntu 22.04 Desktop, it passes all the time... (from the resume jenkins record, it looks the backend dead then the ICD-RxJS does not receive the first RegionHistogramData) Can it manually reproduce? because the resume branch does not change anything related to the RegionHistogramData. And they are all the same to the current ICD-RxJS dev branch. If there is a problem, it should fail on daily Jenkins too. I would suggest to see tomorrow's Jenkins' result.