APS3 Suite ODB tasks - Githubissues

penguian commented 9 years ago

keyword_APS3_Suite_ODB | by ycx548@nci.org.au

Need DA group to work on OPS rose stem

add bombufr apps for all obs types (can copy from kma's)
use pre-build OPS32 exe to create ODB from bufr files (for G & R)
create known good outputs (KGO)

Background: Adam Maycock and I have tried to create ODB from BOM bufr files, with limited success. Now we have OPS32 available, it is time to revisit this. It is important that we have this task done asap, estimated two weeks.

Issue migrated from trac:213 at 2024-01-31 18:15:31 +1100

penguian commented 9 years ago

ycx548@nci.org.au changed component from ACCESS model to BOM ACCESS

penguian commented 9 years ago

ycx548@nci.org.au commented

Moved this ticket to "BOM ACCESS" Peter pointed out some phase in original request may not be very clear.

two weeks is the amount time required to do this work. Not within 2 weeks of the request is made. Peter will organize a meeting on this.

penguian commented 9 years ago

ycx548@nci.org.au commented

EWG has built OPS32 exe with rose stem, and the bin directory is in /projects/access/nwpdir/share/APS2/OPS/ops_32.0.0/build/bin Currently, the built was done using odb/30.0.0. However some limited test indicated exe here does allow for some basic testing, such as createODB. In the coming week, once Tan is happy with new ODB module (see ticket 212 for details) the bin directory will be updated.

EWG will be asking UKMO for a shared OPS32 branch to be created.

penguian commented 9 years ago

ycx548@nci.org.au commented

The OPS branch is https://code.metoffice.gov.uk/trac/ops/browser/main/branches/dev/yixiao/r175_bom_nci (As emailed to Fabrizio about 10 days ago.

penguian commented 9 years ago

ycx548@nci.org.au commented

To assist the work along this line, I have created tasks about creating odb and process obs in the rose stem. I also tested ATMS and surface. ATMS appears to have works, surface clearly failed. Other types are not tested, and apps needs further modification. However with examples provided it should be much easier to modify the apps. [20] Added an attachment to Summarize the status of CreateODB in OPS32 [12] Additional fixes (See Tan's comments below) have been accepted by UKMO, and merged into trunk. A Shared branch has been created after the merge from trunk@858 at https://code.metoffice.gov.uk/svn/ops/main/branches/dev/Share/r858_165_bom_nci_ops_32.1.0. It is used to build APS3 OPS excutables. The build directory is under /projects/access/nwpdir/share/APS3/OPS/vn32.1.0

Remaining problems:

sonde and amv's problem in ODB may have been fixed by Tan. Need to check in OPS (Xiao)
Disabled obs types: CrIS,MtsatClear,Windsat are to be fixed by Jin

[29] Big Thanks to Tan has worked out (very complicated) all Obstypes for creating ODB (not Mtsatclear), plus a bug fix in OPS code.

Xiao has tested and continued to test in collaboration with UKMO. Part of our modification has passed review at UK, and changes have been put on the OPS trunk. I have created a branch under Share (ticket #165) https://code.metoffice.gov.uk/svn/ops/main/branches/dev/Share/r686_165_bom_nci_ops_32.0.0 the bin directory is under ~access/nwpdir/APS3/OPS This directory will be updated when our coed passed review fully.

'''[12]

I have spent sometime on rose-stem createODB tasks, and tidied up apps, and added a few more.

I have also started to create data for KGO. Inputs and KGO are under /g/data/access/OPS

This is where UM KGO are at.

For obs types that you are responsible for, please check if the created ODB are valid under KGO, and let others know outcomes '''

penguian commented 9 years ago

ycx548@nci.org.au changed _comment0 which not transferred by tractive

penguian commented 9 years ago

ycx548@nci.org.au changed _comment1 which not transferred by tractive

penguian commented 9 years ago

ycx548@nci.org.au changed _comment2 which not transferred by tractive

penguian commented 9 years ago

ycx548@nci.org.au changed _comment3 which not transferred by tractive

penguian commented 9 years ago

ycx548@nci.org.au changed _comment4 which not transferred by tractive

penguian commented 9 years ago

ycx548@nci.org.au changed _comment5 which not transferred by tractive

penguian commented 9 years ago

ffb548@nci.org.au commented

Managing rose-stem tests

To make more flexible the use of rose-stem tests the following changes should be made:

1) OPS_CYCLE_TIME must be defined in "rose-stem/site/nci/common/suite-env.rc" as done by Met Office and not in the rose-app.conf of the single app.

This means that change the basedate is simply localized in 1 file only and we do not have to do multiple times in every single rose-app.conf

2) OPS_BUFR_INPUT_DIR is now specified in the rose-app.conf of the single app as follows:

OPS_BUFR_INPUT_DIR=/g/data1/dp9/itf/arbufr/2015031200

Changes which we must introduce: --> the path = "/g/data1/dp9/itf/arbufr/" should be writable for everyone --> OPS_CYCLE_TIME should be introduced instead of manually set the basedate

penguian commented 9 years ago

ffb548@nci.org.au uploaded file bufrSubtypes.xlsx (13.1 KiB)

Updated table bufr/obs type

penguian commented 9 years ago

sjr548@nci.org.au commented

Possibly better to have a rose stem input data archive separate from other archives, with all data copied to it from other archives. Archives used to provide input for routine cycling, like /g/data1/dp9/itf/arbufr, probably shouldn't be universal writable for safety.

penguian commented 9 years ago

@peter.steinle@bom.gov.au commented

Replying to ACCESS-NRI/accessdev-Trac-archive#213 (comment:7):

Possibly better to have a rose stem input data archive separate from other archives, with all data copied to it from other archives. Archives used to provide input for routine cycling, like /g/data1/dp9/itf/arbufr, probably shouldn't be universal writable for safety.

Agree - and am uneasy about world write in general. Can understand the ease of use, but need to be meticulous about recording changes and which commands are issued within these directories. This is not something we have a good record of doing. Is there another option (sudo?) which helps inadvertent destruction?

penguian commented 9 years ago

ffb548@nci.org.au commented

Replying to sjr548 and pjs548

I agree with the solution you describe when our rose-stem tests will be well-established and hopefully we will also have an official OPS manager.

My general thought is that in any development phase everyone should have a more flexible environment to manage input/output

penguian commented 9 years ago

ffb548@nci.org.au commented

Managing rose-stem tests

Also OPS_UMBACK_LIST is a path we mitgh manage in a more general way:

OPS_UMBACK_LIST=/short/dp9/ycx548/data/testdata/ops/cx.background.201503111800

No user data dir
Using OPS_CYCLE_TIME wherever is possible

example from Met Office:

OPS_UMBACK_LIST=$OPSDIR/AcceptanceTests/startfiles/$OPS_CYCLE_TIME_global/qwqg06.pp_006

penguian commented 9 years ago

ffb548@nci.org.au commented

rose stem-tests

I introduced some changes in my OPS branch "r272_ops_at_bom" (copy of Xiao's branch r175_bom_nci) to localize the set up of the rose-stem tests in 1 file only. More adoptable to changes and more user-oriented (in my opinion).

Changes are:

1) 2 new env variables to set up base data and data source for the tests:

in "rose-stem/site/nci/common/suite-env.rc", I defined:

ROSE_STEM_DATA_NCI = /g/data/dp9/da/rosestem_data OPS_CYCLE_TIME_nci = 2015031200

2) env variables in 1 are used in the "rose-app.conf" of the single rose-stem test to define the following necessary inputs:

OPS_CYCLE_TIME=$OPS_CYCLE_TIME_nci OPS_UMBACK_LIST=$ROSE_STEM_DATA_NCI/cx_bkg/$OPS_CYCLE_TIME_nci/cx.bkg OPS_BACKERR=$ROSE_STEM_DATA_NCI/bgerr/$OPS_CYCLE_TIME_nci/bgerr OPS_BUFR_INPUT_DIR=$ROSE_STEM_DATA_NCI/bufr/$OPS_CYCLE_TIME_nci

3) To implement 1 and 2, I arbitrary created a data dir which contains the nessary data input for the rose-stem tests. The data dir is (ROSE_STEM_DATA_NCI):

"/g/data/dp9/da/rosestem_data"

accessible to any user and organized as follows:

bgerr/
   YYYYMMDDHH
bufr/
   YYYYMMDDHH  
cx_bkg/
   YYYYMMDDHH

At any time a new base date (OPS_CYCLE_TIME_nci) can be tested in rose-stem simply coping the right source of data in ROSE_STEM_DATA_NCI. For instance APS2 bgerr/cx_bkg can be copied from /g/data3/rr4/samnmc/access-g/

If you are happy with these changes, [175] Xiao's branch should be updated (merged with [272]), e.g.:

https://code.metoffice.gov.uk/svn/ops/main/branches/dev/yixiao/r175_bom_nci

https://code.metoffice.gov.uk/svn/ops/main/branches/dev/fabriziobaordo/r272_ops_at_bom

penguian commented 9 years ago

ffb548@nci.org.au changed _comment0 which not transferred by tractive

penguian commented 9 years ago

ffb548@nci.org.au changed _comment1 which not transferred by tractive

penguian commented 9 years ago

ycx548@nci.org.au commented

I have edited the ticket yesterday [12] about the progress I made on this and further work required for experts to check ODB's KGO. But no email was sent out about my update. So now I try modify the ticket see if message can be sent out.

penguian commented 9 years ago

sjr548@nci.org.au commented

Further information on this issue can be found linked off MOSRS ticket 1, in https://code.metoffice.gov.uk/trac/roses-u/wiki/ticket/1/TicketSummary/BufrCreateConformingToUKBufrFormat.

penguian commented 9 years ago

ffb548@nci.org.au commented

My OPS branch has been merged with Susan's in order to have 1 branch which contains the rose-stem tests for ODBcreate developed so far:

https://code.metoffice.gov.uk/svn/ops/main/branches/dev/fabriziobaordo/r272_ops_at_bom

They are

Developed/tested by Susan: rose-stem/app/ops_createbufrdirodb_nci_aircraftsonde_global rose-stem/app/ops_createbufrdirodb_nci_gpsro_global/rose-app.conf rose-stem/app/ops_createbufrdirodb_nci_mtsatclear_global rose-stem/app/ops_createbufrdirodb_nci_satwind_global/rose-app.conf rose-stem/app/ops_createbufrdirodb_nci_satwind_global/suite-app-run.rc rose-stem/app/ops_createbufrdirodb_nci_surface_global/rose-app.conf rose-stem/app/ops_createbufrdirodb_nci_surface_global/suite-app-run.rc

Developed/tested by Fabrizio: rose-stem/app/ops_createbufrdirodb_nci_atms_global/suite-app-run.rc rose-stem/app/ops_createbufrdirodb_nci_iasi_global/suite-app-run.rc rose-stem/app/ops_createbufrdirodb_nci_airs_global/suite-app-run.rc rose-stem/app/ops_createbufrdirodb_nci_atovs_global/suite-app-run.rc

I have also created a common dir where to store the logs which are generated by each rose-stem ODBcreate test. The dir is as follows:

/g/data/dp9/da/rosestem_logs/odbcreate airs/ atms/ atovs/ iasi/

These logs might provide a reference to list the problems we have with each ODBCreate. In my test cases:

Looks OK == I check that data are written into the ODB. My basic check is as follows

Number of obs in job.stats MUST BE THE SAME of the result of the following query odbsql -q 'select count(lat) from hdr, sat where satellite_identifier == 224'

Does not work == ODB empty

ATMS looks OK IASI looks OK, but a strange error in job.out (TO BE CHECK) AIRS does not work ATOVS does not work

penguian commented 9 years ago

ttl548@nci.org.au commented

Jin approached me late last week and asked for the status of my work re createODB, with a suggestion that we could share the workload (Jin also kindly volunteered to look at MTSatClear).

from Xiao: The branch that included our work at the time and been merged on OPS truck, and I have tested has been informed all of you via ticket 165 and it is
r686_165_bom_nci_ops_32.0.0 the bin directory is under ~access/nwpdir/APS3/OPS

and summary of my work,

There is a bug in the OPS code that prevents bufr with section 2 to be processed. Fixed with

OpsMod_TURBO/Ops_Convert1dbufr.f90

!BOM 14/10/2015 Section2Present = bit 1 of octet 8 set to 1, !BOM ie. 8th byte = 128 dec !BOM IF (ICHAR(SingleBUFR(BufrPointer + 10:BufrPointer + 10)) > 128) THEN IF (ICHAR(SingleBUFR(BufrPointer + 8:BufrPointer + 8)) >= 128) THEN Section2Present = .TRUE. END IF

This fixed up most sat data except AMV, CRIS, MTSATCLEAR, WINDSAT

sfc/aircraft/sonde

These are BOM's specific bufr format retrieved from rtdb. For each obs type we will need to add a definition in

tables/elements_index/OBSTYPE

where OBSTYPE = AIREPS, BUOY, METARS, PILOT, SATWIND, SHPSYN, TEMP, WNDPRO

CRIS

in pp/*nci_cris_global/rose-app.conf

[file:ops_extract_control/CrIS.nl] source=namelist:extractcontrolnl{cris}

[namelist:extractcontrolnl{cris}] buffersize=5000 bufrbuffersize=200 maxbatchessubtype=24 maxbufrbatchessubtype=12 metdbkeywords='PLATFORM 224' metdbsubtype='CRIMSS' timewindowafter=179 timewindowbefore=180

in 2. MetDB_Bufr_Retrieval/apps/create_mdblseq/mkmdblseq.sh

CRIMSS ${METDB_BASE_DIR}/tables/bufr_localseq/iasi

in 3. OpsMod_TURBO/OpsMod_TURBO.f90

! MetDB BUFR Extraction details !BOM INTEGER, PARAMETER :: LenRep = 28672 !- size of CREP (o/p from MDB), ! MUST BE 28K !BOM 28/10/2015 CRIMSS msg len can exceed 28K, need to ignore restriction in !BOM MDB INTEGER, PARAMETER :: LenRep = 50000 !- size of CREP

...

! To ensure a new subtype is BUFR extracted add it below. !BOM 26/10/2015 added def for CRIMSS CHARACTER(len=8),PARAMETER :: BUFRSubType(NumBUFRSubtypes) = & (/"AIRS ","ATOVSG ","ATOVSL ","ESACMW ","ESACSWVW", & "ESAHRVW ","ESAHRWVW","ESAURA ","ESAUWI ","ATMS ", & "GPSIWV ","SEAWINDS","SSMI ","SSMIS ", & "UKMOSST ","CRIMSS ","MODIS ","IASIG ","IASIL ", & "AIRSWF ","MSGCSR ","MSGRAD ","ASCAT ", & "MSGWINDS","JMAWINDS","AIRSMS ","MSGRADFD","AIRSL ", & "AMSR2 ","IASIHR ","ESACSR ","GOESCSR ","RADRRATE", & "MSGAOD ","KMAWINDS","COMSCSR ","MSGRADUK", & "MSGCTP ","MSGCTPUK","MSGCTPFD","ASCATHR ","ASCATCO ","AMDARS ","AIREPS ", & "LNDSYN ","SHPSYN ","BUOY ","METARS ","TEMP ", & "PILOT ","WINPRO ","DROPSOND","GPSRO ","GOESBUFR", & "STEREOMV"/)

With CRIS (399 radiance channels) the number of elements in a bufr message is 1057, and with 30 obs packed to a message the array size required is 30*1057 > 28672, hence the need to increase LenRep.

Note however this is not accepted by the MetOffice. We will either exclude assimilating CRIS and wait for the met office for a better solution, or to re-encode the bufr file with less obs packed to a message (say 15).

amv

in addition to elements_index/SATWIND, we will also need to change a few entries in tableb as some bufr ids are in conflict with ECMWF table

status.

amv: createODB is OK but the content of ODB is wrong sonde: high rejection rate cris: required alternative solution mtsatclear/windsat: yet to be done

penguian commented 9 years ago

@jin.lee@bom.gov.au commented

Jin tested OpsProg_CreateODB for obs_type=mtsatclear:

the build used is from the branch, fcm:ops.x/branches/dev/Share/r686_165_bom_nci_ops_32.0.0@709
Base date/time is 20150615T0600Z

The task fails with the following message in stderr,

OpsProg_CreateODB
FATAL ERROR
Bufr file or directory input not supported for subtype JMACSR
Error Summary: There has been 1 WARNING message.
gc_abort (Processor     0): Bufr file or directory input not supported for subtype JMACSR

The fact that the array variable, BUFRSubType(:) defined in OpsMod_TURBO/OpsMod_TURBO.f90 doesn't contain "JMACSR" seems to be causing this failure.

penguian commented 9 years ago

ttl548@nci.org.au commented

I had a closer look at element_index/SATWIND and realized that the met office have already defined the bufr sequences for both JMA/EUMETSAT (sequence 2) and GOESBUFR (sequence 5).

So I believe a proper solution is to re-produce the amv data in their original format, rather than the current format as a direct mapping from the content of rtdb.

This approach has the advantage of

Consistency in bufr input format for data retrieved from rtdb, and that of direct bufr input from their original met centres.
There is no need to change element_index/SATWIND as the bufr sequences have been defined by UKMO
There is no need to use our own tableb, as the re-constructed bufr uses bufr defs from UK's table

There is however an extra step to convert our amv bufr data to the original format used by other centres.

The program to do the amv conversion is on ngamai:$CWSHARE/bin/bufr_amv

Usage : bufr_amv inout.bufr output.bufr

The src code is in twister:~ttl/mars/bufr_amv. Let me know the appropriate place on accessdev/svn to put the code in.

NAME

    bufr_amv - converts BOM's amv bufr as retrieved from rtdb to their
    original bufr format

SYNOPSIS

    bufr_amv <input_bufr_file> <output_bufr_file>

DESCRIPTION

    converts Bureau's amv bufr files as retrieved from rtdb to their
    original bufr formats

    1. amv254msg.* -> EUMETSAT's MSGWINDS: IUV???_EUMG_yyyymmddhhmn

    2. amv254.* -> EUMETSAT's ESACMW: IXC???_EUMS_yyyymmddhhmn

    3. amv160.* -> NESDIS's GOESBUFR: J?CX??_KNES_yyyymmddhhmn

    4. amv34.* -> JMA's JMAWINDS: IUC???_RJTD_yyyymmddhhmn

    5. local amv encoded as NESDIS's

    as required by OPS32.0 elements_index/SATWIND bufr sequence 2 (jma/
    eumetsat), and bufr sequence 5 (local/nesdis)

SOFTWARE REQUIRED

    The program requires link to ECMWF's bufr decoding software

    https://software.ecmwf.int/wiki/display/BUFR/BUFRDC+Home   or
    https://software.ecmwf.int/wiki/display/EMOS/Emoslib

penguian commented 9 years ago

sjr548@nci.org.au commented

A while back, in an earlier version of the suite, I tried including WINDSAT in the OPS supported subtypes for bufr, just to see what happened. I just ran a comparison between the bufr file, and the ODB. The observation values are the same in both files. So it doesn't look like any further changes to the OPS should be necessary for WINDSAT.

penguian commented 9 years ago

ycx548@nci.org.au _uploaded file CreateODB_update.docx (35.4 KiB)_

penguian commented 8 years ago

sjr548@nci.org.au commented

An issue with JMAWINDS has been observed. Some observations with longitude of 180° are excluded due to precision error resulting in a value > 180° on conversion from single (in bufr) to double (in OPS). More information can be found here

penguian commented 8 years ago

ttl548@nci.org.au commented

I have been looking at the issue with AIRS for the case 20150428 00z and here is what I found:

number of messages = 2529 number of obs in a message # 35> total number of obs # 2529*35 88515 with average length of each message = 23000 (bytes/words?)

Now if we have LenRep = 28672 as required by MetDB, and set bufrbuffersize=200

we have a total array size of 28672*200 = 5734400

which can accommodate 5734400/23000 = 250 messages

or 250*35 = 8750 obs

or roughly one tenth of total obs as per Xiao's summary.

so for this case, the required bufrbuffersize # (2529*23000)/28672 2028

or solution is to set bufrbuffersize to 2500

in summary, set bufrbuffersize to

AIRS: 2500 IASI: 4000 ATOVS: 2500 GPSRO: 500

with the exception of CRIS, the average length of bufr message is 29000 > 28672. splitting the obs does not seem to work, so the only solution is to increase LenRep, or to use bufr2odb.

penguian commented 8 years ago

ttl548@nci.org.au commented

I had another attempt at bufr_split to fix the problem with CRIS (message size > 28672).

This time I break down the number of obs in a message, thus effectively cut down the message size.

bufr_split [-n N] input.bufr

with N = 2 we have CRIMSS_1 and CRIMSS_2 as output, each of which has message size roughly reduced by half (from ~30000 to ~15000)

createODB worked with all obs accounted for.

Note that with N > 12 you will need to increase maxbufrbatchessubtype in rose-app.conf as well (currently set to 12)

penguian commented 8 years ago

ycx548@nci.org.au _uploaded file CreateODB_update.20151203.docx (43.7 KiB)_

penguian commented 8 years ago

@jin.lee@bom.gov.au commented

BUOY elements index file fixed

There was a problem with the BUOY elements index file, which meant the values of dewpoint temperature at 2m and relative humidity in ODB were different to those in input Bufr files. This problem is now fixed by updating the elements index file for BUOY,

src/code/MetDB_Bufr_Retrieval/tables/elements_index/BUOY

I tested the change by running my secondary suite, u-aa858 and then testing using Metview and odbsql. E.g. I used Metview to examine Bufr messages 2462 (buoy identifier 31053) and then compared the values in the Bufr message with what's in ODB by running following ODB SQL queries,

odbsql -q 'select buoy_identifier,initial_obsvalue from hdr,body,conv where buoy_identifier=31053 and varno=$td2m'

and

odbsql -q 'select buoy_identifier,initial_obsvalue from hdr,body,conv where buoy_identifier=31053 and varno=$rh2m'

The changeset is,

https://code.metoffice.gov.uk/trac/ops/changeset/1114

penguian commented 8 years ago

sjr548@nci.org.au commented

At 20151207, pending changes to OPS branch to run tasks 1) BUOY/surface. pressure tendency/characteristic see MOSRS OPS ticket 222, changeset 1113 (Susan's branch) - will be put in trunk. 2) BUOY elements_index file modified, see MOSRS OPS changeset 1114 (Jin's branch). 3) WINDSAT, need to add WINDSAT to OpsMod_TURBO.f90 subtypes. Windsat looks ok. 3) CRIS, need to add CRIMSS to OpsMod_TURBO.f90 subtypes.

penguian commented 8 years ago

ttl548@nci.org.au commented

GPSRO:

added fix to MetDB_Bufr_Retrieval/lib/source/retpos.f90

! Counts for higher levels DO J=ILEV,JLEV+1,-1 ICOUNT = MCOUNT(NUMREP(J,ISEG)) !BOM 09/12/2015 MCOUNT can be zero for nested levels in GPSRO !BOM check to avoid division by zero If (ICOUNT > 0) Then IREP(J) = MOD(I,ICOUNT) + 1 I = I/ICOUNT Else IREP(J) = 1 I = 0 End If END DO

to fix a divide by zero error when the nested level number is zero. The obs is still written out to ODB but the data content is all issing.

==MTSATClear:==

to follow the procedure as per CRiS, ie.

set metdbsubtype='JMACSR' in rose-app.conf
add entry for JMACSR in MetDB_Bufr_Retrieval/apps/create_mdblseq/mkmdblseq.sh

JMACSR ${METDB_BASE_DIR}/tables/bufr_localseq/jmawinds

add entry for JMSCSR in BUFRSubType OpsMod_TURBO/OpsMod_TURBO.f90, note: need to increase size of NumBUFRSubtypes as well.
bufr_split modified to handle MTSATClear. Note some message lengths can be as large as 60K so need N # 4. Also suggest to set bufrbuffersize 500

penguian commented 8 years ago

sjr548@nci.org.au commented

GPSRO gives different results depending on the version of the mdb elements file. in APS2

Total number of GPSRO            =      696
 --------------------------------------------------------------------
 Total ODB GPSRO obs              =      696
 --------------------------------------------------------------------

Ops_QcStats
 2015  6 15  6  GPSRO               696 obs 

GPSRO    696 observations
         139 had prior reject of whole report and are not included below.
           0 were surplus (near-duplicate) reports and are not included below.
           0 had other reject of whole report, they are included below.

In APS3

Total number of GPSRO            =      696
 --------------------------------------------------------------------
 Total ODB GPSRO obs              =      696
 --------------------------------------------------------------------

Ops_QcStats
 2015  6 15  6  GPSRO            185849 obs 

GPSRO 185849 observations
       28843 had prior reject of whole report and are not included below.
           0 were surplus (near-duplicate) reports and are not included below.
           0 had other reject of whole report, they are included below.

APS3 has 3 additional level elements LEVL_LTTD LEVL_LNGD LEVL_STLT_AZMH

ACCESS-NRI / accessdev-Trac-archive

APS3 Suite ODB tasks #213

Managing rose-stem tests

Managing rose-stem tests

rose stem-tests