pscload V2 Component to load RASS data in psc db
Please note that psc-rass-loader
depends on deprecated JDK APIs, and the build
breaks from JDK 21 on.
This can be addressed by :
This ecosystem uses many independant components, some of which live an independant life in distinct repositories.
For each release of psc-ps-api
, the psc-compoents' distribution history
file will need to be updated with the new version, so that we can keep track of compatible component versions,
and go back to a previous working distribution if need be.
Whenever a version is ready for release, run the following commands on the main
branch (or on the maintenance branch if we're about to issue a production FIX). This should run on any shell, be it bash
, cmd
or if needed gitbash
.
mvn release:prepare -DautoVersionSubmodules=true -DtagNameFormat=@{version}
git push
git push origin <new_version_tag>
where <new_version_tag>
stands for the new version.
Eg to relase 1.0.1
:
mvn release:prepare -DautoVersionSubmodules=true -DtagNameFormat=@{version}
git push
git push origin 1.0.1
For trouble shooting see the DEBUG section.
To help qualification, two test datasets have been added (see below).
To use extract files from these datasets, set the psload key extract_download_url
in Vault for the instance to https://raw.githubusercontent.com/ansforge/psc-rass-loader/main/pscload/src/test/resources/<dataset_name>/<extract name>
. For example, to use the BaseExtract.zip
extract from dataset qualif-extracts, use : https://raw.githubusercontent.com/ansforge/psc-rass-loader/main/pscload/src/test/resources/qualif-extracts/BaseExtract.zip
This datasets consists in 5 extracts + 1 toggle file (see the qualif-extracts directory) around 100,000 data lines :
BaseExtract.zip
will trigger 20000 more updates). 20.000 deletes are 20% updated lines and willtrigger an alert.BaseExtract.zip
will trigger 20000 creates). 20.000 deletes are 20% deleted lines and will trigger an alert (but 20000 creates won't)This dataset consists in 3 extracts and 2 CSV files, in order (see the CT-AMAR directory):