spacepy / dbprocessing

Automated processing controller for heliophysics data
5 stars 4 forks source link

Force empty output products to null (Closes #95) #98

Closed jtniehof closed 2 years ago

jtniehof commented 2 years ago

This PR changes DButils.addProcess such that, if it is passed an empty string for the output product, it inserts null (None) into the database. This matches the handling of some other things which may be empty strings in the config file but null in the database, most notably extra arguments for processes. It fixes processes with no output product on postgresql and closes #95.

There is still some code in DButils.getChildTree to support databases (sqlite) where the column is an empty string, and those shouldn't be removed until #7 is closed (so that "version 1" database will not have the empty string there.)

PR Checklist

I'm not making a release note entry because I'm considering this part of the greater project of getting postgres support going...this most likely would have been found, for instance, if we had the unit tests going on postgres, which continues on my agenda.

Testing is in the unit test, but the functionality in postgres was tested explicitly using the PSP/ISOIS config, which has several processes of this sort:

createdb dbp -U postgres
export PGUSER=postgres
export PGDATABASE=dbp
python ./CreateDB.py -d postgresql dbp
python ./addFromConfig.py -m dbp ~/scm/isois/glue/dbp_config/psp_dbprocessing.conf 

Then verified with psql that the column was null:

$ psql -U postgres
dbp=# select * from process where output_timebase = 'RUN';
dnadeau-lanl commented 2 years ago

Sounds good thanks!