package information not showing up in JSON

lee1043 commented 5 years ago

Before the pmp update, bottom of my JSON created using below commend

JSON = pcmdi_metrics.io.base.Base(os.path.join(json_file)
JSON.write(my_dict, json_structure=["model","realization","reference","mode","season","statistic"],
               sort_keys=True, indent=4, separators=(',', ': '))

was looking like below:

    "json_version": 3.0,
    "provenance": {
        "commandLine": "run_psd_loop.py -p my_Param.py",
        "conda": {
            "DefaultEnvironment": "/export_backup/lee1043/anaconda2/envs/pmp_nightly_20180124",
            "IsPrivate": "False",
            "Platform": "linux-64",
            "PythonVersion": "2.7.13.final.0",
            "RootEnvironment": "/export_backup/lee1043/anaconda2  (writable)",
            "Version": "4.3.24",
            "buildVersion": "not installed",
            "envVersion": "4.3.24"
        },
        "date": "2018-03-22 15:07:58",
        "openGL": {
            "GLX": {
                "client": {},
                "server": {}
            }
        },
        "osAccess": false,
        "packages": {
            "CDP": "2017.09.07",
            "PMP": "1.1.2.2017.06.30.19.40.66ce8fec99c4e1d404dc0e5f25d4e6f8e6339f66",
            "PMPObs": null,
            "blas": null,
            "cdms": "2.12.2017.11.30.06.06.14b9b9380c14680498a0ba8e9aefb4c84d2756be",
            "cdtime": "2018.01.24",
            "cdutil": "2.12.2017.11.16.21.46.e717f782e07d0a187f2b616a408103af20ceb57f",
            "clapack": "3.2.1",
            "esmf": "7.0.0",
            "esmpy": "7.0.0",
            "genutil": "2017.11.15",
            "lapack": "3.6.1",
            "matplotlib": "2.1.2",
            "mesalib": null,
            "numpy": "1.13.1",
            "python": "2.7.14",
            "vcs": "2.12.2018.01.24.16.21.d67c83aa153d32062df5c20f0c68e5a6f2103e8f",
            "vtk": "7.1.0.2.12"
        },
        "platform": {
            "Name": "crunchy.llnl.gov",
            "OS": "Linux",
            "Version": "2.6.32-696.23.1.el6.x86_64"
        },
        "userId": "lee1043"
    }
}

However, after I update pmp nightly to recent (version below), I got below, which is loosing conda and package info:

$ conda list pcmdi_metrics
# Name                    Version                   Build  Channel
pcmdi_metrics             1.2.2018.11.15.19.02.g9cf5870    pyh39e3cac_0    pcmdi/label/nightly

    "json_structure": [
        "model",
        "realization",
        "reference",
        "mode",
        "season",
        "statistic"
    ],
    "json_version": 3.0,
    "provenance": {
        "commandLine": "run_psd_loop.py -p my_Param.py",
        "conda": {},
        "date": "2018-12-06 15:56:36",
        "openGL": {
            "GLX": {
                "client": {},
                "server": {}
            }
        },
        "osAccess": false,
        "packages": {},
        "platform": {
            "Name": "crunchy.llnl.gov",
            "OS": "Linux",
            "Version": "2.6.32-696.30.1.el6.x86_64"
        },
        "userId": "lee1043"
    }
}

@doutriaux1 any idea? @gleckler1 @durack1 any similar experience?

doutriaux1 commented 5 years ago

I'll take a look. Thanks for spotting this!

gleckler1 commented 5 years ago

(pmpn_0718) bash-4.1$ parallelize_driver.py --driver mean_climate_driver.py -p cmip5_historical_pmp_parameterfile.py (37, ' ', [u'ACCESS1-0', u'ACCESS1-3', u'BNU-ESM', u'CCSM4', u'CESM1-BGC', u'CESM1-CAM5', u'CESM1-CAM5-1-FV2', u'CESM1-FASTCHEM', u'CESM1-WACCM', u'CNRM-CM5', u'CNRM-CM5-2', u'CSIRO-Mk3-6-0', u'CanCM4', u'CanESM2', u'GFDL-CM2p1', u'GFDL-CM3', u'GFDL-ESM2G', u'GFDL-ESM2M', u'HadGEM2-AO', u'IPSL-CM5A-LR', u'IPSL-CM5A-MR', u'IPSL-CM5B-LR', u'MIROC-ESM', u'MIROC-ESM-CHEM', u'MIROC4h', u'MIROC5', u'MPI-ESM-LR', u'MPI-ESM-MR', u'MPI-ESM-P', u'MRI-CGCM3', u'MRI-ESM1', u'NorESM1-M', u'NorESM1-ME', u'bcc-csm1-1', u'bcc-csm1-1-m', u'fio-esm', u'inmcm4'])

/export_backup/gleckler1/processing/metrics_package/my_test/parameter_files/mean_climate/pcmdiobs_info_dictionary.json Traceback (most recent call last): File "/export_backup/gleckler1/anaconda3/envs/pmpn_0718/bin/parallelize_driver.py", line 4, in import('pkg_resources').run_script('pcmdi-metrics===v1.2-41-g62d52b3', 'parallelize_driver.py') File "/export_backup/gleckler1/anaconda3/envs/pmpn_0718/lib/python2.7/site-packages/pkg_resources/init.py", line 666, in run_script self.require(requires)[0].run_script(script_name, ns) File "/export_backup/gleckler1/anaconda3/envs/pmpn_0718/lib/python2.7/site-packages/pkg_resources/init.py", line 1460, in run_script exec(script_code, namespace, namespace) File "/export_backup/gleckler1/anaconda3/envs/pmpn_0718/lib/python2.7/site-packages/pcmdi_metrics-v1.2_41_g62d52b3-py2.7.egg/EGG-INFO/scripts/parallelize_driver.py", line 81, in

AttributeError: 'PMPParameter' object has no attribute 'granularize'

From: Charles Doutriaux notifications@github.com Reply-To: PCMDI/pcmdi_metrics reply@reply.github.com Date: Thursday, July 18, 2019 at 12:48 PM To: PCMDI/pcmdi_metrics pcmdi_metrics@noreply.github.com Cc: gleckler1 pgleckler@llnl.gov, Mention mention@noreply.github.com Subject: Re: [PCMDI/pcmdi_metrics] package information not showing up in JSON (#587)

Closed #587https://github.com/PCMDI/pcmdi_metrics/issues/587 via 04e449dhttps://github.com/PCMDI/pcmdi_metrics/commit/04e449dcc64282b0ced1fe7ccd69623ecca60ce0.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/PCMDI/pcmdi_metrics/issues/587?email_source=notifications&email_token=ABCXVLJXIS2BWOVZ6YB6XRLQADCGBA5CNFSM4GI6GML2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOSSSNEVI#event-2493829717, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABCXVLNYNVZICZWXLIPO56LQADCGBANCNFSM4GI6GMLQ.

PCMDI / pcmdi_metrics

package information not showing up in JSON #587