Replacing python.shelve with json

[ledm]

The python module shelve has some limitations:

• It's not great with disk errors,
• It's not great for parrallelisation
• it's confusing for some users
• It's python exclusive
• iot's not that portable.
• Shelve now writes three files instead of one - making everything a bit hard.

I've also not been great at implementing it. My pattern has been:

sh = shelve,open('path')
sh[field] = data
sh.close()

but the preferred style uses context managers:

with shelve.open(path, mode="w") as file:
    file[field] = data

This is safer, clearer and more reusable.

Step 1 is to move all the shelve writes into separate functions.

Step 2 is to convert the shelve open commands into context managers.

Step 3 is to write a json version of the separate function.

Step 4 is to write a shelve to json conversion function.

Step 5: Write some tests?

[ledm]

Working on this here: https://github.com/valeriupredoi/bgcval2/tree/dev_fixing_shelve_opens

Requires PR #124

[ledm]

Okay, made some progress on changing the shOpen pattern from:

sh = open(filename)
sh['data'] = data
sh.close()

to the "correct" pattern:

with open(filename) as sh:
    sh['data'] = data

Early testing seems to have worked. As discussed via email, I don't think it's worth changing every instance, as several scripts in bgcval2 are not used for most purposes.