JustinGOSSES / wellio.js

JavaScript for converting well-log standard .las file format to json format
https://justingosses.github.io/wellio.js/
MIT License
34 stars 5 forks source link

Document 3 ways of Lasio-to-Wellio via Jupyter #48

Closed dcslagel closed 4 years ago

dcslagel commented 4 years ago

This change demonstrates 3 ways to reading from Lasio generated LAS/JSON to Wellio.js/Wellioviz.js within a Jupyter notebook.

I included a markdown version of the ipynb file. It is helpful set 'display the rich diff' setting in-order to read the markdown file easily.

After generating a LAS structure in Lasio, output the structure to a LAS 2.0 file with the following code example, then one of the subsequent 3 methods can be used in Jupyter to successfully read the structure into wellio.js format.

Create LAS structure, and set LAS version to 2.0 by writing out 'scratch_v2.las'

# python
las = lasio.LASFile()

las.well.DATE = datetime.today().strftime('%Y-%m-%d %H:%M:%S')
las.params['ENG'] = lasio.HeaderItem('ENG', value='Kent Inverarity')
las.params['LMF'] = lasio.HeaderItem('LMF', value='GL')
las.other = 'Example of how to create a LAS file from scratch using lasio'
depths = np.arange(10, 50, 0.5)
synth = np.log10(depths)*5+np.random.random(len(depths))
synth[:8] = np.nan
las.add_curve('DEPT', depths, unit='m')
las.add_curve('SYNTH', synth, descr='fake data')
# Write internal structure to a LAS file
las.write('scratch_v2.las', version=2)
# Create an in-memory lasio-json structure
json_images = json.dumps(las, cls=lasio.JSONEncoder)

Lasio LAS access/translation methods

  1. Translate the Lasio LAS structure to an in-memory JSON structure, then translate this structure to Wellio.js json via: JSON.parse() -> wellio.lasio_obj_2_wellio_obj(). This is the most flexible method because it doesn't require interacting with the host file system.
// node
let lasio_obj = '';
let wellio_obj = '';

try {
    lasio_obj = JSON.parse(json_images);
    wellio_obj = wellio.lasio_obj_2_wellio_obj(lasio_obj);
} catch (e) {
    console.log('[');
    console.log(e.name + ":: " + e.message);
    console.log(']');
}
console.log(wellio_obj);
  1. Have Lasio dump a Lasio-JSON encoded version to a file ('data.json'). Then read this Lasio-JSON file into Wellio and translate it to Wellio JSON. The example demonstrated in the markdown and ipynb files show how to obtain the current directory. The current directory is needed to find and read the file successfully.
# python
las_json_dict =json.loads(json_images)
with open('data.json', 'w') as outfile:
    json.dump(las_json_dict, outfile)
// node
const path = require('path');
let mydir = process.env.PWD;
let myfile = mydir + path.sep + 'data.json';

let lasio_json_str = '';
let lasio_obj_2 = '';
let wellio_obj_2 = '';

try {
    lasio_json_str = wellio.read_lasio_json_file(myfile);
    lasio_obj_2 = JSON.parse(lasio_json_str);
    wellio_obj_2 = wellio.lasio_obj_2_wellio_obj(lasio_obj_2);
} catch (e) {
    console.log('[');
    console.log(e.name + ":: " + e.message);
    console.log(']');
}

console.log(wellio_obj_2);
  1. Have Wellio.js read the LAS file that Lasio wrote. In these examples the file is named 'scratch_v2.las'. The example in the markdown and .ipynb files show how to obtain the current directory. The current directory is needed to find and read the file successfully.
// node
const path = require('path');
let mydir_3 = process.env.PWD;
let myfile_3 = mydir + path.sep + 'scratch_v2.las';

let las_str_3 = '';
let wellio_obj_3 = '';

try {
    las_str_3 = wellio.loadLAS(myfile_3);
    wellio_obj_3 = wellio.las2json(las_str_3);
} catch (e) {
    console.log('[');
    console.log(e.name + ":: " + e.message);
    console.log(']');
}
console.log(wellio_obj_3);

Summary These have been tested on a local jupiter instance. They don't require any changes to Wellio.js to work. So for issue #40, this would resolve the issue just by implementing one of these methods in Jupyter notebooks. Could you check these methods work for you. If they do should we close issue 40 ? (We could merge this branch in-order to have these examples available and documented).

Thank you,

DC

JustinGOSSES commented 4 years ago

This looks awesome. I had some problems but I think they were due to trying this on computer that initially didn't have wellio and wellioviz installed via npm.

In the notebook, I think we need to add cells for #npm.install('wellio') and #npm.install('wellioviz') They can be commented out initially and only run if not already installed.

JustinGOSSES commented 4 years ago

It also looks like I have a problem in latest version of wellioviz that I need to fix with an extra } before it will work here. I'll give that a look.

JustinGOSSES commented 4 years ago

pixiedust is a little frustrating with the inability to re-associated variables with a name once used once and needing to restart the kernal. Maybe we should note that in the notebook explicitly.

JustinGOSSES commented 4 years ago

@kinverarity1 do you have any thoughts on this from the lasio perspective?

kinverarity1 commented 4 years ago

It all looks very impressive! I would stick to option 1 or 2. The main purpose of lasio's JSON output is serialization and interaction with things like wellio so use that if it suffices?

dcslagel commented 4 years ago

Commit ac18d9a adds steps to install Wellio.js and Wellioviz.js only if they are not installed. Note: I cleared the result outputs from this checkin.

Commit 7039546 adds a note at the top to restart the Jupyter Kernal when needing to re-init pixiedust_node variables.

Let me know if this needs some additional changes/improvements to reach approval for merging.

Thank you, DC

JustinGOSSES commented 4 years ago

When I try to run it I'm getting errors for anything wellioviz:

/Users/justingosses/node/node_modules/wellioviz/dist/index.js:1328 catch{ ^ SyntaxError: Unexpected token { ReferenceError: wellioviz is not defined at repl:1:13 at ContextifyScript.Script.runInContext (vm.js:53:29) at REPLServer.defaultEval (repl.js:241:29) at bound (domain.js:301:14) at REPLServer.runBound [as eval] (domain.js:314:12) at REPLServer.onLine (repl.js:433:10) at emitOne (events.js:115:13) at REPLServer.emit (events.js:210:7) at REPLServer.Interface._onLine (readline.js:278:10) at REPLServer.Interface._normalWrite (readline.js:419:12)

JustinGOSSES commented 4 years ago

I fixed the problem in wellioviz with the try catch statement forgetting the err part of catch(err). This is not fixed in version 0.0.22.

JustinGOSSES commented 4 years ago

However, when wellioviz handles multiple logs at the same time, it uses well name to tell them apart. This is a problem as it seems the JSON encoder of LASIO drop well header information???

It doesn't in this example notebook, https://github.com/kinverarity1/lasio/blob/master/notebooks/exporting%20a%20LAS%20file%20to%20JSON.ipynb but it seems to in the current version of LASIO?

For instance, if I work with this example las file that has well name information: https://github.com/JustinGOSSES/wellio.js/blob/master/assets/00-01-01-073-05W5-0.LAS and I load it like: import os
l = lasio.read(os.path.join("../assets/00-01-01-073-05W5-0.LAS")) json_images_l = json.dumps(l, cls=lasio.JSONEncoder) print(json_images_l)

In the output string of JSON, the metadata fields are all blank even though the original LAS file had data in most of them: "metadata": {"Version": [{}, {}], "Well": [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}], "Curves": [{}, {}, {}, {}, {}, {}], "Parameter": [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}], "Other": ""} The data part of the JSON looks fine.

The code in question in lasio is probably in https://github.com/kinverarity1/lasio/blob/master/lasio/las.py on lines 901-908.

kinverarity1 commented 4 years ago

@JustinGOSSES what version of lasio is being imported? I hope this is an old version. I remember us fixing a bug at some point where the output of "print" was to show {} incorrectly. I've just re-run the notebook you mention and it seems to be OK:

https://github.com/kinverarity1/lasio/blob/d480b249ecb796ea214aa6dfbddfc27ce54e26d1/notebooks/exporting%20a%20LAS%20file%20to%20JSON.ipynb

JustinGOSSES commented 4 years ago

Yep, I was using 0.24.1 which is a bit out of date. I remembered to check javascript libraries for being to date and forgot lasio!

JustinGOSSES commented 4 years ago

Let's merge this and then keep working on it to get all the way to wellioviz visualization. I'm unsure whethe pixiedust is the best way to get there or some modification of this: https://www.stefaanlippens.net/jupyter-custom-d3-visualization.html or this: https://medium.com/@stallonejacob/d3-in-juypter-notebook-685d6dca75c8