Electrofacies source code depreciated numpy "as_matrix" command

[cdmeyer23]

Stoked to keep working with this program!

I am attempting to run the wolfcamp_single example and I am running into an error: AttributeError: 'DataFrame' object has no attribute 'as_matrix'

Looking at the Electofacies definition when defining minibatch_input:

components = pd.DataFrame(data = pc.transform(X), index = df[not_null_rows].index)

minibatch_input = components.as_matrix()

it appears that "df.as_matrix()" has depreciated in a the newer version of pandas. I cannot seem to get the correct command to get the example to run. Could someone help?

[Philliec459]

You are working on wolfcamp_single.py?

I used this about a year ago and do not remember how I got it working, but on mine I not even see where pandas is loaded?

Best Regards,

---

E. Craig Phillips CEO and Chief Petrophysicist Crested Butte Petrophysical Consultants

459 Cisneros Lane Crested Butte, CO 81224 USA Office: +1 970-343-0730 Mobile: +1 970-343-0730 email: craig@cbpetro.com Website: www.cbpetro.com GitHub: https://github.com/Philliec459

The information contained in this electronic message is confidential, it constitutes a profesional and/or industrial secret in terms of the current legislation, and is intended for its recipient only. If you receive this message by mistake or if you are not the recipient thereof, please notify the sender and destroy it.

La informacion contenida en este mensaje de datos es confidencial, constituye un secreto industrial y/o profesional en terminos de la legislacion vigente y se encuentra dirigida exclusivamente al destinatario indicado en dicho mensaje. Si usted recibe esta informacion por error o si usted no es el destinatario del mensaje, favor de notificar al emisor, y destruyalo.

On Jul 9, 2020, at 11:15 AM, cdmeyer23 notifications@github.com wrote:

Stoked to keep working with this program!

I am attempting to run the wolfcamp_single example and I am running into an error: AttributeError: 'DataFrame' object has no attribute 'as_matrix'

Looking at the Electofacies definition when defining minibatch_input:

components = pd.DataFrame(data = pc.transform(X), index = df[not_null_rows].index)

minibatch_input = components.as_matrix() it appears that "df.as_matrix()" has depreciated in a the newer version of pandas. I cannot seem to get the correct command to get the example to run. Could someone help?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/toddheitmann/PetroPy/issues/7, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANSKYPA3YGCDWWCIBJDUV2DR2X3KNANCNFSM4OVZ3LDA.

[cdmeyer23]

@Philliec459 Yes, I am working with the wolfcamp_single.py

If you look at the electrofacies source code here: https://github.com/toddheitmann/PetroPy/blob/master/petropy/electrofacies.py it requires the import of numpy and pandas.

The example wolfcamp_single.py calls ptr.electrofacies on line 147:

logs = ptr.electrofacies(logs, f, electro_logs, 6, log_scale = ['RESDEEP_N'])

[Philliec459]

As I said, I set this up ages ago and do not have all the details.

I definitely see the pandas in this module. Thanks. Let me see if I can retrace my setup and see what I can come up with. I had some issues and solved them, but back then I was not documenting how I did what.

In the plot below I see the Electrofacies, but I have no idea of how I did this.

Did you use setup from Todd’s GitHub?

Best Regards,

---

E. Craig Phillips CEO and Chief Petrophysicist Crested Butte Petrophysical Consultants

459 Cisneros Lane Crested Butte, CO 81224 USA Office: +1 970-343-0730 Mobile: +1 970-343-0730 email: craig@cbpetro.com Website: www.cbpetro.com GitHub: https://github.com/Philliec459

The information contained in this electronic message is confidential, it constitutes a profesional and/or industrial secret in terms of the current legislation, and is intended for its recipient only. If you receive this message by mistake or if you are not the recipient thereof, please notify the sender and destroy it.

La informacion contenida en este mensaje de datos es confidencial, constituye un secreto industrial y/o profesional en terminos de la legislacion vigente y se encuentra dirigida exclusivamente al destinatario indicado en dicho mensaje. Si usted recibe esta informacion por error o si usted no es el destinatario del mensaje, favor de notificar al emisor, y destruyalo.

On Jul 9, 2020, at 1:32 PM, cdmeyer23 notifications@github.com wrote:

@Philliec459 https://github.com/Philliec459 Yes, I am working with the wolfcamp_single.py

If you look at the electrofacies source code here: https://github.com/toddheitmann/PetroPy/blob/master/petropy/electrofacies.py https://github.com/toddheitmann/PetroPy/blob/master/petropy/electrofacies.py it requires the import of numpy and pandas.

The example wolfcamp_single.py calls ptr.electrofacies on line 147:

logs = ptr.electrofacies(logs, f, electro_logs, 6, log_scale = ['RESDEEP_N'])

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/toddheitmann/PetroPy/issues/7#issuecomment-656310961, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANSKYPDYVBR4EJFOQU4BXITR2YLMNANCNFSM4OVZ3LDA.

[Philliec459]

I did install Petropy and I see that it resides in my …./Anaconda/Lib/site-packages/petropy subdirectory. It was either a pip or "python install setup.py" type of install that I did back in October of last year. Too long ago to remember the details.

I also see from the emails that they were working in petropy back in April. I have not upgraded and do not know if the changes are affecting you now or not. Below is my electrofacies.py if that is any help. Below that is my wolfcamp_single.py.

I do remember having to make some mods to get it to run, but did not take any notes back then on what I did to get it running. I could have removed some code too???? However, I do have petropy working on my system now. At least on my system it does take a while to run.

electrofacies.py

-- coding: utf-8 --

""" Electrofacies is a model to calculate numerical facies from log data. It uses sckit-learn for standardization and clustering.

""" import numpy as np import pandas as pd from sklearn.preprocessing import StandardScaler from sklearn.decomposition import PCA from sklearn.cluster import MiniBatchKMeans

def electrofacies(logs, formations, curves, n_clusters, log_scale = [], n_components = 0.85, curve_name = 'FACIES'): """ Electrofacies function to group intervals by rock type. Also referred to as heterogenous rock analysis.

Parameters
----------
logs : list of :class:`ptr.Log` objects
    List of Log objects
formations: list of formation names
    List of strings containg formation names which should be
    previously loaded into Log objects
curves : list of curve names
    List of strings containing curve names as inputs in the
    electrofacies calculations
n_clusters : int
    Number of clusters to group intervals. Number of electrofacies.
log_scale : list of curve names
    List of string containing curve names which are preprocessed
    on a log scale. For example, deep resistivity separates better
    on a log scale, and is graph logarithmically when viewing data
    in a log viewer.
n_components : int, float, None or string (default 0.85)
    Number of principal components to keep. If value is less than
    one, the number of principal components be the number required
    to exceed the explained variance.
curve_name : str (default 'FACIES')
    Name of the new electrofacies curve.

Returns
-------
list
    list of :class:`petropy.Log` objects

Examples
--------
>>> # loads sample Wolfcamp calculates electrofacies for that well
>>> import petropy as ptr
# reads sample Wolfcamp Log from las file
>>> log = ptr.log_data('WFMP')
>>> logs = [log]
>>> f = ['WFMPA', 'WFMPB', 'WFMPC']
>>> c = ['GR_N', 'RESDEEP_N', 'RHOB_N', 'NPHI_N', 'PE_N']
>>> scale = ['RESDEEP_N']
>>> logs = ptr.electrofacies(logs, f, c, 8, log_scale = scale)

>>> import petropy as ptr
# loads logs from a list of paths and
# calculates electrofacies across the wells
#
# defin file_paths for las files to analyze
>>> file_paths = ['path/to/log1.las', 'path/to/log2.las',
... 'path/to/log3.las', 'path/to/log4.las']
# create list of Log objects
>>> logs = [ptr.Log(x) for x in file_paths]
# define csv with tops for all wells
>>> tops_csv = 'path/to/tops.csv'
# add formation tops to wells
>>> for log in logs:
...     log.tops_from_csv(tops_csv)
# define list of formation tops. If single formation, f = ['FORM']
>>> f = ['FORM1', 'FORM2']
# list of curves to use for classification
>>> c = ['GR_N', 'RESDEEP_N', 'RHOB_N', 'NPHI_N', 'PE_N']
>>> scale = ['RESDEEP_N']
# run electrofacies across logs in list
>>> logs = electrofacies(logs, f, c, 8, log_scale = scale)
# save las in renamed file
>>> for i, log in enumerate(logs):
...     new_file_name = file_paths[i].split('.')[0]+'_with_HRA.las'
...     log.write(new_file_name)

"""

df = pd.DataFrame()

for log in logs:

    if log.well['UWI'] is None:
        raise ValueError('UWI required for log identification.')

    log_df = log.df()
    log_df['UWI'] = log.well['UWI'].value
    log_df['DEPTH_INDEX'] = np.arange(0, len(log[0]))

    for formation in formations:
        top = log.tops[formation]
        bottom = log.next_formation_depth(formation)
        depth_index = np.intersect1d(np.where(log[0] >= top)[0],
                                     np.where(log[0] < bottom)[0])
        df = df.append(log_df.iloc[depth_index])

for s in log_scale:
    df[s] = np.log(df[s])

not_null_rows = pd.notnull(df[curves]).any(axis = 1)

X = StandardScaler().fit_transform(df.loc[not_null_rows, curves])

pc = PCA(n_components = n_components).fit(X)

components = pd.DataFrame(data = pc.transform(X),
                          index = df[not_null_rows].index)

minibatch_input = components.as_matrix()

components.columns = \
               ['PC%i' % x for x in range(1, pc.n_components_ + 1)]

components['UWI'] = df.loc[not_null_rows, 'UWI']
components['DEPTH_INDEX'] = df.loc[not_null_rows, 'DEPTH_INDEX']

size = len(components) // 20
if size > 10000:
    size = 10000
elif size < 100:
    size = 100

df.loc[not_null_rows, curve_name] = \
                    MiniBatchKMeans(n_clusters = n_clusters,
                    batch_size = size).fit_predict(minibatch_input)
df.loc[not_null_rows, curve_name] += 1

for log in logs:

    uwi = log.well['UWI'].value

    for v, vector in enumerate(pc.components_):
        v += 1
        pc_curve = 'PC%i' % v

        ### add eigenvector data to header ###

        if pc_curve in log.keys():
            data = log[pc_curve]
            depth_index = components.loc[components.UWI == uwi,
                                         'DEPTH_INDEX']
            data[depth_index] = \
                      np.copy(components.loc[components.UWI == uwi,
                                            pc_curve])
        else:
            data = np.empty(len(log[0]))
            data[:] = np.nan
            depth_index = components.loc[components.UWI == uwi,
                                         'DEPTH_INDEX']
            data[depth_index] = \
                      np.copy(components.loc[components.UWI == uwi,
                      pc_curve])

            log.add_curve(pc_curve, np.copy(data),
            descr = 'Pricipal Component %i from electrofacies' % v)

    if curve_name in log.keys():
        data = log[curve_name]
        depth_index = df.loc[df.UWI == uwi, 'DEPTH_INDEX']
        data[depth_index] = df.loc[df.UWI == uwi, curve_name]
    else:
        data = np.empty(len(log[0]))
        data[:] = np.nan
        depth_index = df.loc[df.UWI == uwi, 'DEPTH_INDEX']

        data[depth_index] = \
                         np.copy(df.loc[df.UWI == uwi, curve_name])

        log.add_curve(curve_name, np.copy(data),
                      descr = 'Electrofacies')

return logs

wolfcamp_single.py

"""

Wolfcamp Example - Single las file

This example shows the full petrophysical workflow avaiable in PetroPy for a single wolfcamp las file courtesy of University Lands Texas.

The workflow progresses in these 11 steps

1. Read las file and create a :class:petropy.Log object
2. Load tops from a csv file using :meth:petropy.Log.tops_from_csv
3. Create a :class:petropy.LogViewer show in edit_mode to fix data
4. Define formations for calculations.
5. Calculate fluid properties by
   1. Loading parameters via :meth:petropy.Log.fluid_properties_parameters_from_csv
   2. Calculating over formations via :meth:petropy.Log.formation_fluid_properties
6. Calculate mulitmineral properties by
   1. Loading parameters via :meth:petropy.Log.multimineral_parameters_from_csv
   2. Calculating over formations via :meth:petropy.Log.formation_multimineral_model
7. Curve summations via :meth:petropy.Log.summations
8. Adding pay flags via :meth:petropy.Log.add_pay_flag
9. Clustering intervals into Electrofacies via :meth:petropy.electrofacies
10. Exporting log statistics via :meth:petropy.Log.statistics
11. Saving LogViewer to png and Log to las

To bulk process a folder of las files at once, use the bulk example_ .

.. _bulk example: wolfcamp_bulk.html """

import petropy as ptr

import pyplot to add logo to figure

import matplotlib.pyplot as plt

1. Read las file

create a Log object by reading a file path

las_file_path = '42303347740000.las' log = ptr.Log(las_file_path)

2. load tops

tops_file_path = 'tops.csv' log.tops_from_csv(tops_file_path)

3. graphically edit log

use manual mode for fixing borehole washout

and other changes requiring redrawing data

use bulk shift mode to linearly adjust all

curve data

close both windows to continue program

viewer = ptr.LogViewer(log, top = 6950, height = 100) viewer.show(edit_mode = True)

overwrite log variable with updated log

from LogViewer edits

log = viewer.log

4. define formations

f = ['WFMPA', 'WFMPB', 'WFMPC']

5. fluid properties

load fluid properties from a csv file

since path is not specified, load default

csv file included with petropy

log.fluid_properties_parameters_from_csv()

calculate fluid properties over defined

formations with parameter WFMP from

previously loaded csv

log.formation_fluid_properties(f, parameter = 'WFMP')

6. multimineral model

load multimineral parameters from csv file

since path is not specified, load default

csv file included with petropy

log.multimineral_parameters_from_csv()

calculate multiminearl model over defined

formations with parameter WFMP from

previously loaded csv

log.formation_multimineral_model(f, parameter = 'WFMP')

7. summations

define curves to calculate cumulative values

c = ['OIP', 'BVH', 'PHIE']

calculate cumulative values over formations

log.summations(f, curves = c)

8. pay flags

define pay flogs as list of tuples for

(curve, value)

flag_1_gtoe = [('PHIE', 0.03)] flag_2_gtoe = [('PAY_FLAG_1', 1), ('BVH', 0.02)] flag_3_gtoe = [('PAY_FLAG_2', 1)] flag_3_ltoe = [('SW', 0.2)]

add pay flags over defined formations

log.add_pay_flag(f, greater_than_or_equal = flag_1_gtoe)

log.add_pay_flag(f, greater_than_or_equal = flag_2_gtoe)

log.add_pay_flag(f, greater_than_or_equal = flag_3_gtoe, less_than_or_equal = flag_3_ltoe)

9. electrofacies

define curves to use in electofaceis module

electro_logs = ['GR_N', 'RESDEEP_N', 'NPHI_N', 'RHOB_N', 'PE_N']

make a list of Log objects as input

logs = [log]

calculate electrofacies for the defined logs

over the specified formations

finding 6 clusters of electrofacies

with RESDEEP_N logarithmically scaled

logs = ptr.electrofacies(logs, f, electro_logs, 6, log_scale = ['RESDEEP_N'])

unpack log object from returned list

log = logs[0]

10. statistics

define list of curves to find statistics

stats_curves = ['OIP', 'BVH', 'PHIE', 'SW', 'VCLAY', 'TOC']

calculate stats over specified formation and

save to csv file wfmp_statistics.csv

update the line if the well, formation is

already included in the csv file

log.statistics_to_csv('wfmp_statistics.csv', replace = True, formations = f, curves = stats_curves)

11. export data

find way to name well, looking for well name

or UWI or API

if len(log.well['WELL'].value) > 0: well_name = log.well['WELL'].value elif len(str(log.well['UWI'].value)) > 0: well_name = str(log.well['UWI'].value) elif len(log.well['API'].value) > 0: well_name = str(log.well['API'].value) else: well_name = 'UNKNOWN' well_name = well_name.replace('.', '')

scale height of viewer to top and bottom

of calculated values

wfmpa_top = log.tops['WFMPA'] wfmpc_base = log.next_formation_depth('WFMPC')

top = wfmpa_top height = wfmpc_base - wfmpa_top

create LogViewer with the default full_oil

template included in petropy

viewer = ptr.LogViewer(log, top = top, height = height, template_defaults = 'full_oil')

set viewer to 17x11 inches size for use in

PowerPoint or printing to larger paper

viewer.fig.set_size_inches(17, 11)

add well_name to title of LogViewer

viewer.fig.suptitle(well_name, fontweight = 'bold', fontsize = 30)

add logo to top left corner

logo_im = plt.imread('cbpetro_logo.png') logo_ax = viewer.fig.add_axes([0, 0.85, 0.2, 0.2]) logo_ax.imshow(logo_im) logo_ax.axis('off')

add text to top right corner

if len(str(log.well['UWI'].value)) > 0: label = 'UWI: ' + str(log.well['UWI'].value) + '\n' elif len(log.well['API'].value) > 0: label = 'API: ' + str(log.well['API'].value) + '\n' else: label = ''

label += 'County: Reagan\nCreated By: Todd Heitmann\n' label += 'Creation Date: October 23, 2017' viewer.axes[0].annotate(label, xy = (0.99,0.99), xycoords = 'figure fraction', horizontalalignment = 'right', verticalalignment = 'top', fontsize = 14)

save figure and log

viewer_file_name=r'%s_processed.png' % well_name las_file_name = r'%s_processed.las' % well_name

viewer.fig.savefig(viewer_file_name) viewer.log.write(las_file_name)

Best Regards,

---

E. Craig Phillips CEO and Chief Petrophysicist Crested Butte Petrophysical Consultants

459 Cisneros Lane Crested Butte, CO 81224 USA Office: +1 970-343-0730 Mobile: +1 970-343-0730 email: craig@cbpetro.com Website: www.cbpetro.com GitHub: https://github.com/Philliec459

The information contained in this electronic message is confidential, it constitutes a profesional and/or industrial secret in terms of the current legislation, and is intended for its recipient only. If you receive this message by mistake or if you are not the recipient thereof, please notify the sender and destroy it.

La informacion contenida en este mensaje de datos es confidencial, constituye un secreto industrial y/o profesional en terminos de la legislacion vigente y se encuentra dirigida exclusivamente al destinatario indicado en dicho mensaje. Si usted recibe esta informacion por error o si usted no es el destinatario del mensaje, favor de notificar al emisor, y destruyalo.

On Jul 9, 2020, at 1:32 PM, cdmeyer23 notifications@github.com wrote:

@Philliec459 https://github.com/Philliec459 Yes, I am working with the wolfcamp_single.py

If you look at the electrofacies source code here: https://github.com/toddheitmann/PetroPy/blob/master/petropy/electrofacies.py https://github.com/toddheitmann/PetroPy/blob/master/petropy/electrofacies.py it requires the import of numpy and pandas.

The example wolfcamp_single.py calls ptr.electrofacies on line 147:

logs = ptr.electrofacies(logs, f, electro_logs, 6, log_scale = ['RESDEEP_N'])

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/toddheitmann/PetroPy/issues/7#issuecomment-656310961, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANSKYPDYVBR4EJFOQU4BXITR2YLMNANCNFSM4OVZ3LDA.

[cdmeyer23]

@Philliec459 Thank you so much helping me out, I think the only way to get around it for now is to use the setup that is provided. I didn't see that before. While it is frusterating to have to run it with older versions of numpy and pandas, which makes other code that is run with it not function properly, at least it works that way.

Thanks again!

[Philliec459]

I think that it is easiest to load petropy to work from it.

As a Petrophysicist, this is probably the first python project that I worked on when I was getting started with Python. Now that I look back there are a lot of good things here that I want to pursue further.

I have my own GitHub with some petrophysical repositories that might help you too, but lately I am into Object Detection and so my repositories are deviating from petrophysics.

https://github.com/Philliec459?tab=repositories https://github.com/Philliec459?tab=repositories

My next goal will be to scan a log and do almost object detection picking out invervals, electrofacies, pay just from the log signatures. I will be using the Equinor Volve dataset for this.

Best Regards,

---

E. Craig Phillips CEO and Chief Petrophysicist Crested Butte Petrophysical Consultants

459 Cisneros Lane Crested Butte, CO 81224 USA Office: +1 970-343-0730 Mobile: +1 970-343-0730 email: craig@cbpetro.com Website: www.cbpetro.com GitHub: https://github.com/Philliec459

The information contained in this electronic message is confidential, it constitutes a profesional and/or industrial secret in terms of the current legislation, and is intended for its recipient only. If you receive this message by mistake or if you are not the recipient thereof, please notify the sender and destroy it.

La informacion contenida en este mensaje de datos es confidencial, constituye un secreto industrial y/o profesional en terminos de la legislacion vigente y se encuentra dirigida exclusivamente al destinatario indicado en dicho mensaje. Si usted recibe esta informacion por error o si usted no es el destinatario del mensaje, favor de notificar al emisor, y destruyalo.

On Jul 10, 2020, at 10:43 AM, cdmeyer23 notifications@github.com wrote:

@Philliec459 https://github.com/Philliec459 Thank you so much helping me out, I think the only way to get around it for now is to use the setup that is provided. I didn't see that before. While it is frusterating to have to run it with older versions of numpy and pandas, which makes other code that is run with it not function properly, at least it works that way.

Thanks again!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/toddheitmann/PetroPy/issues/7#issuecomment-656772798, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANSKYPDBSQ6JF2OFGLIA2HDR25AJ7ANCNFSM4OVZ3LDA.

[dianaceroallard]

@cdmeyer23 thank you for posting, I was stuck too in that electrofacies function, I am using Jupyter Notebook. Thanks to my friend Julio (st4tic0) who advised me to change "minibatch_input = components.as_matrix()" to "minibatch_input = components.to_numpy()" , it worked... at least the function electrofacies.py