search

uqfoundation / dill

serialize all of Python
http://dill.rtfd.io
Other
2.27k stars 181 forks source link

module has no attribute '__name__' error happening in a 'random' manner #204

Closed MarcoMagl closed 7 years ago

MarcoMagl commented 7 years ago

Sometimes, I get this error: ` AttributeError: module has no attribute 'name'

/Users/marco.magliulo/miniconda2/envs/EnvAvecPython3/lib/python3.5/site-packages/dill/dill.py(251)load() 249 pik._main = _main_module 250 obj = pik.load() --> 251 if type(obj).module == _main_module.name: # point obj class to main 252 try: obj.class == getattr(pik._main, type(obj).name) 253 except AttributeError: pass # defined in a file /Users/marco.magliulo/miniconda2/envs/EnvAvecPython3/lib/python3.5/site-packages/dill/dill.py(251) `

This error happens in the following part of the code: `
if TypeRigidBody == 'RigidSeg':

generate sympy expression that are dumped in the current directory

        express_ctct_line.main()
        with open('normal_ctc_penalty.dill', 'rb') as f:  
            (dWN, DdWN) = dill.load(f)

`

It happens when I launch my script inside ipython. So, when I exit() ipython and launch a new session, this error does not occur anymore. I am using IPython 5.1.0 and Python 3.5.2

mmckerns commented 7 years ago

@MarcoMagl: Can you post some minimalistic code that reproduces this issue? It will be hard to diagnose in the abstract. AttributeError: module has no attribute '__name__' and the traceback you've posted is a good start, but having code that can be tested against will make it much easier.

mmckerns commented 7 years ago

it would seem that the error you're reporting is that __main__ does not have an attribute __name__… which I find really odd, and is probably a bug in whatever module it happens in, if that's indeed true. Please support this with some minimal code that demonstrates the issue.

MarcoMagl commented 7 years ago

Here a simplified version of my code, but I can't reproduce the bug that happens in my main program:

file 1

import dill
import sympy as sp
import numpy as np

def run_symbo_comp():

    sp.var("Nx,Ny,Xix,Xiy,kPenN,uix,uiy,x1x,x1y,x2x,x2y" )
    sp.var("Ksi0, kPenT, Mu, sigmaT_tr ") 
    Xi          = sp.Matrix([Xix, Xiy])
    ui          = sp.Matrix([uix, uiy])
    x1          = sp.Matrix([x1x, x1y])
    x2          = sp.Matrix([x2x, x2y])
    xix         = Xix + uix
    xiy         = Xiy + uiy   
    xi          = sp.Matrix([xix, xiy])
    a1_bar      = x2 - x1
    l           = ((x2 - x1).dot( (x2 - x1).T))**(0.5)
    a1          = (x2 - x1)/l
    N           = sp.Matrix([Nx, Ny])
    gN = (xi - x1).dot(N)
    W_N = 0.5 * kPenN * gN**2
    dW_N = sp.Matrix([W_N.diff(uix), W_N.diff(uiy)])
    DdW_N = sp.Matrix([dW_N.jacobian(ui)]) 
    dW_N = sp.Matrix(2,1, sp.flatten(dW_N[0:2]))
    DdW_N = sp.Matrix(2,2, sp.flatten( DdW_N[0:2, 0:2]))

    with open('normal_ctc_penalty.dill', 'wb') as f:  # Python 3: open(..., 'wb')
        dill.dump((dW_N, DdW_N), f)
    Ksi = (xi - x1).dot(a1) * (1/l)
    gT = (Ksi - Ksi0)*l
    dgT = sp.Matrix([gT.diff(uix), gT.diff(uiy)])
    dW_stick = kPenT * gT * dgT
    DdW_stick = dW_stick.jacobian(ui)
    FN = ((kPenN *gN)**2)**0.5
    dW_slip = Mu * FN * sp.sign(sigmaT_tr) * dgT
    DdW_slip = Mu * FN * sp.sign(sigmaT_tr) * dgT.jacobian(ui)

    dW_stick = sp.Matrix(2,1, sp.flatten(dW_stick[0:2]))
    DdW_stick = sp.Matrix(2,2, sp.flatten( DdW_stick[0:2, 0:2]))
    dW_slip = sp.Matrix(2,1, sp.flatten(dW_slip[0:2]))
    DdW_slip = sp.Matrix(2,2, sp.flatten( DdW_slip[0:2, 0:2]))

    with open('tgt_ctct_symbo.dill', 'wb') as f:  # Python 3: open(..., 'wb')
        dill.dump((dW_stick ,  dW_slip  ,  DdW_stick ,  DdW_slip), f)

file 2

import dill
import sympy as sp
import numpy as np
import FooSymbo

class FooClass():
    def __init__(self):
        # call the subroutine that generate sympy expressions and dump them
        self.set_lambdas()

    def set_lambdas(self): 
        """
        generate lambda functions and store them as attribute
        """
        FooSymbo.run_symbo_comp()

        with open('normal_ctc_penalty.dill', 'rb') as f:  
            (dWN, DdWN) = dill.load(f)

        Nx,Ny,Xix,Xiy,kPenN,uix,uiy,x1x,x1y,x2x,x2y = sp.var('Nx,Ny,Xix,Xiy,kPenN,uix,uiy,x1x,x1y,x2x,x2y' )
        #lambdification of the sympy expressions
        # sympyfy the expression that I imported as .txt
        args = (Nx,Ny,Xix,Xiy,kPenN,uix,uiy,x1x,x1y,x2x,x2y)
        self.dWN = sp.lambdify(sp.flatten(args), dWN, 'numpy') 
        self.DdWN = sp.lambdify(sp.flatten(args), DdWN, 'numpy') 

if __name__ == '__main__':
    Foo = FooClass()
    # raise Error on purpose
    raise ValueError('')

I will try again to trigger the error tomorrow In my code I dump and load using an absolute path. Can it be related ?

Anyway, this is a more complete description of the error that happened just now, in the case it helps:

`

AttributeError Traceback (most recent call last) /Users/marco.magliulo/repos/COMMON_ROOT_MYPROG/Python/OOP_2DLattice/Scripts/TESTS/Test1_unconstrained.py in () 136 137 if name == 'main': --> 138 result = main() 139 if result == 1: 140 print('Test1 : success \n')

/Users/marco.magliulo/repos/COMMON_ROOT_MYPROG/Python/OOP_2DLattice/Scripts/TESTS/Test1_unconstrained.py in main() 57 58 Latt = ElastoPlast2DLatt(Xmin, Ymin, nbrX, nbrY, dX, dY, E_arr, A_arr,\ ---> 59 diagTrusses,ElastoPlast, sigma_y0_arr, H_arr, Contact) 60 61 ################################################################################

/Users/marco.magliulo/repos/COMMON_ROOT_MYPROG/Python/OOP_2DLattice/Classes/ElastoPlast2D.py in init(self, Xmin, Ymin, nbrX, nbrY, dX, dY, E_arr, A_arr, diagTrusses, ElastoPlast, sigma_y0_arr, H_arr, Contact, TupleContactVar) 116 self.Friction = False 117 # generation lambda functions for Fint and Kint --> 118 self.unconstrain_lambda() 119 # spot the nodes on the border of the lattice 120 self.onS = np.zeros((self.Nn), dtype=bool)

/Users/marco.magliulo/repos/COMMON_ROOT_MYPROG/Python/OOP_2DLattice/Classes/ElastoPlast2D.py in unconstrain_lambda(self) 400 401 with open('PDer_Energy_Unconstrained.pickle', 'rb') as f: # Python 3: open(..., 'rb') --> 402 (F, K) = dill.load(f) 403 404 uix, uiy, Xix, Xiy, ujx, ujy, Xjx, Xjy, E, A, eps_p = sp.symbols('uix, uiy, Xix, Xiy, ujx, ujy, Xjx, Xjy, E, A, eps_p')

/Users/marco.magliulo/miniconda2/envs/EnvAvecPython3/lib/python3.5/site-packages/dill/dill.py in load(file) 249 pik._main = _main_module 250 obj = pik.load() --> 251 if type(obj).module == _main_module.name: # point obj class to main 252 try: obj.class == getattr(pik._main, type(obj).name) 253 except AttributeError: pass # defined in a file

AttributeError: module has no attribute 'name' `

mmckerns commented 7 years ago

If you can't reproduce the error, could you then (1) add dill.detect.trace(True) so there's a trace of what's causing the error, and (2) try to add some print statements to your code that might elucidate what's happening?

When you say "random", what do you mean? Does it happen sometimes and not others? How are the runs different each time?

And, again, it would be the biggest help to have a simple code that demonstrates the error.

MarcoMagl commented 7 years ago

Yes, this is what I meant. My code raises some error, so I enter a pdb session, and at the next run I have someting like: / Users/marco.magliulo/repos/COMMON_ROOT_MYPROG/Python/OOP_2DLattice/Classes/ElastoPlast2D.py in unconstrain_lambda(self) 400 401 with open('PDer_Energy_Unconstrained.pickle', 'rb') as f: # Python 3: open(..., 'rb') --> 402 (F, K) = dill.load(f) 403 404 uix, uiy, Xix, Xiy, ujx, ujy, Xjx, Xjy, E, A, eps_p = sp.symbols('uix, uiy, Xix, Xiy, ujx, ujy, Xjx, Xjy, E, A, eps_p')

/Users/marco.magliulo/miniconda2/envs/EnvAvecPython3/lib/python3.5/site-packages/dill/dill.py in load(file)
    249     pik._main = _main_module
    250     obj = pik.load()
--> 251     if type(obj).__module__ == _main_module.__name__: # point obj class to main
    252         try: obj.__class__ == getattr(pik._main, type(obj).__name__)
    253         except AttributeError: pass # defined in a file

AttributeError: module has no attribute '__name__'
> /Users/marco.magliulo/miniconda2/envs/EnvAvecPython3/lib/python3.5/site-packages/dill/dill.py(251)load()
    249     pik._main = _main_module
    250     obj = pik.load()
--> 251     if type(obj).__module__ == _main_module.__name__: # point obj class to main
    252         try: obj.__class__ == getattr(pik._main, type(obj).__name__)
    253         except AttributeError: pass # defined in a file
> /Users/marco.magliulo/miniconda2/envs/EnvAvecPython3/lib/python3.5/site-packages/dill/dill.py(251)

Regarding dill.detect.trace(True), I get:

T4: <class 'sympy.matrices.dense.MutableDenseMatrix'>
# T4
D2: <dict object at 0x109344548>
T4: <class 'sympy.core.mul.Mul'>
# T4
T4: <class 'sympy.core.numbers.Float'>
# T4
D2: <dict object at 0x11d64c048>
# D2
T4: <class 'sympy.core.symbol.Symbol'>
# T4
D2: <dict object at 0x11d64c648>
T4: <class 'sympy.core.assumptions.StdFactKB'>
# T4
D2: <dict object at 0x1090a04c8>
D2: <dict object at 0x11ab3acc8>
# D2
# D2
# D2
D2: <dict object at 0x11d64ccc8>
D2: <dict object at 0x11ca95588>
D2: <dict object at 0x109066b48>
# D2
# D2
# D2
T4: <class 'sympy.core.power.Pow'>
# T4
T4: <class 'sympy.core.add.Add'>
# T4
D2: <dict object at 0x1092d4e88>
D2: <dict object at 0x112f3da88>
D2: <dict object at 0x11ab28048>
# D2
# D2
# D2
D2: <dict object at 0x11c9aa048>
D2: <dict object at 0x11c3e0b48>
D2: <dict object at 0x11a7f6cc8>
# D2
# D2
# D2
T4: <class 'sympy.core.numbers.NegativeOne'>
# T4
D2: <dict object at 0x10928d1c8>
# D2
D2: <dict object at 0x11c248d88>
D2: <dict object at 0x11c41ba08>
D2: <dict object at 0x11c2692c8>
# D2
# D2
# D2
D2: <dict object at 0x11aad6a08>
# D2
D2: <dict object at 0x11d5c5088>
D2: <dict object at 0x11c41bf48>
D2: <dict object at 0x11ab4f708>
# D2
# D2
# D2
D2: <dict object at 0x11d6633c8>
# D2
D2: <dict object at 0x1090258c8>
# D2
T4: <class 'sympy.core.numbers.Integer'>
# T4
D2: <dict object at 0x11c269888>
# D2
D2: <dict object at 0x11aac3d88>
# D2
D2: <dict object at 0x1090a3e88>
D2: <dict object at 0x11a87ff48>
D2: <dict object at 0x11a8ec0c8>
# D2
# D2
# D2
D2: <dict object at 0x11ab28b08>
D2: <dict object at 0x10935d248>
D2: <dict object at 0x10901ff48>
# D2
# D2
# D2
D2: <dict object at 0x11ca05b88>
D2: <dict object at 0x11ab28588>
D2: <dict object at 0x11d5f2408>
# D2
# D2
# D2
D2: <dict object at 0x11ab4d308>
# D2
D2: <dict object at 0x11c253788>
D2: <dict object at 0x10901f348>
D2: <dict object at 0x11c970388>
# D2
# D2
# D2
D2: <dict object at 0x1091e9e48>
# D2
D2: <dict object at 0x10901c748>
# D2
D2: <dict object at 0x11a7d5708>
# D2
D2: <dict object at 0x11aac1148>
# D2
D2: <dict object at 0x11a90fe88>
# D2
D2: <dict object at 0x11a7d3ec8>
# D2
D2: <dict object at 0x11a7a4a08>
D2: <dict object at 0x11aad0848>
D2: <dict object at 0x11aad0a08>
# D2
# D2
# D2
D2: <dict object at 0x11a839708>
# D2
D2: <dict object at 0x11aad63c8>
# D2
D2: <dict object at 0x11c373908>
# D2
D2: <dict object at 0x11aad60c8>
# D2
D2: <dict object at 0x1093449c8>
# D2
D2: <dict object at 0x11c373c08>
# D2
D2: <dict object at 0x11d688248>
# D2
D2: <dict object at 0x11c373208>
# D2
D2: <dict object at 0x11aad6b88>
# D2
D2: <dict object at 0x11aad64c8>
# D2
D2: <dict object at 0x109015cc8>
# D2
D2: <dict object at 0x11aad65c8>
# D2
D2: <dict object at 0x11aad6cc8>
# D2
D2: <dict object at 0x11ca0a588>
# D2
D2: <dict object at 0x1092da388>
# D2
D2: <dict object at 0x11aab9a88>
# D2
D2: <dict object at 0x11aad6e88>
# D2
D2: <dict object at 0x11aad69c8>
# D2
D2: <dict object at 0x11c983ec8>
# D2
D2: <dict object at 0x11aad6f08>
# D2
D2: <dict object at 0x11cab1048>
# D2
D2: <dict object at 0x11a7a2848>
# D2
D2: <dict object at 0x11cab1b48>
# D2
D2: <dict object at 0x11cab1808>
# D2
D2: <dict object at 0x11ab26808>
# D2
D2: <dict object at 0x11d64c188>
# D2
D2: <dict object at 0x11cab11c8>
# D2
D2: <dict object at 0x11cab1708>
# D2
D2: <dict object at 0x11cab1548>
# D2
D2: <dict object at 0x11cab1448>
# D2
D2: <dict object at 0x11cab16c8>
# D2
D2: <dict object at 0x11cab1f48>
# D2
D2: <dict object at 0x11cab13c8>
# D2
D2: <dict object at 0x11cab19c8>
# D2
D2: <dict object at 0x11cab1fc8>
# D2
D2: <dict object at 0x11cab1e08>
# D2
D2: <dict object at 0x11d64c6c8>
# D2
D2: <dict object at 0x11cab1588>
# D2
D2: <dict object at 0x11cab1a88>
# D2
D2: <dict object at 0x11cab1608>
# D2
D2: <dict object at 0x11cab1208>
# D2
D2: <dict object at 0x11cab14c8>
# D2
D2: <dict object at 0x11cab1688>
# D2
D2: <dict object at 0x11cab1ac8>
# D2
D2: <dict object at 0x11cab1148>
# D2
D2: <dict object at 0x11cab1488>
# D2
D2: <dict object at 0x11cab1948>
# D2
D2: <dict object at 0x11c3733c8>
# D2
D2: <dict object at 0x11cab1888>
# D2
D2: <dict object at 0x11cab1388>
# D2
D2: <dict object at 0x11cab1248>
# D2
D2: <dict object at 0x11cab1d88>
# D2
D2: <dict object at 0x11cab15c8>
# D2
D2: <dict object at 0x11cab1648>
# D2
D2: <dict object at 0x11cab1bc8>
# D2
D2: <dict object at 0x11cab1908>
# D2
D2: <dict object at 0x11cab18c8>
# D2
D2: <dict object at 0x11cab1ec8>
# D2
D2: <dict object at 0x11cab1288>
# D2
# D2
D2: <dict object at 0x10902a388>
D2: <dict object at 0x11cab1dc8>
# D2
D2: <dict object at 0x11cab1f08>
# D2
D2: <dict object at 0x11c298148>
# D2
D2: <dict object at 0x109346e08>
# D2
D2: <dict object at 0x11c298948>
# D2
D2: <dict object at 0x11c298d88>
# D2
D2: <dict object at 0x11cab1b08>
# D2
D2: <dict object at 0x11a88d308>
# D2
D2: <dict object at 0x11a88dd48>
# D2
D2: <dict object at 0x11a88d848>
# D2
D2: <dict object at 0x11cab1a08>
# D2
D2: <dict object at 0x11cab1348>
# D2
D2: <dict object at 0x11a88d548>
# D2
D2: <dict object at 0x11a88df08>
# D2
D2: <dict object at 0x11a88d208>
# D2
D2: <dict object at 0x11a88d9c8>
# D2
D2: <dict object at 0x11cab1788>
# D2
D2: <dict object at 0x11a88d688>
# D2
D2: <dict object at 0x11a88df88>
# D2
D2: <dict object at 0x11a88d048>
# D2
D2: <dict object at 0x11a88d8c8>
# D2
D2: <dict object at 0x11a88dbc8>
# D2
D2: <dict object at 0x11a88da08>
# D2
D2: <dict object at 0x11a88db48>
# D2
D2: <dict object at 0x11a88da48>
# D2
D2: <dict object at 0x11a88d808>
# D2
D2: <dict object at 0x11d61e188>
# D2
D2: <dict object at 0x11a88d388>
# D2
D2: <dict object at 0x11a88d7c8>
# D2
D2: <dict object at 0x11d61ec48>
# D2
D2: <dict object at 0x11d61e408>
# D2
D2: <dict object at 0x11d61ebc8>
# D2
D2: <dict object at 0x11d61e3c8>
# D2
D2: <dict object at 0x11a88d448>
# D2
D2: <dict object at 0x11d61e448>
# D2
D2: <dict object at 0x11c988a48>
# D2
D2: <dict object at 0x11c988888>
# D2
D2: <dict object at 0x11c9881c8>
# D2
D2: <dict object at 0x11c988bc8>
# D2
D2: <dict object at 0x11c988f88>
# D2
D2: <dict object at 0x11d6ddb08>
# D2
D2: <dict object at 0x11c988488>
# D2
D2: <dict object at 0x11c988848>
# D2
D2: <dict object at 0x11c988208>
# D2
D2: <dict object at 0x11c988508>
# D2
D2: <dict object at 0x11d61e7c8>
# D2
D2: <dict object at 0x11c9883c8>
# D2
D2: <dict object at 0x11c988b88>
# D2
D2: <dict object at 0x11c988448>
# D2
D2: <dict object at 0x11c988708>
# D2
D2: <dict object at 0x11c988548>
# D2
D2: <dict object at 0x11c988608>
# D2
D2: <dict object at 0x11c988ec8>
# D2
D2: <dict object at 0x11c9884c8>
# D2
D2: <dict object at 0x11c988588>
# D2
D2: <dict object at 0x11c988dc8>
# D2
D2: <dict object at 0x11c988308>
# D2
D2: <dict object at 0x11c988288>
# D2
D2: <dict object at 0x11c9882c8>
# D2
D2: <dict object at 0x11c988c08>
# D2
D2: <dict object at 0x11c988388>
# D2
D2: <dict object at 0x11c988e08>
# D2
D2: <dict object at 0x11c988d48>
# D2
D2: <dict object at 0x11c988cc8>
# D2
D2: <dict object at 0x11c988a08>
# D2
D2: <dict object at 0x11c9888c8>
# D2
D2: <dict object at 0x11c988e48>
# D2
D2: <dict object at 0x11c9880c8>
# D2
D2: <dict object at 0x11c988d88>
# D2
D2: <dict object at 0x11c988188>
# D2
D2: <dict object at 0x11c9885c8>
# D2
D2: <dict object at 0x11c988648>
# D2
D2: <dict object at 0x11c988048>
# D2
D2: <dict object at 0x11c988ac8>
# D2
D2: <dict object at 0x11c988748>
# D2
D2: <dict object at 0x11c9889c8>
# D2
D2: <dict object at 0x11c988408>
# D2
D2: <dict object at 0x11c988088>
# D2
D2: <dict object at 0x11c988c48>
# D2
D2: <dict object at 0x11c988788>
# D2
D2: <dict object at 0x11c9887c8>
# D2
D2: <dict object at 0x11c988a88>
# D2
D2: <dict object at 0x11c379c88>
# D2
D2: <dict object at 0x11c9886c8>
# D2
D2: <dict object at 0x11c379fc8>
# D2
D2: <dict object at 0x11c988e88>
# D2
D2: <dict object at 0x11c988688>
# D2
D2: <dict object at 0x11c379b88>
# D2
D2: <dict object at 0x11c379408>
# D2
D2: <dict object at 0x11c379688>
# D2
D2: <dict object at 0x11c379708>
# D2
D2: <dict object at 0x11c988d08>
# D2
D2: <dict object at 0x11c379348>
# D2
D2: <dict object at 0x11a9112c8>
# D2
D2: <dict object at 0x11c379088>
# D2
D2: <dict object at 0x11a9110c8>
# D2
D2: <dict object at 0x11c379148>
# D2
D2: <dict object at 0x11c3795c8>
# D2
# D2

Is this helpful or you need something else ? What could I print that would help you more ?

As I told on the comment above, I tried to trigger the error in a similar simpler code but haven't succeeded yet

Hope this helps a bit more

EDIT

The bug appears even if no exception was raised on a previous bug. For example:

In [38]: run Test24_simplest_latt_frictionless.py sucess

So for this run, dill was working properly. No exception was raised. But for the next run, I have the dill exception that is raised. I tried a %reset within ipython but it does not change anything, the exception of dill is still raised.

I looked at _main__module in:

246 def load(file): 247 """unpickle an object from a file""" 248 pik = Unpickler(file) 249 pik._main = _main_module 250 obj = pik.load() 251 -> if type(obj).module == _main_module.name: # point obj class to main 252 try: obj.class == getattr(pik._main, type(obj).name) 253 except AttributeError: pass # defined in a file 254 #_main_module.dict.update(obj.dict) #XXX: should update globals ? 255 return obj

which gives me: <module '?' from '/Users/marco.magliulo/repos/COMMON_ROOT_MYPROG/Python/OOP_2DLattice/Scripts/TESTS/Test24_simplest_latt_frictionless.py'>

but I do not understand why name is causing trouble because in the file Test24_simplest_latt_frictionless.py I have a if __name__ == '__main__' to launch the script which does not cause any trouble

mmckerns commented 7 years ago

What you have posted of your code is pretty hard to follow... i.e. it's a bit dense and not really a simple example. However, I do have something interesting. Look what happens when you delete the __name__ method from a module... (note that I have created an empty file with the name as below):

>>> import Test24_simplest_latt_frictionless as t
>>> t 
<module 'Test24_simplest_latt_frictionless' from 'Test24_simplest_latt_frictionless.py'>
>>> del t.__name__
>>> t
<module '?' from 'Test24_simplest_latt_frictionless.py'>
>>>

As a matter of fact, if I make a simple module like this...

# File: Test24_simplest_latt_frictionless.py
del __name__

if __name__ == '__main__':
  print("hi")

Then the result is:

>>> import Test24_simplest_latt_frictionless as t
>>> t
<module '?' from 'Test24_simplest_latt_frictionless.py'>
>>> import dill
>>> dill.detect.errors(t)
AttributeError("'module' object has no attribute '__name__'",)

and running the same file from the shell window gives:

$ python Test24_simplest_latt_frictionless.py 
$

which means the __main__ block is ignored.

So this is interesting... I didn't realize a module could be missing the __name__ attribute. I'd expect that it's a really rare corner case, but obviously one dill can fail on. (@matsjoyce: weird, right?)

So are you deleting or modifying __name__?

With the above information, can you produce a simple example that reproduces the error you are experiencing?

mmckerns commented 7 years ago

I believe I can reproduce the error you get with this:

>>> import __main__ as _main_module
>>> _main_module.t = lambda x:x
>>> del _main_module.__name__
>>> import dill
>>> x = dill.dumps(_main_module.t)
>>> dill.loads(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mmckerns/lib/python2.7/site-packages/dill-0.2.6.dev0-py2.7.egg/dill/dill.py", line 276, in loads
    return load(file)
  File "/Users/mmckerns/lib/python2.7/site-packages/dill-0.2.6.dev0-py2.7.egg/dill/dill.py", line 267, in load
    if type(obj).__module__ == _main_module.__name__: # point obj class to main
AttributeError: 'module' object has no attribute '__name__'
>>>

I have no idea why that would be happening in your code (as it's hard to follow your example).

matsjoyce commented 7 years ago

Well, since python is so dynamic, almost anything could be missing, but I can't think of any good reason that a module should be missing __name__... For this specific case, maybe use getattr(_main_module, "__name__", "__main__") as a backup?

If you want to extend support to all modules without __name__ the problem is where to get the name from for re-import it. module.__spec__.name might work, but it is python 3 only. Deriving from __file__ could also work, but it makes a lot of assumptions and won't work on builtins. Or just make a note that it is unsupported.

mmckerns commented 7 years ago

If I make an innocuous change in dill.py...

if type(obj).__module__ == getattr(_main_module, '__name__', None):

then I can retrieve objects from a module with no name:

>>> import __main__ as _main_module
>>> _main_module.t = lambda x:x
>>> del _main_module.__name__
>>> import dill
>>> x = dill.dumps(_main_module.t)
>>> t = dill.loads(x)
>>> t
<function <lambda> at 0x10b3d7578>
>>>

I'm going to commit this change. Tell me if it fixes it for you

mmckerns commented 7 years ago

@matsjoyce: I was thinking of the same change... and I also think this has got to be a really really rare case that probably should not be dealt with... however, the change I'm suggesting (same one as you), is not going to hurt anything (I think), so I'm going to put it in.

mmckerns commented 7 years ago

Wait... what's the reason you choose __main__ as the default if __name___ is missing? If you use None as the default, I think it corresponds better (... and this also works):

>>> with open('xxx.py', 'wb') as f:
...   f.write('')
... 
>>> import xxx
>>> del xxx.__name__
>>> 
>>> xxx.t = lambda x:x
>>> 
>>> import dill
>>> x = dill.dumps(xxx.t)
>>> t = dill.loads(x)
>>> t
<function <lambda> at 0x107798578>
mmckerns commented 7 years ago

committed in: c998b3999623e7337c79117585da0b05f2127369. @MarcoMagl: let me know if this fixes the issue for you, and this can be closed.

matsjoyce commented 7 years ago

Because now this happens:

>>> import dill
>>> class A: pass
... 
>>> del __name__
>>> a=dill.copy(A())
<__main__.A object at 0x7f8fd2f37240>
>>> type(a)
<class '__main__.A'>
>>> type(a) is A
False

BTW, there is an error in load:

def load(file):
    """unpickle an object from a file"""
    pik = Unpickler(file)
    pik._main = _main_module
    obj = pik.load()
    if type(obj).__module__ == getattr(_main_module, '__name__', None):
        # point obj class to main
        try: obj.__class__ == getattr(pik._main, type(obj).__name__)
        except AttributeError: pass # defined in a file
   #_main_module.__dict__.update(obj.__dict__) #XXX: should update globals ?
    return obj

The try: obj.__class__ == getattr(pik._main, type(obj).__name__) should be obj.__class__ = ...? If you change load to (fix above and use "main" as default):

def load(file):
    """unpickle an object from a file"""
    pik = Unpickler(file)
    pik._main = _main_module
    obj = pik.load()
    if type(obj).__module__ == getattr(_main_module, '__name__', "__main__"):
        # point obj class to main
        try: obj.__class__ = getattr(pik._main, type(obj).__name__)
        except AttributeError: pass # defined in a file
   #_main_module.__dict__.update(obj.__dict__) #XXX: should update globals ?
    return obj

You get:

>>> import dill
>>> class A:pass
... 
>>> a=A()
>>> a2=dill.copy(a)
>>> type(a) is type(a2)
True
mmckerns commented 7 years ago

... @matsjoyce: you are right. I've committed what you have above, with one small modification to correct the code for 3.5+ in e93c3d4e927a7f42927313b63a8c18a902b8bd5d

mmckerns commented 7 years ago

... since there's no comment, I assume it's working for you. Reopen if it's not working for you.