search

51Degrees / Device-Detection

THE Fastest and most Accurate device detection for C / PHP / Perl / Python and Node.js - professionally maintained device data
https://51degrees.com/device-detection
Other
112 stars 46 forks source link

Python quit unexpectedly (Mac, fiftyone_degrees_mobile_detector_v3_wrapper) #39

Closed PavMel closed 4 years ago

PavMel commented 5 years ago

Hi, there. Thank you for your very nice job. This is report for you: Python libs has been Installed from source from master branch. But version from pypi affected too.

Code:

def populate_db_init():
    import os
    from settings import ABSPATH
    from collections import OrderedDict
    from FiftyOneDegrees import fiftyone_degrees_mobile_detector_v3_wrapper
    from fiftyone_degrees.mobile_detector.conf import settings
    dataFile = os.path.join(ABSPATH, 'data', '51degrees', '51Degrees-PremiumV3_2.dat')
    properties = settings.PROPERTIES
    cacheSize = settings.CACHE_SIZE
    poolSize = settings.POOL_SIZE
    provider = fiftyone_degrees_mobile_detector_v3_wrapper.Provider(
        dataFile,
        properties,
        cacheSize,
        poolSize
    )
    profiles = provider.findProfiles("IsMobile", "True")
    print("There are %d mobile profiles in the %s data file." % (profiles.getCount(), provider.getDataSetName()))
    for profile_index in reversed(range(0, profiles.getCount())):
        DeviceId = profiles.getProfileId(profile_index)
        device_object = provider.getMatchForDeviceId("%s" % DeviceId)
        item = OrderedDict([
            ('HardwareVendor', device_object.getValue('HardwareVendor')),
            ('HardwareModel', device_object.getValue('HardwareModel')),
            ('HardwareName', device_object.getValue('HardwareName') or None),
            ('HardwareFamily', device_object.getValue('HardwareFamily') or None),
            ('HardwareProfileSource', device_object.getValue('HardwareProfileSource') or None),
        ])
        print(dict(item), profile_index)
    return ### <<< HAPPENED HERE

if __name__ == '__main__':
    populate_db_init()

Log:

Process:               Python [12397]
Path:                  /Library/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.7.4 (3.7.4)
Code Type:             X86-64 (Native)
Parent Process:        pycharm [75569]
Responsible:           Python [12397]
User ID:               502

Date/Time:             2019-10-16 01:11:09.148 +0300
OS Version:            Mac OS X 10.14.6 (18G103)
Report Version:        12

Time Awake Since Boot: 770000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       EXC_I386_GPFLT
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [12397]

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_pthread.dylib         0x00007fff7c8f0d1b pthread_mutex_destroy + 27
1   _fiftyone_degrees_mobile_detector_v3_wrapper.cpython-37m-darwin.so  0x0000000110529e2f fiftyoneDegreesWorksetRelease + 127
2   _fiftyone_degrees_mobile_detector_v3_wrapper.cpython-37m-darwin.so  0x000000011055090d Match::~Match() + 29
3   _fiftyone_degrees_mobile_detector_v3_wrapper.cpython-37m-darwin.so  0x000000011053ca40 _wrap_delete_Match(_object*, _object*) + 96
4   org.python.python               0x000000010dde0cf4 _PyMethodDef_RawFastCallDict + 548
5   org.python.python               0x000000010dddfeb3 _PyObject_FastCallDict + 227
6   org.python.python               0x000000010dde214c object_vacall + 316
7   org.python.python               0x000000010dde2334 PyObject_CallFunctionObjArgs + 148
8   _fiftyone_degrees_mobile_detector_v3_wrapper.cpython-37m-darwin.so  0x0000000110544f99 SwigPyObject_dealloc(_object*) + 249
9   org.python.python               0x000000010de0be04 dict_dealloc + 132
10  org.python.python               0x000000010de294f5 subtype_dealloc + 949
11  org.python.python               0x000000010ddf5854 frame_dealloc + 180
12  org.python.python               0x000000010dde0a97 function_code_fastcall + 263
13  org.python.python               0x000000010de9f8c2 call_function + 738
14  org.python.python               0x000000010de9c92e _PyEval_EvalFrameDefault + 25342
15  org.python.python               0x000000010dea0413 _PyEval_EvalCodeWithName + 2467
16  org.python.python               0x000000010de96554 PyEval_EvalCode + 100
17  org.python.python               0x000000010ded3c31 PyRun_FileExFlags + 209
18  org.python.python               0x000000010ded34aa PyRun_SimpleFileExFlags + 890
19  org.python.python               0x000000010def2903 pymain_main + 6915
20  org.python.python               0x000000010def2e6a _Py_UnixMain + 58
21  libdyld.dylib                   0x00007fff7c6fe3d5 start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0xf000000000000150  rcx: 0x0000000000000307  rdx: 0x0000000110562510
  rdi: 0xf000000000000150  rsi: 0x0000000000000000  rbp: 0x00007ffee1e45bc0  rsp: 0x00007ffee1e45ba0
   r8: 0x00007ffee1e45970   r9: 0x000000010e2878d0  r10: 0x00007ffee1e45be0  r11: 0x0000000000000001
  r12: 0x0000000000000000  r13: 0x0000000000000001  r14: 0xf000000000000158  r15: 0x00007feaa18123a0
  rip: 0x00007fff7c8f0d1b  rfl: 0x0000000000010246  cr2: 0x000000c00008b010

Logical CPU:     2
Error Code:      0x00000000
Trap Number:     13
JoshGrew51D commented 5 years ago

Thanks for providing so much detail, sorry to hear you are having issues.

I am taking a look at this at the moment. Have you been able to run the code successfully at all or does it always segfault?

PavMel commented 5 years ago

Hi, Josh. Thank you for your comment. Time to time I didn't see any error.

JoshGrew51D commented 5 years ago

It looks to be an issue with the objects being cleared up during garbage collection.

Can you try changing some of the scope to global like below and let me know how you get on.

import os
from collections import OrderedDict
from FiftyOneDegrees import fiftyone_degrees_mobile_detector_v3_wrapper
from fiftyone_degrees.mobile_detector.conf import

dataFile = os.path.join(ABSPATH, 'data', '51degrees', '51Degrees-PremiumV3_2.dat')
properties = settings.PROPERTIES
cacheSize = settings.CACHE_SIZE
poolSize = settings.POOL_SIZE

provider = None

def populate_db_init():
    provider = fiftyone_degrees_mobile_detector_v3_wrapper.Provider(
        dataFile,
        properties,
        cacheSize,
        poolSize
    )
    profiles = provider.findProfiles("IsMob8ile", "True")
    print("There are %d mobile profiles in the %s data file." % (profiles.getCount(), provider.getDataSetName()))
    for profile_index in reversed(range(0, profiles.getCount())):
        DeviceId = profiles.getProfileId(profile_index)
        device_object = provider.getMatchForDeviceId("%s" % DeviceId)
        item = OrderedDict([
            ('HardwareVendor', device_object.getValue('HardwareVendor')),
            ('HardwareModel', device_object.getValue('HardwareModel')),
            ('HardwareName', device_object.getValue('HardwareName') or None),
            ('HardwareFamily', device_object.getValue('HardwareFamily') or None),
            ('HardwareProfileSource', device_object.getValue('HardwareProfileSource') or None),
        ])
        print(dict(item), profile_index)
    return ### <<< HAPPENED HERE

if __name__ == '__main__':
    populate_db_init()
PavMel commented 5 years ago

I did it, but nothing changed.

JoshGrew51D commented 4 years ago

Hi There

Apologies for the delay in getting back to you on this one. After further review we have managed to replicate the issue.

It was down to objects being destructed in the wrong order on some occasions. Please can you try the code below...

def populate_db_init():
    import os

    from collections import OrderedDict
    from FiftyOneDegrees import fiftyone_degrees_mobile_detector_v3_wrapper
    from fiftyone_degrees.mobile_detector.conf import settings
    dataFile = '../../data/51Degrees-EnterpriseV3.2.dat'
    properties = settings.PROPERTIES
    cacheSize = settings.CACHE_SIZE
    poolSize = settings.POOL_SIZE
    provider = fiftyone_degrees_mobile_detector_v3_wrapper.Provider(
        dataFile,
        properties,
        cacheSize,
        poolSize
    )
    profiles = provider.findProfiles("IsMobile", "True")
    print("There are %d mobile profiles in the %s data file." % (profiles.getCount(), provider.getDataSetName()))
    for profile_index in reversed(range(0, profiles.getCount())):
        DeviceId = profiles.getProfileId(profile_index)
        device_object = provider.getMatchForDeviceId("%s" % DeviceId)
        item = OrderedDict([
            ('HardwareVendor', device_object.getValue('HardwareVendor')),
            ('HardwareModel', device_object.getValue('HardwareModel')),
            ('HardwareName', device_object.getValue('HardwareName') or None),
            ('HardwareFamily', device_object.getValue('HardwareFamily') or None),
            ('HardwareProfileSource', device_object.getValue('HardwareProfileSource') or None),
        ])
        del device_object
        print(dict(item), profile_index)
    del provider
    return ### <<< HAPPENED HERE

if __name__ == '__main__':
    populate_db_init()
PavMel commented 4 years ago

fixed, closed