Closed tw-bert closed 11 years ago
Don't create huge transactions, you will slow down your redis server.
From redis docs http://redis.io/topics/pipelining
while the client sends commands using pipelining, the server will be forced to queue the replies, using memory. So if you need to send many many commands with pipelining it's better to send this commands up to a given reasonable number, for instance 10k commands, read the replies, and send again other 10k commands and so forth. The speed will be nearly the same, but the additional memory used will be at max the amount needed to queue the replies for this 10k commands.
Hi Luca,
You are right ofcourse. In normal business logic, I would never use huge transactions. I am interested however in checking the size of my transaction (and checking available memory, and comparing) while doing it. How would you do it? I can dive into it myself, minor issue.
Kind regards, tw
That is a good question. I should put a signal/callback hook just before the transaction is sent to redis. I will do that so that smart clients can do something clever in terms of managing the transaction.
At the moment I tend to split big writes into sessions of predefined size (as number of model in the transaction). And the number of models to include in a transaction depends on the size of the model itself. In other words, it is application specific.
If you want to have a crack on that, you can start from where the transaction is committing to redis
https://github.com/lsbardel/python-stdnet/blob/dev/stdnet/backends/redisb/__init__.py
line 839
Hi Luca,
A signal or callback hook sounds like a practical idea. I might look into it, but I'm exploring other stuff in stdnet at the moment, so it may take a while. Right now I'm focussing on read and write performance, and will most probably have a question or two about that. Stdnet adds (naturally) quite some overhead compared to "raw" redis reads and writes. If I transfer a clob of 4MB to redis as a single key, it takes about 300ms to read and 300ms to write. Inside that clob are actually about 25000 parent and child records. Writing with stdnet as singular objects takes about 37 seconds (split into 6 transactions of about 10000 records), and reading (all) about 8 seconds. Completely different things ofcourse, but I'd like the performance to meet somewhere halfway, doing less in stdnet's backend and more in "raw" format. Would the json field help me here? Can I put all child records in one json field of the parent record? I mean ofcourse I can (JSONField.as_string to true - but there's no documentation on 'true', or my own bytearray in a non-symbol field), but can I still deserialize it to stdnet child objects on the client? In this specific scenario, the child records don't need to be serverside queryable at all, and they won't have any relationships except for the parent key.
Interested in your thoughts about this.
Kind regards, tw
That sounds like a huge overhead. 0.3 to 37 is off the scale. Can you profile your code? I can't help you much if I don't see your models and how you are saving them. A JSONField with as_string true is json serialisable object, and it can be used instead of a foreign key.
Hi Luca,
I profiled it, see below. You can ignore my 'decode' calls, they are not optimized yet (I get serialized encoded utf-8 data from a char* in a shared object, which the shared object transfers to a bytearray python object, which I have to decode). Because of company restrictions / proprietary issues, I can't submit the complete code here. Can I send it to the e-mail address mentioned in your profile?
Here's the (obfuscated) profile:
Start testMyDataToStdnet
Tables added: 1524. Fields added: 23825. Number of transactions: 6.
32825530 function calls (32571729 primitive calls) in 55.531 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
367 5.292 0.014 5.292 0.014 {method 'recv' of '_socket.socket' objects}
2303807 4.438 0.000 8.865 0.000 extensions.py:87(encode)
5913838 3.678 0.000 3.805 0.000 {isinstance}
3455730 3.464 0.000 12.644 0.000 extensions.py:98(__pack_gen)
958131 3.299 0.000 7.526 0.000 fields.py:490(to_python)
1949309 2.926 0.000 2.926 0.000 {hasattr}
1025034 2.167 0.000 11.212 0.000 models.py:84(set_field_value)
25349 2.122 0.000 13.956 0.001 base.py:228(is_valid)
1561376/1513726 1.848 0.000 12.652 0.000 fields.py:40(_)
1 1.540 1.540 55.030 55.030 sc_mytest_dummyexport_r.py:43(MyDataToStdnet)
433422 1.507 0.000 2.734 0.000 fields.py:405(to_python)
296568 1.354 0.000 4.058 0.000 sc_mytest_dummyplatform.py:11(FluxDecode)
1391553 1.204 0.000 3.796 0.000 fields.py:345(to_python)
172/133 1.171 0.007 13.817 0.104 {method 'join' of 'str' objects}
1186272 1.078 0.000 1.078 0.000 {method 'replace' of 'bytearray' objects}
25349 1.040 0.000 6.837 0.000 models.py:20(__init__)
537866 0.946 0.000 1.405 0.000 models.py:66(fieldvalue_pairs)
934306 0.875 0.000 1.535 0.000 encoders.py:109(loads)
345744 0.864 0.000 0.864 0.000 {_codecs.utf_8_decode}
344218 0.728 0.000 0.728 0.000 {method 'encode' of 'unicode' objects}
50698 0.672 0.000 2.095 0.000 session.py:139(add)
1075860 0.672 0.000 0.672 0.000 {setattr}
296568 0.568 0.000 1.626 0.000 {method 'decode' of 'bytearray' objects}
463343 0.528 0.000 4.291 0.000 fields.py:306(serialize)
102310/381 0.474 0.000 1.757 0.005 parser.py:48(gets)
101946/408 0.464 0.000 1.760 0.004 parser.py:122(gets)
14 0.451 0.032 0.451 0.032 {method 'sendall' of '_socket.socket' objects}
12 0.448 0.037 36.905 3.075 __init__.py:801(execute_session)
12 0.426 0.035 2.608 0.217 session.py:223(post_commit)
345744 0.415 0.000 1.279 0.000 utf_8.py:15(decode)
25349 0.404 0.000 7.494 0.000 session.py:903(__call__)
344218 0.384 0.000 0.487 0.000 client.py:109(encoding)
76047 0.383 0.000 0.699 0.000 session.py:189(pop)
101452 0.370 0.000 0.370 0.000 parser.py:23(__init__)
344218 0.353 0.000 0.461 0.000 client.py:113(encoding_errors)
1219099/1218666 0.340 0.000 0.340 0.000 {len}
177443 0.330 0.000 0.737 0.000 base.py:469(get_state)
76047 0.316 0.000 0.407 0.000 base.py:391(__init__)
740006 0.310 0.000 0.310 0.000 {getattr}
152520 0.278 0.000 0.388 0.000 parser.py:101(read)
25349 0.260 0.000 0.710 0.000 __init__.py:63(flat_mapping)
715243 0.253 0.000 0.295 0.000 {method 'get' of 'dict' objects}
563309 0.236 0.000 0.236 0.000 {method 'extend' of 'list' objects}
98348 0.223 0.000 0.223 0.000 fields.py:387(to_python)
614166 0.207 0.000 0.207 0.000 {method 'pop' of 'dict' objects}
25349 0.204 0.000 0.204 0.000 {method 'split' of 'bytearray' objects}
25349 0.191 0.000 0.254 0.000 base.py:450(__new__)
71475 0.188 0.000 0.749 0.000 fields.py:644(to_python)
99872 0.169 0.000 0.328 0.000 base.py:477(pkvalue)
50710 0.164 0.000 0.263 0.000 session.py:42(__getitem__)
202792 0.149 0.000 0.149 0.000 base.py:220(pkname)
25349 0.141 0.000 1.766 0.000 session.py:646(add)
25361 0.137 0.000 0.193 0.000 __init__.py:55(_wrap_commit)
6 0.129 0.021 39.707 6.618 session.py:601(commit)
49177 0.127 0.000 0.348 0.000 {method 'decode' of 'str' objects}
99872 0.114 0.000 0.159 0.000 fields.py:281(get_value)
23 0.109 0.005 0.109 0.005 {method 'read' of 'file' objects}
50710 0.107 0.000 0.370 0.000 mapper.py:102(__getitem__)
25361 0.103 0.000 0.103 0.000 collections.py:54(__setitem__)
25349 0.098 0.000 0.112 0.000 collections.py:64(__delitem__)
25361 0.085 0.000 0.281 0.000 session.py:732(manager)
25373 0.075 0.000 0.075 0.000 {repr}
25463 0.072 0.000 0.127 0.000 abc.py:128(__instancecheck__)
189048 0.069 0.000 0.069 0.000 {method 'append' of 'list' objects}
25349 0.069 0.000 0.181 0.000 collections.py:132(pop)
126745 0.068 0.000 0.068 0.000 base.py:412(persistent)
101606 0.067 0.000 0.067 0.000 {method 'readline' of '_io.BytesIO' objects}
25361 0.065 0.000 0.430 0.000 session.py:711(model)
50698 0.062 0.000 0.062 0.000 {method 'find' of 'bytearray' objects}
126745 0.057 0.000 0.057 0.000 base.py:417(iid)
12 0.056 0.005 0.143 0.012 client.py:250(execute_script)
76071 0.056 0.000 0.056 0.000 session.py:88(model)
50727 0.051 0.000 0.078 0.000 session.py:54(meta)
44522 0.048 0.000 0.123 0.000 fields.py:241(get_default)
12 0.047 0.004 0.080 0.007 extensions.py:220(_call)
25349 0.047 0.000 0.529 0.000 base.py:224(pk_to_python)
50988 0.044 0.000 0.044 0.000 {built-in method __new__ of type object at 0x9001000a12f07f8}
25349 0.044 0.000 0.255 0.000 __init__.py:36(iterpair)
51281 0.043 0.000 0.043 0.000 {method 'read' of '_io.BytesIO' objects}
25695 0.042 0.000 0.042 0.000 _weakrefset.py:68(__contains__)
25349 0.041 0.000 13.998 0.001 models.py:130(is_valid)
25361 0.037 0.000 0.051 0.000 session.py:78(backend)
12 0.036 0.003 0.179 0.015 __init__.py:788(odmrun)
50698 0.033 0.000 0.033 0.000 base.py:185(type)
25409 0.033 0.000 0.042 0.000 session.py:974(__hash__)
5 0.032 0.006 0.032 0.006 {posix.popen}
15 0.031 0.002 0.031 0.002 client.py:1774(execute_command)
25349 0.026 0.000 0.037 0.000 py2py3.py:47(<lambda>)
22 0.024 0.001 0.824 0.037 __init__.py:1(<module>)
24 0.024 0.001 0.030 0.001 collections.py:282(namedtuple)
49174 0.023 0.000 0.023 0.000 fields.py:394(serialize)
25385 0.022 0.000 0.038 0.000 collections.py:116(itervalues)
25349 0.021 0.000 0.021 0.000 base.py:495(__set_session)
50710 0.021 0.000 0.021 0.000 {function __getitem__ at 0x110d039e8}
381/82 0.019 0.000 0.054 0.001 sre_parse.py:380(_parse)
25409 0.015 0.000 0.015 0.000 collections.py:73(__iter__)
25349 0.015 0.000 0.015 0.000 models.py:177(get_state_action)
26263 0.015 0.000 0.015 0.000 {method 'pop' of 'list' objects}
25349 0.014 0.000 0.014 0.000 base.py:406(action)
25361 0.014 0.000 0.014 0.000 session.py:875(backend)
25385 0.013 0.000 0.013 0.000 session.py:564(router)
772/79 0.013 0.000 0.039 0.000 sre_compile.py:33(_compile)
25349 0.013 0.000 0.013 0.000 base.py:525(raise_kwargs)
5295 0.012 0.000 0.014 0.000 sre_parse.py:183(__next)
2 0.011 0.006 0.036 0.018 jsontools.py:1(<module>)
25368 0.011 0.000 0.011 0.000 {method 'iteritems' of 'dict' objects}
356 0.011 0.000 0.016 0.000 sre_compile.py:208(_optimize_charset)
1 0.010 0.010 0.071 0.071 httpurl.py:44(<module>)
25409 0.009 0.000 0.009 0.000 {hash}
12 0.008 0.001 0.046 0.004 session.py:116(dirty)
12 0.008 0.001 21.347 1.779 client.py:345(execute)
2 0.008 0.004 0.008 0.004 {method 'connect' of '_socket.socket' objects}
18 0.007 0.000 39.575 2.199 session.py:464(_commit)
1111/426 0.007 0.000 0.008 0.000 sre_parse.py:141(getwidth)
1 0.007 0.007 0.017 0.017 sc_mytest_dummyexport_r_idetest.py:6(getTestData)
12 0.006 0.000 0.086 0.007 extensions.py:49(execute_script)
1 0.006 0.006 0.017 0.017 platform.py:1000(_syscmd_uname)
3287 0.005 0.000 0.008 0.000 sre_parse.py:131(__getitem__)
4594 0.005 0.000 0.017 0.000 sre_parse.py:202(get)
58 0.005 0.000 0.005 0.000 sre_compile.py:259(_mk_bitmap)
3 0.005 0.002 0.197 0.066 __init__.py:2(<module>)
1 0.005 0.005 0.022 0.022 client.py:1(<module>)
1 0.005 0.005 0.006 0.006 __init__.py:4(<module>)
3 0.004 0.001 0.012 0.004 base.py:1(<module>)
1 0.004 0.004 0.006 0.006 random.py:40(<module>)
1 0.004 0.004 0.009 0.009 posixsystem.py:1(<module>)
1 0.003 0.003 0.029 0.029 __init__.py:46(<module>)
3 0.003 0.001 0.003 0.001 {posix.open}
41 0.003 0.000 7.059 0.172 client.py:69(_read_response)
1 0.003 0.003 0.003 0.003 socket.py:45(<module>)
367 0.003 0.000 0.003 0.000 parser.py:117(feed)
1 0.003 0.003 0.005 0.005 __init__.py:44(<module>)
1 0.003 0.003 0.004 0.004 collections.py:1(<module>)
356 0.003 0.000 0.020 0.000 sre_compile.py:179(_compile_charset)
1 0.003 0.003 0.006 0.006 subprocess.py:389(<module>)
1 0.003 0.003 0.003 0.003 struct.py:97(setcache)
1 0.002 0.002 0.007 0.007 urllib2.py:76(<module>)
284/80 0.002 0.000 0.054 0.001 sre_parse.py:302(_parse_sub)
1 0.002 0.002 0.012 0.012 _compat.py:1(<module>)
3 0.002 0.001 0.002 0.001 {posix.unlink}
223 0.002 0.000 0.112 0.001 re.py:226(_compile)
3 0.002 0.001 0.005 0.002 utils.py:1(<module>)
1 0.002 0.002 0.024 0.024 jsonrpc.py:11(<module>)
1 0.002 0.002 55.565 55.565 sc_mytest_dummyexport_r_idetest.py:1(<module>)
1 0.002 0.002 0.011 0.011 tempfile.py:18(<module>)
1 0.002 0.002 0.003 0.003 io.py:34(<module>)
1 0.002 0.002 0.009 0.009 utils.py:5(<module>)
1 0.002 0.002 0.002 0.002 __init__.py:24(<module>)
1 0.002 0.002 0.005 0.005 py2py3.py:3(<module>)
1 0.002 0.002 0.004 0.004 pickle.py:25(<module>)
1 0.002 0.002 0.064 0.064 __init__.py:67(<module>)
1 0.002 0.002 0.016 0.016 decimal.py:116(<module>)
1 0.002 0.002 0.009 0.009 threads.py:1(<module>)
1 0.002 0.002 0.073 0.073 sockets.py:9(<module>)
2452 0.002 0.000 0.003 0.000 sre_parse.py:196(match)
1 0.002 0.002 0.002 0.002 httplib.py:67(<module>)
1 0.002 0.002 0.006 0.006 config.py:28(<module>)
2 0.002 0.001 0.102 0.051 util.py:91(_findLib_gcc)
1 0.002 0.002 0.003 0.003 urllib.py:23(<module>)
2 0.002 0.001 0.002 0.001 exceptions.py:2(<module>)
1 0.001 0.001 0.006 0.006 config.py:25(<module>)
2 0.001 0.001 0.010 0.005 version.py:1(<module>)
1 0.001 0.001 0.158 0.158 uuid.py:45(<module>)
1 0.001 0.001 0.001 0.001 csv.py:4(<module>)
2 0.001 0.001 0.002 0.001 __init__.py:5(<module>)
454 0.001 0.000 0.001 0.000 {method 'format' of 'str' objects}
116 0.001 0.000 0.003 0.000 config.py:477(__init__)
97 0.001 0.000 0.002 0.000 {method '__reduce_ex__' of 'object' objects}
1143 0.001 0.000 0.002 0.000 sre_parse.py:139(append)
1 0.001 0.001 0.026 0.026 __init__.py:48(<module>)
347 0.001 0.000 0.002 0.000 sre_compile.py:355(_simple)
4 0.001 0.000 0.010 0.003 {__import__}
1631 0.001 0.000 0.002 0.000 sre_parse.py:127(__len__)
1672 0.001 0.000 0.001 0.000 {min}
79 0.001 0.000 0.109 0.001 sre_compile.py:496(compile)
2 0.001 0.001 0.035 0.018 util.py:214(_findSoname_ldconfig)
1 0.001 0.001 0.198 0.198 async.py:9(<module>)
1 0.001 0.001 0.003 0.003 scanner.py:2(<module>)
1 0.001 0.001 0.002 0.002 html.py:14(<module>)
1 0.001 0.001 0.001 0.001 hashlib.py:55(<module>)
2 0.001 0.001 0.002 0.001 importer.py:1(<module>)
1 0.001 0.001 0.022 0.022 __init__.py:9(<module>)
79 0.001 0.000 0.011 0.000 sre_compile.py:362(_compile_info)
1 0.001 0.001 0.001 0.001 pep.py:1(<module>)
2 0.001 0.001 0.002 0.001 dispatcher.py:1(<module>)
1 0.001 0.001 0.012 0.012 util.py:4(<module>)
1 0.001 0.001 0.008 0.008 __init__.py:99(<module>)
1 0.001 0.001 0.200 0.200 session.py:1(<module>)
791 0.001 0.000 0.001 0.000 sre_parse.py:91(__init__)
1 0.001 0.001 0.005 0.005 connection.py:1(<module>)
1 0.001 0.001 0.002 0.002 client.py:26(<module>)
1 0.001 0.001 0.001 0.001 __init__.py:64(<module>)
1 0.001 0.001 0.003 0.003 content.py:76(<module>)
2 0.001 0.000 0.001 0.001 handlers.py:1(<module>)
1 0.001 0.001 0.001 0.001 __init__.py:82(<module>)
30/18 0.001 0.000 39.576 2.199 {method 'send' of 'generator' objects}
1 0.001 0.001 0.008 0.008 urlparse.py:29(<module>)
80/79 0.001 0.000 0.056 0.001 sre_parse.py:676(parse)
1 0.001 0.001 0.006 0.006 query.py:1(<module>)
919 0.001 0.000 0.001 0.000 collections.py:323(<genexpr>)
1 0.001 0.001 0.001 0.001 util.py:35(<module>)
1 0.001 0.001 0.001 0.001 _parseaddr.py:7(<module>)
1 0.001 0.001 0.001 0.001 mimetypes.py:24(<module>)
1 0.001 0.001 0.001 0.001 fields.py:1(<module>)
1 0.001 0.001 0.003 0.003 sc_mytest_dummyexport_r.py:3(<module>)
39 0.001 0.000 7.059 0.181 client.py:357(parse_response)
1 0.001 0.001 0.001 0.001 handlers.py:24(<module>)
1 0.001 0.001 0.001 0.001 __init__.py:36(<module>)
1 0.001 0.001 0.001 0.001 __init__.py:68(<module>)
1 0.001 0.001 0.001 0.001 middleware.py:51(<module>)
485 0.001 0.000 0.001 0.000 urllib.py:1200(<genexpr>)
97 0.001 0.000 0.002 0.000 copy.py:306(_reconstruct)
1 0.001 0.001 0.004 0.004 defer.py:1(<module>)
91/74 0.001 0.000 0.002 0.000 abc.py:148(__subclasscheck__)
49 0.001 0.000 0.002 0.000 collections.py:38(__init__)
1 0.001 0.001 0.001 0.001 {method 'encode' of 'str' objects}
1 0.001 0.001 0.002 0.002 Queue.py:1(<module>)
242 0.001 0.000 0.001 0.000 sre_parse.py:226(_class_escape)
1 0.001 0.001 0.001 0.001 heapq.py:31(<module>)
1 0.001 0.001 0.036 0.036 inspect.py:25(<module>)
1 0.001 0.001 0.001 0.001 bisect.py:1(<module>)
1 0.001 0.001 0.004 0.004 multipart.py:7(<module>)
1 0.001 0.001 0.003 0.003 related.py:1(<module>)
1 0.001 0.001 0.001 0.001 server.py:7(<module>)
1 0.001 0.001 0.006 0.006 log.py:1(<module>)
485 0.001 0.000 0.001 0.000 urlparse.py:331(<genexpr>)
1 0.001 0.001 0.002 0.002 mailbox.py:39(<module>)
18 0.001 0.000 0.001 0.000 {open}
1 0.001 0.001 0.007 0.007 mapper.py:1(<module>)
1 0.001 0.001 0.006 0.006 struct.py:1(<module>)
3 0.001 0.000 0.001 0.000 __init__.py:349(__init__)
1 0.001 0.001 0.001 0.001 transport.py:1(<module>)
1 0.001 0.001 0.005 0.005 decoder.py:2(<module>)
3 0.001 0.000 0.007 0.002 config.py:326(update_settings)
5 0.001 0.000 0.001 0.000 {method 'close' of 'file' objects}
58 0.001 0.000 0.001 0.000 {method 'sub' of '_sre.SRE_Pattern' objects}
1 0.001 0.001 0.001 0.001 argparse.py:62(<module>)
12 0.001 0.000 21.339 1.778 client.py:142(request_pipeline)
1 0.001 0.001 0.018 0.018 cookielib.py:26(<module>)
4 0.001 0.000 0.001 0.000 __init__.py:382(__getitem__)
1 0.001 0.001 0.003 0.003 actor.py:1(<module>)
266 0.001 0.000 0.001 0.000 sre_parse.py:258(_escape)
1 0.001 0.001 0.001 0.001 runtime.py:10(<module>)
1 0.001 0.001 0.001 0.001 eventloop.py:1(<module>)
2 0.001 0.000 0.001 0.000 {posix.getcwd}
1 0.001 0.001 0.004 0.004 arbiter.py:1(<module>)
1 0.001 0.001 0.005 0.005 wrappers.py:39(<module>)
35 0.001 0.000 0.005 0.000 config.py:352(__new__)
1 0.001 0.001 0.056 0.056 rpc.py:17(<module>)
1 0.001 0.001 0.001 0.001 mimetools.py:1(<module>)
1 0.001 0.001 0.001 0.001 encoders.py:26(<module>)
97 0.001 0.000 0.004 0.000 copy.py:66(copy)
1 0.001 0.001 0.002 0.002 access.py:1(<module>)
1512 0.001 0.000 0.001 0.000 {chr}
19 0.000 0.000 0.002 0.000 base.py:138(__init__)
1 0.000 0.000 0.000 0.000 {_socket.gethostbyname}
16 0.000 0.000 0.000 0.000 {_hashlib.openssl_sha1}
1 0.000 0.000 0.001 0.001 process.py:35(<module>)
11 0.000 0.000 0.002 0.000 abc.py:86(__new__)
12 0.000 0.000 7.523 0.627 client.py:350(send_commands)
18/6 0.000 0.000 39.577 6.596 defer.py:729(_consume)
12 0.000 0.000 2.609 0.217 session.py:479(_post_commit)
1 0.000 0.000 0.029 0.029 tokenize.py:23(<module>)
24/12 0.000 0.000 2.616 0.218 defer.py:797(__init__)
1 0.000 0.000 0.000 0.000 pool.py:35(<module>)
2 0.000 0.000 0.002 0.001 extensions.py:1(<module>)
81/6 0.000 0.000 39.578 6.596 defer.py:131(default_maybe_async)
79 0.000 0.000 0.051 0.001 sre_compile.py:481(_code)
36 0.000 0.000 0.001 0.000 client.py:1739(reset)
1 0.000 0.000 0.001 0.001 info.py:4(<module>)
110 0.000 0.000 0.002 0.000 {all}
40 0.000 0.000 0.001 0.000 fields.py:151(__init__)
157 0.000 0.000 0.000 0.000 {method 'update' of 'dict' objects}
1 0.000 0.000 0.001 0.001 calendar.py:6(<module>)
1 0.000 0.000 0.001 0.001 __init__.py:14(<module>)
2 0.000 0.000 0.000 0.000 structures.py:1(<module>)
12 0.000 0.000 0.000 0.000 encoder.py:212(iterencode)
79 0.000 0.000 0.000 0.000 {_sre.compile}
1 0.000 0.000 0.000 0.000 exceptions.py:1(<module>)
34 0.000 0.000 0.000 0.000 defer.py:231(__call__)
39/24 0.000 0.000 0.004 0.000 defer.py:864(_add)
12 0.000 0.000 13.816 1.151 extensions.py:113(pack_pipeline)
2 0.000 0.000 0.001 0.000 saferef.py:6(<module>)
12 0.000 0.000 0.087 0.007 extensions.py:202(__call__)
24 0.000 0.000 0.047 0.002 session.py:289(backends_data)
158 0.000 0.000 0.000 0.000 sre_compile.py:475(isstring)
429 0.000 0.000 0.000 0.000 {method 'startswith' of 'str' objects}
81/6 0.000 0.000 39.578 6.596 defer.py:144(maybe_async)
157 0.000 0.000 0.000 0.000 sre_parse.py:73(opengroup)
2 0.000 0.000 0.000 0.000 local.py:1(<module>)
1 0.000 0.000 0.001 0.001 handlers.py:98(<module>)
4 0.000 0.000 0.000 0.000 {posix.urandom}
1 0.000 0.000 0.007 0.007 route.py:52(<module>)
1 0.000 0.000 0.001 0.001 threading.py:1(<module>)
1 0.000 0.000 0.000 0.000 tempfile.py:497(SpooledTemporaryFile)
1 0.000 0.000 0.001 0.001 monitor.py:1(<module>)
1 0.000 0.000 0.002 0.002 plugins.py:1(<module>)
1 0.000 0.000 0.001 0.001 models.py:65(<module>)
97 0.000 0.000 0.002 0.000 config.py:398(set)
1 0.000 0.000 0.005 0.005 globals.py:1(<module>)
1 0.000 0.000 0.000 0.000 protocols.py:1(<module>)
18/6 0.000 0.000 39.578 6.596 defer.py:721(__init__)
1 0.000 0.000 0.001 0.001 glob.py:1(<module>)
31 0.000 0.000 0.000 0.000 {method 'findall' of '_sre.SRE_Pattern' objects}
48 0.000 0.000 0.001 0.000 access.py:41(thread_local_data)
49 0.000 0.000 0.001 0.000 _abcoll.py:526(update)
19 0.000 0.000 0.002 0.000 base.py:18(get_fields)
21 0.000 0.000 0.000 0.000 posixpath.py:68(join)
31 0.000 0.000 0.002 0.000 textwrap.py:374(dedent)
809 0.000 0.000 0.000 0.000 {method 'isalnum' of 'str' objects}
1068 0.000 0.000 0.000 0.000 {ord}
27 0.000 0.000 4.194 0.155 client.py:1898(parse_response)
1 0.000 0.000 0.000 0.000 __init__.py:43(<module>)
39 0.000 0.000 13.793 0.354 extensions.py:116(<lambda>)
1 0.000 0.000 0.000 0.000 async.py:13(<module>)
24 0.000 0.000 0.000 0.000 inspect.py:743(getargs)
42/18 0.000 0.000 0.001 0.000 copy.py:145(deepcopy)
1 0.000 0.000 0.000 0.000 structfields.py:1(<module>)
48 0.000 0.000 0.000 0.000 _weakrefset.py:36(__init__)
19 0.000 0.000 0.005 0.000 base.py:368(__new__)
1 0.000 0.000 0.000 0.000 pidfile.py:1(<module>)
68 0.000 0.000 0.001 0.000 _weakrefset.py:58(__iter__)
1 0.000 0.000 0.000 0.000 websocket.py:5(<module>)
1 0.000 0.000 0.000 0.000 urllib.py:851(ftpwrapper)
1 0.000 0.000 0.001 0.001 dis.py:1(<module>)
42 0.000 0.000 0.001 0.000 defer.py:531(callback)
1 0.000 0.000 0.000 0.000 hex_codec.py:8(<module>)
117 0.000 0.000 0.002 0.000 re.py:134(match)
22 0.000 0.000 0.001 0.000 __init__.py:1007(getLogger)
916 0.000 0.000 0.000 0.000 sre_compile.py:25(_identityfunction)
1 0.000 0.000 0.000 0.000 tcp.py:1(<module>)
31 0.000 0.000 0.000 0.000 sre_parse.py:217(isname)
1 0.000 0.000 0.018 0.018 __init__.py:372(HttpClient)
1 0.000 0.000 0.005 0.005 Cookie.py:206(<module>)
12 0.000 0.000 0.000 0.000 base.py:308(as_dict)
1 0.000 0.000 0.000 0.000 zset.py:1(<module>)
1 0.000 0.000 0.000 0.000 clients.py:1(<module>)
1 0.000 0.000 0.000 0.000 colors.py:22(<module>)
1 0.000 0.000 0.000 0.000 text.py:1(<module>)
27 0.000 0.000 4.194 0.155 client.py:386(parse_response)
31 0.000 0.000 0.000 0.000 {posix.getpid}
12 0.000 0.000 0.001 0.000 session.py:61(__init__)
97 0.000 0.000 0.000 0.000 copy_reg.py:92(__newobj__)
357 0.000 0.000 0.000 0.000 sre_parse.py:135(__setitem__)
1 0.000 0.000 0.000 0.000 <string>:1(range_lookup_info)
1 0.000 0.000 0.000 0.000 opcode.py:5(<module>)
128 0.000 0.000 0.000 0.000 {method 'split' of 'str' objects}
1 0.000 0.000 0.000 0.000 populate.py:1(<module>)
19 0.000 0.000 0.000 0.000 related.py:67(do_pending_lookups)
52 0.000 0.000 0.000 0.000 fields.py:202(register_with_model)
1 0.000 0.000 0.000 0.000 proxy.py:1(<module>)
14/12 0.000 0.000 0.462 0.038 connection.py:283(send_packed_command)
85 0.000 0.000 0.004 0.000 config.py:416(copy)
120 0.000 0.000 0.000 0.000 {method 'match' of '_sre.SRE_Pattern' objects}
110 0.000 0.000 0.001 0.000 collections.py:349(<genexpr>)
3 0.000 0.000 0.000 0.000 __init__.py:78(CFUNCTYPE)
17 0.000 0.000 0.001 0.000 extensions.py:129(read_lua_file)
41 0.000 0.000 7.059 0.172 client.py:58(read_response)
24/12 0.000 0.000 2.615 0.218 defer.py:848(update)
80 0.000 0.000 0.001 0.000 sre_parse.py:179(__init__)
23 0.000 0.000 0.000 0.000 threading.py:146(acquire)
157 0.000 0.000 0.000 0.000 sre_parse.py:84(closegroup)
208 0.000 0.000 0.000 0.000 {method 'items' of 'dict' objects}
34 0.000 0.000 0.000 0.000 base.py:26(all_signals)
57 0.000 0.000 0.000 0.000 client.py:319(response_callbacks)
1 0.000 0.000 0.000 0.000 ssl.py:56(<module>)
42 0.000 0.000 0.000 0.000 defer.py:423(__init__)
1 0.000 0.000 0.002 0.002 encoder.py:2(<module>)
3 0.000 0.000 0.001 0.000 tempfile.py:135(next)
1 0.000 0.000 0.000 0.000 concurrency.py:1(<module>)
31 0.000 0.000 0.002 0.000 config.py:374(fmt_desc)
14 0.000 0.000 0.000 0.000 __init__.py:147(_check_size)
1 0.000 0.000 0.000 0.000 skiplist.py:4(<module>)
24 0.000 0.000 0.000 0.000 extensions.py:43(loaded_scripts)
3 0.000 0.000 0.000 0.000 __init__.py:493(PYFUNCTYPE)
1 0.000 0.000 0.004 0.004 tempfile.py:176(_get_default_tempdir)
18 0.000 0.000 0.459 0.026 socket.py:223(meth)
28 0.000 0.000 0.027 0.001 {map}
24 0.000 0.000 0.001 0.000 config.py:582(_validate_callable)
89 0.000 0.000 0.000 0.000 {method 'copy' of 'dict' objects}
12 0.000 0.000 0.001 0.000 client.py:377(on_response)
5 0.000 0.000 0.000 0.000 handlers.py:101(__new__)
1 0.000 0.000 0.000 0.000 base64.py:3(<module>)
5 0.000 0.000 0.000 0.000 {dir}
110 0.000 0.000 0.000 0.000 collections.py:347(<genexpr>)
1 0.000 0.000 0.001 0.001 headers.py:6(<module>)
1 0.000 0.000 0.000 0.000 plugins.py:32(<module>)
1 0.000 0.000 0.000 0.000 client.py:1692(BasePipeline)
24 0.000 0.000 0.001 0.000 inspect.py:803(getargspec)
60 0.000 0.000 0.000 0.000 client.py:315(connection_pool)
42 0.000 0.000 0.000 0.000 copy.py:267(_keep_alive)
79 0.000 0.000 0.002 0.000 config.py:85(_wrapped)
1 0.000 0.000 0.003 0.003 __init__.py:100(<module>)
18 0.000 0.000 0.001 0.000 access.py:25(get_request_loop)
93 0.000 0.000 0.000 0.000 {range}
1 0.000 0.000 0.000 0.000 handlers.py:42(BaseHandler)
12 0.000 0.000 0.000 0.000 connection.py:394(get_connection)
4 0.000 0.000 0.007 0.002 re.py:139(search)
81 0.000 0.000 0.000 0.000 defer.py:119(default_maybe_failure)
7 0.000 0.000 0.001 0.000 handlers.py:209(__new__)
1 0.000 0.000 0.000 0.000 _endian.py:4(<module>)
12 0.000 0.000 0.000 0.000 __init__.py:41(callback)
31 0.000 0.000 0.001 0.000 dispatcher.py:148(send)
110/75 0.000 0.000 0.001 0.000 {issubclass}
19 0.000 0.000 0.000 0.000 __init__.py:1053(_fixupParents)
1 0.000 0.000 0.000 0.000 {_io.open}
257 0.000 0.000 0.000 0.000 Cookie.py:313(<genexpr>)
1 0.000 0.000 0.000 0.000 connection.py:35(<module>)
12 0.000 0.000 0.001 0.000 __init__.py:782(meta)
39 0.000 0.000 0.000 0.000 defer.py:916(_setitem)
48 0.000 0.000 0.000 0.000 access.py:28(process_local_data)
159 0.000 0.000 0.000 0.000 defer.py:106(is_async)
42 0.000 0.000 0.000 0.000 defer.py:566(_run_callbacks)
8 0.000 0.000 0.000 0.000 handlers.py:196(get_roule_methods)
1 0.000 0.000 0.000 0.000 handlers.py:160(TimedRotatingFileHandler)
81 0.000 0.000 0.000 0.000 inspect.py:164(isgenerator)
81 0.000 0.000 0.000 0.000 defer.py:78(is_exc_info_error)
1 0.000 0.000 0.000 0.000 base.py:427(Model)
6 0.000 0.000 0.047 0.008 session.py:751(backends_data)
24 0.000 0.000 0.001 0.000 defer.py:902(_finish)
174 0.000 0.000 0.000 0.000 defer.py:469(done)
157 0.000 0.000 0.000 0.000 {method 'remove' of 'list' objects}
80 0.000 0.000 0.000 0.000 sre_parse.py:68(__init__)
49 0.000 0.000 0.000 0.000 extensions.py:62(address)
3 0.000 0.000 0.000 0.000 tempfile.py:124(rng)
24 0.000 0.000 0.000 0.000 client.py:103(_checkpid)
1 0.000 0.000 0.000 0.000 utf_8.py:8(<module>)
14 0.000 0.000 0.000 0.000 globals.py:38(hashmodel)
51 0.000 0.000 0.000 0.000 _weakrefset.py:81(add)
1 0.000 0.000 0.001 0.001 numbers.py:6(<module>)
2 0.000 0.000 0.138 0.069 util.py:241(find_library)
85 0.000 0.000 0.000 0.000 config.py:502(__getstate__)
24 0.000 0.000 0.001 0.000 defer.py:836(lock)
1 0.000 0.000 0.000 0.000 search.py:1(<module>)
142 0.000 0.000 0.000 0.000 {method 'lower' of 'str' objects}
12 0.000 0.000 0.000 0.000 client.py:326(__init__)
20 0.000 0.000 0.000 0.000 __init__.py:1109(__init__)
262 0.000 0.000 0.000 0.000 {method 'add' of 'set' objects}
12 0.000 0.000 0.001 0.000 encoder.py:186(encode)
52 0.000 0.000 0.000 0.000 abc.py:89(<genexpr>)
5 0.000 0.000 0.000 0.000 re.py:204(escape)
18 0.000 0.000 0.000 0.000 random.py:272(choice)
1 0.000 0.000 0.001 0.001 dates.py:1(<module>)
2 0.000 0.000 0.000 0.000 {_ctypes.POINTER}
171 0.000 0.000 0.000 0.000 sre_parse.py:211(isident)
1 0.000 0.000 0.000 0.000 client.py:184(StrictRedis)
117 0.000 0.000 0.000 0.000 client.py:311(client)
12 0.000 0.000 0.000 0.000 session.py:110(deleted)
12 0.000 0.000 0.001 0.000 __init__.py:193(dumps)
75 0.000 0.000 0.104 0.001 re.py:188(compile)
49 0.000 0.000 0.000 0.000 threading.py:1150(currentThread)
65 0.000 0.000 0.000 0.000 {method 'replace' of 'str' objects}
78 0.000 0.000 0.000 0.000 base.py:319(<genexpr>)
38 0.000 0.000 0.000 0.000 _weakrefset.py:26(__exit__)
14 0.000 0.000 0.000 0.000 session.py:45(__setitem__)
1 0.000 0.000 55.030 55.030 sc_mytest_dummyexport_r_idetest.py:11(testMyDataToStdnet)
6 0.000 0.000 39.578 6.596 session.py:446(commit)
23 0.000 0.000 0.000 0.000 __init__.py:210(_acquireLock)
12 0.000 0.000 2.616 0.218 defer.py:93(multi_async)
2 0.000 0.000 0.000 0.000 socket.py:185(__init__)
1 0.000 0.000 0.000 0.000 structures.py:28(<module>)
1 0.000 0.000 0.000 0.000 security.py:3(<module>)
18 0.000 0.000 0.001 0.000 defer.py:760(_conclude)
5 0.000 0.000 0.000 0.000 posixpath.py:321(normpath)
23 0.000 0.000 0.000 0.000 threading.py:186(release)
3 0.000 0.000 0.000 0.000 decimal.py:3782(__init__)
38 0.000 0.000 0.000 0.000 _weakrefset.py:20(__enter__)
15 0.000 0.000 0.001 0.000 extensions.py:144(__new__)
1 0.000 0.000 0.000 0.000 timeutils.py:2(<module>)
12 0.000 0.000 0.000 0.000 __init__.py:205(basekey)
30 0.000 0.000 0.000 0.000 config.py:172(__setattr__)
12 0.000 0.000 0.000 0.000 extensions.py:122(script_callback)
18/6 0.000 0.000 39.578 6.596 defer.py:239(call)
6 0.000 0.000 0.001 0.000 session.py:546(__init__)
81 0.000 0.000 0.001 0.000 defer.py:169(maybe_failure)
161 0.000 0.000 0.000 0.000 {max}
22 0.000 0.000 0.001 0.000 __init__.py:1549(getLogger)
1 0.000 0.000 0.000 0.000 jsonrpc.py:45(JSONRPC)
15 0.000 0.000 0.000 0.000 fields.py:506(__init__)
110 0.000 0.000 0.000 0.000 {_sre.getlower}
48 0.000 0.000 0.000 0.000 access.py:19(is_mainthread)
1 0.000 0.000 0.000 0.000 consts.py:28(<module>)
19 0.000 0.000 0.000 0.000 __init__.py:96(unique_tuple)
18/6 0.000 0.000 39.578 6.596 defer.py:232(_)
2 0.000 0.000 0.002 0.001 tempfile.py:230(_mkstemp_inner)
18 0.000 0.000 0.001 0.000 copy.py:253(_deepcopy_dict)
1 0.000 0.000 0.000 0.000 util.py:1(<module>)
19 0.000 0.000 0.000 0.000 base.py:31(make_app_label)
1 0.000 0.000 0.000 0.000 decimal.py:506(Decimal)
12 0.000 0.000 0.000 0.000 pep.py:151(get_event_loop)
1 0.000 0.000 0.002 0.002 sc_mytest_dummyimport_dr.py:3(<module>)
2 0.000 0.000 0.008 0.004 connection.py:239(_connect)
1 0.000 0.000 0.000 0.000 socket.py:179(_socketobject)
75 0.000 0.000 0.000 0.000 defer.py:110(is_failure)
2 0.000 0.000 0.001 0.001 __init__.py:71(search_function)
13 0.000 0.000 0.001 0.000 abc.py:105(register)
3 0.000 0.000 0.000 0.000 random.py:100(seed)
2 0.000 0.000 0.000 0.000 urlparse.py:167(urlsplit)
3 0.000 0.000 0.000 0.000 {function seed at 0x110597278}
15 0.000 0.000 0.000 0.000 client.py:1807(pipeline_execute_command)
12 0.000 0.000 0.000 0.000 connection.py:411(release)
18 0.000 0.000 0.001 0.000 eventloop.py:55(get_request_loop)
19 0.000 0.000 0.000 0.000 dispatcher.py:213(_live_receivers)
1 0.000 0.000 0.000 0.000 SocketServer.py:120(<module>)
12 0.000 0.000 0.001 0.000 extensions.py:192(sha1)
15 0.000 0.000 0.000 0.000 extensions.py:36(get_script)
1 0.000 0.000 0.000 0.000 arity.py:1(<module>)
1 0.000 0.000 0.000 0.000 parser.py:1(<module>)
3 0.000 0.000 0.007 0.002 config.py:121(__init__)
27 0.000 0.000 0.001 0.000 re.py:144(sub)
19 0.000 0.000 0.000 0.000 proxy.py:52(__call__)
36 0.000 0.000 0.000 0.000 dispatcher.py:175(send_robust)
12 0.000 0.000 0.002 0.000 defer.py:886(_make)
12 0.000 0.000 0.000 0.000 session.py:122(iterdirty)
6 0.000 0.000 0.000 0.000 decimal.py:515(__new__)
24 0.000 0.000 0.000 0.000 defer.py:269(__init__)
1 0.000 0.000 0.014 0.014 platform.py:10(<module>)
6 0.000 0.000 0.000 0.000 session.py:384(__init__)
48 0.000 0.000 0.000 0.000 inspect.py:142(isfunction)
4 0.000 0.000 0.000 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
281 0.000 0.000 0.000 0.000 {id}
30 0.000 0.000 0.001 0.000 eventloop.py:52(get_event_loop)
1 0.000 0.000 0.000 0.000 <string>:1(<module>)
63 0.000 0.000 0.000 0.000 client.py:338(is_transaction)
2 0.000 0.000 0.011 0.005 mapper.py:118(register)
24 0.000 0.000 0.000 0.000 defer.py:64(iterdata)
42 0.000 0.000 0.000 0.000 defer.py:549(result_or_self)
8/5 0.000 0.000 0.000 0.000 mapper.py:268(models_from_model)
4 0.000 0.000 0.000 0.000 weakref.py:47(__init__)
88 0.000 0.000 0.000 0.000 config.py:89(ordered_settings)
14 0.000 0.000 0.000 0.000 {zip}
51 0.000 0.000 0.000 0.000 extensions.py:117(<genexpr>)
13 0.000 0.000 0.000 0.000 extensions.py:179(__init__)
56 0.000 0.000 0.000 0.000 {method 'strip' of 'str' objects}
110 0.000 0.000 0.000 0.000 {method '__contains__' of 'frozenset' objects}
2 0.000 0.000 0.004 0.002 tempfile.py:258(gettempdir)
73 0.000 0.000 0.000 0.000 {time.time}
1 0.000 0.000 0.000 0.000 session.py:58(SessionModel)
31 0.000 0.000 0.000 0.000 {method 'acquire' of 'thread.lock' objects}
1 0.000 0.000 0.000 0.000 servers.py:1(<module>)
2 0.000 0.000 0.000 0.000 connection.py:329(pack_command)
1 0.000 0.000 0.000 0.000 html.py:1(<module>)
1 0.000 0.000 0.000 0.000 calendar.py:47(_localized_month)
18 0.000 0.000 2.611 0.145 session.py:472(<genexpr>)
1 0.000 0.000 0.017 0.017 platform.py:1166(uname)
12 0.000 0.000 0.000 0.000 client.py:1730(__del__)
12 0.000 0.000 0.001 0.000 defer.py:440(event_loop)
15 0.000 0.000 0.000 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
5 0.000 0.000 0.000 0.000 posixpath.py:350(abspath)
1 0.000 0.000 0.000 0.000 token.py:3(<module>)
1 0.000 0.000 0.000 0.000 tempfile.py:148(_candidate_tempdir_list)
40 0.000 0.000 0.000 0.000 fields.py:190(error_extras)
2 0.000 0.000 0.000 0.000 tempfile.py:120(__init__)
6 0.000 0.000 0.001 0.000 mapper.py:231(session)
2 0.000 0.000 0.000 0.000 tempfile.py:48(_set_cloexec)
4 0.000 0.000 0.000 0.000 uuid.py:101(__init__)
12 0.000 0.000 0.000 0.000 client.py:197(pipeline)
12 0.000 0.000 0.000 0.000 config.py:563(validate_string)
20 0.000 0.000 0.000 0.000 {_struct.calcsize}
2 0.000 0.000 0.010 0.005 __init__.py:374(getdb)
40 0.000 0.000 0.000 0.000 dispatcher.py:13(_make_id)
6 0.000 0.000 0.000 0.000 collections.py:104(values)
38 0.000 0.000 0.000 0.000 _weakrefset.py:52(_commit_removals)
1 0.000 0.000 0.000 0.000 pickle.py:827(Unpickler)
47 0.000 0.000 0.000 0.000 sre_compile.py:52(fixup)
97 0.000 0.000 0.000 0.000 {thread.get_ident}
1 0.000 0.000 0.000 0.000 commands.py:1(<module>)
1 0.000 0.000 0.000 0.000 conf.py:47(<module>)
1 0.000 0.000 0.000 0.000 models.py:1(<module>)
23 0.000 0.000 0.000 0.000 __init__.py:219(_releaseLock)
38 0.000 0.000 0.000 0.000 fields.py:222(add_to_fields)
4 0.000 0.000 0.000 0.000 {fcntl.fcntl}
3 0.000 0.000 0.000 0.000 os.py:510(getenv)
39 0.000 0.000 0.000 0.000 defer.py:73(is_generalised_generator)
1 0.000 0.000 0.000 0.000 posixsystem.py:101(Waker)
12 0.000 0.000 0.000 0.000 __init__.py:193(__eq__)
11 0.000 0.000 0.000 0.000 sockets.py:42(__new__)
10 0.000 0.000 0.000 0.000 log.py:108(local_property)
119 0.000 0.000 0.000 0.000 opcode.py:27(def_op)
38 0.000 0.000 0.000 0.000 _weakrefset.py:16(__init__)
18 0.000 0.000 0.000 0.000 session.py:557(__iter__)
1 0.000 0.000 0.003 0.003 __init__.py:151(TaskQueue)
20 0.000 0.000 0.000 0.000 fields.py:487(get_encoder)
1 0.000 0.000 0.000 0.000 session.py:533(Session)
1 0.000 0.000 0.000 0.000 subprocess.py:649(Popen)
2 0.000 0.000 0.000 0.000 __init__.py:345(parse_backend)
48 0.000 0.000 0.000 0.000 process.py:59(current_process)
36 0.000 0.000 0.000 0.000 py2py3.py:46(<lambda>)
2 0.000 0.000 0.002 0.001 connection.py:256(on_connect)
1 0.000 0.000 0.000 0.000 pickle.py:171(Pickler)
23 0.000 0.000 0.000 0.000 config.py:547(validate_pos_int)
1 0.000 0.000 0.000 0.000 __init__.py:265(TaskBackend)
2 0.000 0.000 0.000 0.000 __init__.py:46(get_connection_string)
58 0.000 0.000 0.000 0.000 httpurl.py:448(<genexpr>)
19 0.000 0.000 0.000 0.000 __init__.py:75(__init__)
40 0.000 0.000 0.000 0.000 {method 'rfind' of 'str' objects}
1 0.000 0.000 0.000 0.000 numbers.py:34(Complex)
1 0.000 0.000 0.000 0.000 os.py:35(_get_exports_list)
40 0.000 0.000 0.000 0.000 fields.py:328(_handle_extras)
24 0.000 0.000 0.000 0.000 {sys._getframe}
1 0.000 0.000 0.000 0.000 sc_mytest_dummyimport_dr.py:68(TestStdChild)
2 0.000 0.000 0.000 0.000 connection.py:204(__init__)
47 0.000 0.000 0.000 0.000 threading.py:63(_note)
12 0.000 0.000 0.000 0.000 session.py:275(get_delete_query)
37 0.000 0.000 0.000 0.000 abc.py:15(abstractmethod)
4 0.000 0.000 0.001 0.000 __init__.py:375(__getattr__)
27 0.000 0.000 0.000 0.000 decimal.py:3809(<genexpr>)
1 0.000 0.000 0.000 0.000 argparse.py:147(HelpFormatter)
1 0.000 0.000 0.000 0.000 rfc822.py:71(<module>)
1 0.000 0.000 0.000 0.000 session.py:799(Manager)
21 0.000 0.000 0.000 0.000 inspect.py:59(isclass)
1 0.000 0.000 0.000 0.000 process.py:86(Process)
12 0.000 0.000 0.000 0.000 client.py:1893(raise_first_error)
2 0.000 0.000 0.000 0.000 __init__.py:164(__init__)
2 0.000 0.000 0.000 0.000 parser.py:89(__init__)
1 0.000 0.000 0.000 0.000 struct.py:179(Structure)
1 0.000 0.000 0.000 0.000 wrappers.py:78(WsgiResponse)
12 0.000 0.000 0.000 0.000 session.py:93(_meta)
3 0.000 0.000 0.000 0.000 fields.py:603(register_with_related_model)
1 0.000 0.000 0.000 0.000 jsonrpc.py:134(JsonProxy)
3 0.000 0.000 0.000 0.000 client.py:1449(script_load)
1 0.000 0.000 0.000 0.000 async.py:89(BackendQuery)
3 0.000 0.000 0.000 0.000 related.py:44(load_relmodel)
6 0.000 0.000 0.000 0.000 hashlib.py:94(__get_openssl_constructor)
1 0.000 0.000 0.000 0.000 <string>:1(stdnet_version)
1 0.000 0.000 0.000 0.000 gzip.py:4(<module>)
1 0.000 0.000 0.000 0.000 udp.py:1(<module>)
20 0.000 0.000 0.000 0.000 __init__.py:182(_checkLevel)
1 0.000 0.000 0.000 0.000 argparse.py:1023(_SubParsersAction)
5 0.000 0.000 0.000 0.000 posixpath.py:59(isabs)
1 0.000 0.000 0.000 0.000 mixins.py:1(<module>)
3 0.000 0.000 0.000 0.000 models.py:123(__new__)
4 0.000 0.000 0.000 0.000 urllib.py:1245(quote)
7 0.000 0.000 0.000 0.000 dispatcher.py:40(__init__)
2 0.000 0.000 0.000 0.000 connection.py:404(make_connection)
50 0.000 0.000 0.000 0.000 {method 'remove' of 'set' objects}
1 0.000 0.000 0.000 0.000 __init__.py:265(_reset_cache)
12 0.000 0.000 0.000 0.000 {method 'difference' of 'set' objects}
4 0.000 0.000 0.000 0.000 {method 'union' of 'frozenset' objects}
20 0.000 0.000 0.000 0.000 __init__.py:578(__init__)
1 0.000 0.000 0.000 0.000 string.py:248(strip)
24 0.000 0.000 0.000 0.000 defer.py:352(__len__)
1 0.000 0.000 0.000 0.000 numbers.py:295(Integral)
1 0.000 0.000 0.000 0.000 runtime.py:30(Platform)
24 0.000 0.000 0.000 0.000 encoders.py:87(__init__)
19 0.000 0.000 0.000 0.000 base.py:382(extend_meta)
27 0.000 0.000 0.000 0.000 decimal.py:3816(<genexpr>)
14 0.000 0.000 0.000 0.000 py2py3.py:79(to_bytes)
24 0.000 0.000 0.000 0.000 inspect.py:209(iscode)
2 0.000 0.000 0.000 0.000 client.py:89(__init__)
2 0.000 0.000 0.000 0.000 saferef.py:45(BoundMethodWeakref)
1 0.000 0.000 0.000 0.000 httpurl.py:782(HttpParser)
2 0.000 0.000 0.000 0.000 connection.py:378(__init__)
4 0.000 0.000 0.000 0.000 random.py:91(__init__)
1 0.000 0.000 0.000 0.000 <string>:1(instance_session_result)
6 0.000 0.000 0.000 0.000 session.py:575(begin)
24 0.000 0.000 0.000 0.000 inspect.py:67(ismethod)
3 0.000 0.000 0.000 0.000 {_ctypes.dlopen}
1 0.000 0.000 0.000 0.000 states.py:1(<module>)
2 0.000 0.000 0.000 0.000 urlparse.py:389(parse_qsl)
1 0.000 0.000 0.000 0.000 <string>:1(orderinginfo)
1 0.000 0.000 0.001 0.001 decoder.py:17(_floatconstants)
11 0.000 0.000 0.000 0.000 client.py:49(string_keys_to_dict)
1 0.000 0.000 0.000 0.000 transport.py:140(SocketTransport)
6 0.000 0.000 0.000 0.000 async.py:80(on_result)
1 0.000 0.000 0.000 0.000 decimal.py:3763(Context)
3 0.000 0.000 0.000 0.000 UserDict.py:58(get)
1 0.000 0.000 0.000 0.000 collections.py:26(OrderedDict)
1 0.000 0.000 0.000 0.000 sc_mytest_dummyimport_dr.py:48(TestStdBase)
1 0.000 0.000 0.000 0.000 uuid.py:53(UUID)
1 0.000 0.000 0.000 0.000 <string>:1(DecimalTuple)
35 0.000 0.000 0.000 0.000 defer.py:228(__init__)
24 0.000 0.000 0.000 0.000 __init__.py:199(__hash__)
18 0.000 0.000 0.000 0.000 {method 'close' of 'generator' objects}
12 0.000 0.000 0.000 0.000 __init__.py:202(issame)
2 0.000 0.000 0.000 0.000 base.py:25(redis_client)
30 0.000 0.000 0.000 0.000 {method 'release' of 'thread.lock' objects}
36 0.000 0.000 0.000 0.000 {method '__subclasses__' of 'type' objects}
15 0.000 0.000 0.000 0.000 {thread.allocate_lock}
1 0.000 0.000 0.000 0.000 process.py:301(__init__)
2 0.000 0.000 0.010 0.005 __init__.py:366(_getdb)
110 0.000 0.000 0.000 0.000 {method 'isdigit' of 'str' objects}
11 0.000 0.000 0.000 0.000 wrappers.py:62(cached_property)
1 0.000 0.000 0.000 0.000 mapper.py:17(Router)
2 0.000 0.000 0.006 0.003 tempfile.py:270(mkstemp)
2 0.000 0.000 0.000 0.000 threading.py:259(__init__)
1 0.000 0.000 0.000 0.000 client.py:181(Redis)
3 0.000 0.000 0.000 0.000 __init__.py:359(_FuncPtr)
6 0.000 0.000 0.000 0.000 sockets.py:60(__new__)
3 0.000 0.000 0.000 0.000 {posix.close}
1 0.000 0.000 0.000 0.000 mapper.py:63(__init__)
8 0.000 0.000 0.000 0.000 {sorted}
1 0.000 0.000 0.000 0.000 __init__.py:1094(Logger)
11 0.000 0.000 0.000 0.000 {built-in method fromkeys}
1 0.000 0.000 0.000 0.000 random.py:72(Random)
2 0.000 0.000 0.000 0.000 urllib.py:1291(urlencode)
3 0.000 0.000 0.000 0.000 fields.py:620(_register_with_related_model)
1 0.000 0.000 0.000 0.000 handlers.py:235(Router)
29 0.000 0.000 0.000 0.000 {method 'endswith' of 'str' objects}
12 0.000 0.000 0.001 0.000 fields.py:249(__deepcopy__)
14 0.000 0.000 0.000 0.000 base.py:217(__str__)
30 0.000 0.000 0.000 0.000 pep.py:140(get_event_loop_policy)
1 0.000 0.000 0.000 0.000 monitor.py:210(Monitor)
1 0.000 0.000 0.000 0.000 <string>:1(session_data)
1 0.000 0.000 0.000 0.000 session.py:326(Transaction)
3 0.000 0.000 0.000 0.000 {posix.uname}
1 0.000 0.000 0.000 0.000 wrappers.py:335(WsgiRequest)
42 0.000 0.000 0.000 0.000 defer.py:452(set_timeout)
1 0.000 0.000 0.000 0.000 config.py:94(Config)
2 0.000 0.000 0.000 0.000 __init__.py:49(normalize_encoding)
1 0.000 0.000 0.000 0.000 struct.py:667(TS)
1 0.000 0.000 0.000 0.000 eventloop.py:295(EventLoop)
18 0.000 0.000 0.000 0.000 {method 'random' of '_random.Random' objects}
1 0.000 0.000 0.000 0.000 protocols.py:46(ProtocolConsumer)
5 0.000 0.000 0.000 0.000 transport.py:35(__new__)
2 0.000 0.000 0.000 0.000 copy_reg.py:95(_slotnames)
27 0.000 0.000 0.000 0.000 {method 'update' of 'set' objects}
1 0.000 0.000 0.000 0.000 sc_mytest_dummyimport_dr.py:61(TestStdParent)
Real 55.86
User 27.80
System 0.10
yes, you can send it to my email.
Great, I just sent it. I'll close this issue.
Hi Luca,
I did some testing with biggish transactions. When adding 25000 records, I get a MemoryError:
(I can't get a traceback somehow, it's running from inside C code through pythons C api on unix and that might complicate things)
10000 records per transaction goes okay, so I split it up in chunks.
Is there a way to check how much memory I got left, or even better: use a streaming interface that streams to the backend and doesn't have a memory limitation?
Kind regards, tw