Serving gpt-2 model with Flask for conditional samples

[312shan]

Hi there , There are so many developers want to try gpt-2. I found a lot of people confuse and suffering for deploy gpt-2 as a web service, may be this commit will help them avoid this trouble and save more time focus on the model itself.

[ryanvolum]

@312shan thanks for creating this. I haven't looked much into the code, but I've tried running it on a GPU rig and am seeing my Tesla K80 GPUs consistently run out of memory, even with the smallest (124M) checkpoint:

2020-02-04 20:31:07.805409: I tensorflow/core/common_runtime/bfc_allocator.cc:818] total_region_allocated_bytes_: 275185664 memory_limit_: 275185664 available bytes: 0 curr_region_allocation_bytes_: 550371328
2020-02-04 20:31:07.809063: I tensorflow/core/common_runtime/bfc_allocator.cc:824] Stats:
Limit:                   275185664
InUse:                   275185664
MaxInUse:                275185664
NumAllocs:                     126
MaxAllocSize:              9437184

Any idea what might be causing this?

[312shan]

@ryanvolum Hello Ryan, my code was test on cpu only. Depends on you log, I think config the gpu session may help, change the line 40 :

self.sess = tf.Session()

with config below may help

config = tf.ConfigProto()  
config.gpu_options.per_process_gpu_memory_fraction = 0.4  
session = tf.Session(config=config) 

if this config work or you find out any other solution, please commented here , thanks.

[lucasmgomez]

When I run 'python3 src/flask_conditional_samples.py" I get this error; AttributeError: module 'tensorflow' has no attribute 'sort'

[312shan]

When I run 'python3 src/flask_conditional_samples.py" I get this error; AttributeError: module 'tensorflow' has no attribute 'sort'

[312shan]

When I run 'python3 src/flask_conditional_samples.py" I get this error; AttributeError: module 'tensorflow' has no attribute 'sort'

tensorflow==1.12.0