averbis / averbis-python-api

Conveniently access the REST API of Averbis products using Python
Apache License 2.0
12 stars 4 forks source link

analyse_texts returns results only after all data has been processed #101

Closed reckart closed 2 years ago

reckart commented 2 years ago

Describe the bug analyse_texts returns results only after all data has been processed - but it should already start returning results as the data is being processed.

To Reproduce

from averbis import Client
import time

client = Client("MYPROFILE")
project = client.get_project("PROJECT")
pipeline = project.get_pipeline("PIPELINE").ensure_started()
start = time.time()

count = 0
for r in pipeline.analyse_texts([f"This is document {k}" for k in range(50)]):
    elapsed = time.time() - start
    print(f"{count=}, {elapsed=}")
    count += 1

The for loop is run only after all data has been processed:

count=0, elapsed=3.5972917079925537
count=1, elapsed=3.5973732471466064
count=2, elapsed=3.59741473197937
count=3, elapsed=3.5974578857421875
count=4, elapsed=3.5974929332733154
count=5, elapsed=3.59753155708313
count=6, elapsed=3.5975656509399414
count=7, elapsed=3.5976064205169678
count=8, elapsed=3.597644805908203
count=9, elapsed=3.5976781845092773
count=10, elapsed=3.597703695297241
...

Expected behavior The for loop should proceed as data is being processed - the elapsed time should be clearly different.

Please complete the following information: