REST API file size limitations

ufal / udpipe

UDPipe: Trainable pipeline for tokenizing, tagging, lemmatizing and parsing Universal Treebanks and other CoNLL-U files

Mozilla Public License 2.0

358 stars 75 forks source link

REST API file size limitations #68

Closed Henry-E closed 6 years ago

Henry-E commented 6 years ago

What are the limitations on file size for the udpipe REST API? I'm getting a "Request was too large." error

Are there any other caps on usage. For example as a workaround if the text file was sent in smaller chunks would that be ok.

[ I posted the same issue on the other udpipe repo, which is where google led me when searching for "request was too large". This repo seems much busier though so reposting here.]

Henry-E commented 6 years ago

Converting the text files to smaller chunks seems to be working well. If it does ever seem like I'm abusing the API please feel free to get in touch.

foxik commented 6 years ago

As per https://github.com/ufal/udpipe/blob/master/src/rest_server/udpipe_server.cpp#L125, the limit on the size of the POST request is 1MB. The reason for the limit is to make sure we do not run out of RAM on the machine.

If you have larger documents, you can either:

split it into smaller chunks (which is what you did)
or download the udpipe binary and models and run locally

Cheers, Milan

Henry-E commented 6 years ago

Thanks for providing such a useful service! It seemed like a much easier approach to use the rest api than getting all the binary and models running localling when I'm not going to be using it that extensively.