service hang due to large file with no new line char

[Tianhao-Gu]

_file_read_from_tail hangs service with a large file with no newline character \n in it.

https://github.com/kbase/staging_service/blob/master/staging_service/metadata.py#L29

reproduce issue:

• make test file: dd if=/dev/zero count=4000000 of=testfile (2GB test file) dd if=/dev/zero count=4000 of=testfile2 (2MB test file)
• call_file_read_from_tail from local console

[Tianhao-Gu]

potential solution

• [ ] set max read buffer. read from the end kilobytes and give up after a certain threshold.

• [x] set a max line length. The service will seek and return only the partial length of a line if the line is longer than max line length.

• [ ] set a timer to break. The service will return whatever it reads or a warning message when it takes the service longer than the time we set to read a line.

[Tianhao-Gu]

We decided to do:

• Fetch only the first 1024 chars and last 1024 chars for _file_read_from_head and _file_read_from_tail respectively