Closed dzaman closed 1 year ago
This significantly modifies almost the entire codebase. I'm not a fan of directly inlining asyncToGenerator
; I'd sooner just drop Node 6 support. I'm not extremely familiar with the details of this codebase, so naturally I'm hesitant to accept such drastic changes.
This PR is conflicted and outdated; closing.
In #23, @dzek69 mentioned that klaw stops on error, and I encountered the same issue.
This behavior exists because the implementation of
readable._read(size)
necessitates a call toreadable.push()
before it will be called again, and it's possible for klaw to return from _read without calling push when it encounters an error. When this occurs,_read
is never called again.I solved this problem by more closely aligning the
_read
implementation with the suggestion in the docs:With this PR,
_read
will continue to iterate over paths untilpush
returns false or it runs out of paths. When an error is encountered before we callpush
, it will continue processing and make subsequent calls to push if there are any eligible paths, so errors no longer terminate the crawl prematurely.Some notes: