Colossus bug fixes for 3.10.0 release

[mnaamani]

1. Multiple bug fixes
2. Refactored and improved pending objects service
3. Allow re-upload of already "accepted" objects via files api endpoint, in-case they are lost.
4. Fixed issue in batch extrinsic sender which was not returning correct failed calls.

closes:

• https://github.com/Joystream/joystream/issues/4991 - removed async lock, added some assertions
• https://github.com/Joystream/joystream/issues/5002 - fixed bug in files api that was injecting undefined key into idCache map
• https://github.com/Joystream/joystream/issues/5003
  • timeout issue sending extrinsic (which caused the next issue) - with use of latest improvements this should no longer happen.
  • failure to re-upload - support now added.
  • basic 404 file note found. Not a problem really, but maybe it shouldn't be logged as an error?

[mnaamani]

Thanks for investigating and fixing the issue. Wouldn't it be much simpler to store data object ID (req.params.id) at the start of the both GET/HEAD handlers, before the try block starts. This way the data object ID in the finally block would still be valid and we won't have to separately do the unpinning at multiple different places.

Yes that was one of the reasons I moved the data object id instantiation outside of the try block, but then realized that I had to get rid of the finally clause because it can be reached before the stream ends. So we would be unpinning too early, defeating the whole point.

It is easy to miss such cases when mixing async/await code with traditional node event and steam based paradigm in the same function.