Data Transfer V2 Integration, Rebased

[hannahhoward]

Goals

Update to go data transfer v2!

Implementation

This is a rebase and update for #718, now using go-data-transfer v2 on master, not quite yet tagged.

@rvagg several of these changes are yours.

Others are my own. None are really "new" code since #718

This is dependent on #755, and also should go in after #752.

Also, we shouldn't merge until we get data transfer tagged. I'd like to deploy this in our autoretrieve instance and also maybe sofia miner first.

[rvagg]

should the ffi have been updated with this too?

[codecov-commenter]

Codecov Report

Merging #757 (76a7087) into master (0fccfc9) will increase coverage by 0.51%. The diff coverage is 84.10%.

Additional details and impacted files

[](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project) ```diff @@ Coverage Diff @@ ## master #757 +/- ## ========================================== + Coverage 57.66% 58.17% +0.51% ========================================== Files 66 64 -2 Lines 5564 5373 -191 ========================================== - Hits 3208 3125 -83 + Misses 2013 1918 -95 + Partials 343 330 -13 ``` | [Impacted Files](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project) | Coverage Δ | | |---|---|---| | [retrievalmarket/events.go](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project#diff-cmV0cmlldmFsbWFya2V0L2V2ZW50cy5nbw==) | `0.00% <ø> (ø)` | | | [retrievalmarket/impl/clientstates/client\_fsm.go](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project#diff-cmV0cmlldmFsbWFya2V0L2ltcGwvY2xpZW50c3RhdGVzL2NsaWVudF9mc20uZ28=) | `66.93% <ø> (ø)` | | | [...etrievalmarket/impl/providerstates/provider\_fsm.go](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project#diff-cmV0cmlldmFsbWFya2V0L2ltcGwvcHJvdmlkZXJzdGF0ZXMvcHJvdmlkZXJfZnNtLmdv) | `47.83% <ø> (+39.01%)` | :arrow_up: | | [retrievalmarket/network/query\_stream.go](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project#diff-cmV0cmlldmFsbWFya2V0L25ldHdvcmsvcXVlcnlfc3RyZWFtLmdv) | `60.00% <ø> (ø)` | | | [shared/retrystream.go](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project#diff-c2hhcmVkL3JldHJ5c3RyZWFtLmdv) | `92.31% <ø> (ø)` | | | [shared/selectors.go](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project#diff-c2hhcmVkL3NlbGVjdG9ycy5nbw==) | `0.00% <0.00%> (ø)` | | | [storagemarket/impl/client\_environments.go](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project#diff-c3RvcmFnZW1hcmtldC9pbXBsL2NsaWVudF9lbnZpcm9ubWVudHMuZ28=) | `0.00% <ø> (ø)` | | | [storagemarket/impl/clientstates/client\_fsm.go](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project#diff-c3RvcmFnZW1hcmtldC9pbXBsL2NsaWVudHN0YXRlcy9jbGllbnRfZnNtLmdv) | `79.63% <ø> (ø)` | | | [storagemarket/impl/provider\_environments.go](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project#diff-c3RvcmFnZW1hcmtldC9pbXBsL3Byb3ZpZGVyX2Vudmlyb25tZW50cy5nbw==) | `10.38% <ø> (ø)` | | | [storagemarket/impl/providerstates/provider\_fsm.go](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project#diff-c3RvcmFnZW1hcmtldC9pbXBsL3Byb3ZpZGVyc3RhdGVzL3Byb3ZpZGVyX2ZzbS5nbw==) | `74.60% <ø> (ø)` | | | ... and [34 more](https://codecov.io/gh/filecoin-project/go-fil-markets/pull/757?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=filecoin-project) | |