search

Joystream / joystream

Joystream Monorepo
http://www.joystream.org
GNU General Public License v3.0
1.42k stars 115 forks source link

Storage Squid integration with Argus/Colossus #5001

Closed zeeshanakram3 closed 9 months ago

zeeshanakram3 commented 11 months ago

closes #4957

This PR:

zeeshanakram3 commented 10 months ago

I expected the supportedObjects to give proper number. I can get objects fine

@kdembler Addressed in 2bbf0af. You should be able to see the supported objects count now

kdembler commented 10 months ago

btw @zeeshanakram3 did you manage to resolve https://github.com/Joystream/storage-squid/issues/1? My squid crashed 1 day after starting

zeeshanakram3 commented 10 months ago

btw @zeeshanakram3 did you manage to resolve Joystream/storage-squid#1? My squid crashed 1 day after starting

I wasn't able to reproduce the error. I will try to test again locally against the mainnet

kdembler commented 10 months ago

I wasn't able to reproduce the error. I will try to test again locally against the mainnet

I'd suggest to do it on a server so that it indexes all the time. Locally (only syncing when my laptop was on) it didn't happen to me as well

kdembler commented 10 months ago

I'd suggest to do it on a server so that it indexes all the time. Locally (only syncing when my laptop was on) it didn't happen to me as well

After your comment I have dropped my squid, started fresh sync and it crashed again

kdembler commented 10 months ago

@zeeshanakram3 one more question: do you think it would be a lot of effort to include storage squid version in argus&colossus status endpoints? https://github.com/Joystream/joystream/issues/4732

zeeshanakram3 commented 10 months ago

@zeeshanakram3 one more question: do you think it would be a lot of effort to include storage squid version in argus&colossus status endpoints? #4732

I will implement it.

zeeshanakram3 commented 9 months ago
  • Bump storage node version to v3.11.0 and the storage cli package to v3.3.0

@mnaamani regarding the package versions, shouldn't we bump the major versions in both Argus & Colossus, since we have a breaking change in configuration options (changed --queryNodeEndpoint to --storageSquidEndpoint)?

mnaamani commented 9 months ago
  • Bump storage node version to v3.11.0 and the storage cli package to v3.3.0

@mnaamani regarding the package versions, shouldn't we bump the major versions in both Argus & Colossus, since we have a breaking change in configuration options (changed --queryNodeEndpoint to --storageSquidEndpoint)?

That might actually make more sense yes.

mnaamani commented 9 months ago

Should we also consider forking current master to keep the old version on a separate branch so we can slowly transition all operators just in-case we need to create some important patch while operators are still on older version?

zeeshanakram3 commented 9 months ago

Should we also consider forking current master to keep the old version on a separate branch so we can slowly transition all operators just in-case we need to create some important patch while operators are still on older version?

Yeah, makes sense

zeeshanakram3 commented 9 months ago

Failing integration tests should be fixed by https://github.com/Joystream/storage-squid/pull/13

kdembler commented 9 months ago

@zeeshanakram3 could you reply to this? https://github.com/Joystream/joystream/pull/5001#discussion_r1443286342 I think Argus (and Colossus) should still provide valid response for /status request even when the squid is down. Currently we're just forwarding the raw error to the user.

As a counterargument, we need that kind of handling for asset requests as well, not only for status and that currently isn't handled