search

Inedo / pgscan

Dependency scanner for ProGet.
MIT License
11 stars 5 forks source link

pgscan: The operation has timed out #19

Closed crotondo-dap closed 1 year ago

crotondo-dap commented 1 year ago

Hi,

we have updated our pgscan version from 1.2.0 to 1.3.1. We did this mainly to use the functionality to send multiple dependents in one request (we did not update to the new ProGet 2022 yet). Unfortunately, our pgscan calls are now running into timeouts. We have projects which have up to 1500 npm dependencies. The default timeout of 100 seconds is not enough to handle this amount of dependents. Before the update, pgscan took 2-3 minutes to handle this amount of dependents.

Maybe you have already encountered this limitation?

I guess there could be two possible solutions:

  1. Increase the timeout before the request gets send (but who knows what value would please anybody?)
  2. Use appsettings where the request can be modified individually.

Please let me know what you think about this issue.

whatatripp commented 1 year ago

Hello,

Are the time-outs happening on the pgscan side, or is ProGet timing out (and then causing pgscan to fail)?

100 seconds does seem like a really long time, but there may be some performance issues with really large data sets due to some missing indexes.

With ProGet 2022, this should not be an issue as these indexes were added.... but especially if you witch to the pgscan identify command, which uses the new API. That works with signficantly larger datasets no problem.

Cheers, Alana

crotondo-dap commented 1 year ago

Hi Alana,

the timeouts are on the pgscan side. Pgscan uses a WebRequest which has a default timeout of 100 seconds.

I tried the pgscan publish command on our test setup of ProGet 2022 and it is responding a lot faster than our live ProGet 6.0.11.

Unfortunately, we did not have time to have a detailed look at the new ProGet 2022 and therefore, we can not update ProGet for now. Should we just continue to use an older version of pgscan in combination with the older ProGet version?

whatatripp commented 1 year ago

Thanks; the alternative is to increase the timeout in the pgscan code, but like you said, we don't know what it should be... In 2022, it will never never be that slow.

The other option is to add indexes to your PackageDependants table in production ProGet 6.0, but then you'll want to delete them before upgrading to ProGet 2022.

crotondo-dap commented 1 year ago

Thank you for your quick feedback. We will try to update to ProGet 2022 as soon as possible. Until then we will just use the older version of pgscan.

crotondo-dap commented 1 year ago

We upgradet to ProGet 2022.8 and are no longer encountering this problem.