Islandora / Alpaca

Event driven middleware based on Apache Camel for Islandora
http://islandora.ca
MIT License
7 stars 20 forks source link

Adding disableStreamCache=true on every http call #66

Closed dannylamb closed 2 years ago

dannylamb commented 4 years ago

GitHub Issue: Part of the fix for https://github.com/Islandora/documentation/issues/1278

What does this Pull Request do?

Disables stream caching so payloads are streamed around and not read into memory.

How should this be tested?

See https://github.com/Islandora/documentation/issues/1278

Interested parties

@Islandora/8-x-committers @kayakr @antbrown

codecov[bot] commented 4 years ago

Codecov Report

Merging #66 into dev will decrease coverage by 4.86%. The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##                dev      #66      +/-   ##
============================================
- Coverage     89.76%   84.89%   -4.87%     
- Complexity       73       96      +23     
============================================
  Files            10       16       +6     
  Lines           293      331      +38     
  Branches          1        1              
============================================
+ Hits            263      281      +18     
- Misses           29       49      +20     
  Partials          1        1              
Impacted Files Coverage Δ Complexity Δ
...paca/connector/derivative/DerivativeConnector.java 100.00% <100.00%> (ø) 3.00 <0.00> (ø)
...slandora/alpaca/indexing/fcrepo/FcrepoIndexer.java 90.47% <100.00%> (-1.98%) 6.00 <0.00> (ø)
...lpaca/indexing/triplestore/TriplestoreIndexer.java 96.55% <100.00%> (ø) 9.00 <0.00> (ø)
...aca/http/client/StaticTokenRequestInterceptor.java 73.33% <0.00%> (ø) 6.00% <0.00%> (ø%)
...a/ca/islandora/alpaca/support/event/AS2Object.java
...va/ca/islandora/alpaca/support/event/AS2Actor.java
...va/ca/islandora/alpaca/support/event/AS2Event.java
.../islandora/alpaca/support/event/AS2Attachment.java
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 8c6bc55...0563261. Read the comment docs.

whikloj commented 4 years ago

@dannylamb I'm sorry I didn't catch you sooner, but couldn't we set this as an option in the blueprint.XML so you don't have to do this?

https://github.com/Islandora/Alpaca/blob/dev/islandora-indexing-fcrepo/src/main/resources/OSGI-INF/blueprint/blueprint.xml#L26

dannylamb commented 4 years ago

@whikloj Depends on the type of parameter IIRC. Some can be set on the http component and some are done on a per endpoint basis. I'll look that up because if we can do it once in the blueprint, we should.

dannylamb commented 4 years ago

Bit of a wormhole but.... I think we already have the groundwork laid for this. I can make an HttpClientConfigurer and add it as a bean here. Then it should be exported out and affect all http components used in Karaf.

Also... sneaky claw reference!!!

antbrown commented 4 years ago

This works for us, I haven't noticed any regressions after upgrading, and the ability to pass huge files around the place is really cool.

Thank you!

whikloj commented 4 years ago

checkstyle fail for long lines

seth-shaw-unlv commented 3 years ago

@dannylamb, do you want to revisit this so we can pull it over the finish line?

simonhm commented 2 years ago

Hi all, Any updates for this patch? We're having the same problem in loading large video file in Islandora 8 (version 2.2.1) without this patch. It's ok to load and generate small video files (<100 MB), but it will retry the generating service file process several times for large video files and get frozen. The service files are not generated for them ofc. Simon.

seth-shaw-unlv commented 2 years ago

@simonhm , I believe this patch was superseded and incorporated into https://github.com/Islandora/Alpaca/commit/7bf983de06b70a102514732323f8cb88680a1a6d (available as part of the latest release).

@whikloj or @dannylamb , can either of you confirm and close this PR?

whikloj commented 2 years ago

Yes this was added to the new one jar, you can find it here.

simonhm commented 2 years ago

We've been testing Islandora 8 using the Ansible Playbook. The Alpaca version on our testing sites is 1.x that was installed in the initial set up. We've been updating PHP modules and Drupal but everything else in the stack is what was initially installed. I think it's better if the Alpaca version in Ansible Playbook or Docker is 2.x instead of 1.x. Thanks! Simon.

whikloj commented 2 years ago

Hi @simonhm, The islandora-playbook now installs the new Alpaca all-in-one jar, however before trying to update you might want to wait for https://github.com/Islandora-Devops/islandora-playbook/pull/210