search

neilbeveridge / zuul-netty

Production-ready culmination of this project now available at https://github.com/HotelsDotCom/styx
Apache License 2.0
34 stars 11 forks source link

[Question] Proxy for large files #1

Open aivans opened 9 years ago

aivans commented 9 years ago

Hi guys,

I know there was no commit for about a year.

Did you test large files upload by chance? Is this work continued now as Zuul 2.0?

thanks

neilbeveridge commented 9 years ago

Hi Adrian

This project is really just a poc for the Netflix Cloud Prize. My employer plans to open source the productionized code that we run in the near future. It has numerous enhancements over what you see in this repo to make it stand up to real world traffic.

Cheers Neil

On 14 Mar 2015, at 19:45, Adrian Ivan notifications@github.com wrote:

Hi guys,

I know there was no commit for about a year.

Did you test large files upload by chance? Is this work continued now as Zuul 2.0?

thanks

— Reply to this email directly or view it on GitHub.

aivans commented 9 years ago

Hi Neil,

Thank you for the reply. Sounds very good.

I am evaluating the Netflix stack for my employer and I embarked on a journey with Spring Cloud. So far Zuul is not really what I read in the "marketing" slides or maybe I have too high expectations. Asynch end to end in Zuul makes complete sense.

I dare to ask whether you have a roadmap for going open source with your work?

Thanks, Adrian

neilbeveridge commented 9 years ago

Hi Adrian

Zuul is great as an API proxy and perhaps where you're happy to throw resources at the problem to get around the synchronous constraints. For us then we wanted to be Async from the beginning as the vast majority of time in the proxy is spent waiting on the network and we need to carefully control outliers.

Our internal project (Styx) and Zuul 2 are built on Netty 4 and so may be similar in their performance characteristics. I've ensured that our focus is on plugability, tooling and hardening however as we use the proxy in front of our website and experience a lot of interesting conditions ;)

What is it in particular that you are looking for from the proxy? I hope that we'll be releasing the project in q2.

Thanks Neil

On 14 Mar 2015, at 22:09, Adrian Ivan notifications@github.com wrote:

Hi Neil,

Thank you for the reply. Sounds very good.

I am evaluating the Netflix stack for my employer and I embarked on a journey with Spring Cloud. So far Zuul is not really what I read in the "marketing" slides or maybe I have too high expectations. Asynch end to end in Zuul makes complete sense.

I dare to ask whether you have a roadmap for going open source with your work?

Thanks, Adrian

— Reply to this email directly or view it on GitHub.

aivans commented 9 years ago

Hi Neil,

I am looking in the broader context of an lightweight API Gateway. Our current needs would be:

From my understanding, Zuul capabilities (Zuul and spring cloud Netflix) cover:

Custom work would help for

Upload and download. This is crucial for us as customers would upload files, we would apply some changes and customers would retrieve the changed files. By default, Zuul is about 6-7 times slower than Apache Httpd by testing on localhost with minimum disk usage, basically pure overhead. On top of it, at least with Spring Cloud everything is loaded in memory, so not feasible for production. Dave Syer from Spring is trying to improve that: https://github.com/spring-cloud/spring-cloud-netflix/issues/254 We got to the point where you can send requests with curl that do not load the entire request in memory, but cannot get it working from the browser (we would like users to upload their files via a browser). Being slower than Apache Httpd for this scenario will not be such a huge issue in production (user network latency will attenuate the difference) but the memory is an issue.

About lightweight API Gateway idea. I am aware that Zuul or other proxy would not fit all scenarios and it may be possible that multiple components will complete the API Gateway, for example Transformation and Aggregation may be be another tier. Same applies for Security.

It would be interesting to know your opinion on the topic.

Thanks, Adrian

maximkir commented 7 years ago

Hi Neil, Any updates regarding the zuul-netty opensource project?