search

kiwix / libkiwix

Common code base for all Kiwix ports
https://download.kiwix.org/release/libkiwix/
GNU General Public License v3.0
120 stars 56 forks source link

Simplify our HTTP dependencies #347

Open kelson42 opened 4 years ago

kelson42 commented 4 years ago

We use currently three HTTP libraries in Kiwix library:

These is probably 2 too much.

I would like we consider as well https://nghttp2.org

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

kelson42 commented 4 years ago

@mgautierfr @veloman-yunkan As we are improving the CI/CD for Windows it appears that we would benefit to make a decision here. On my side, it looks like httplib is quite good, but I wonder a bit if the server part is really good at handling a high throughput traffic (pooling strategy, etc...) like we have at https://library.kiwix.org/. I don't think libcurl has anything to manager a HTTP server.

kelson42 commented 4 years ago

On the server side, https://nghttp2.org has always made great first impression to me. Maybe we have here two problems: a lib for the client side and a lib for the server side.

mgautierfr commented 4 years ago

On my side, it looks like httplib is quite good, but I wonder a bit if the server part is really good at handling a high throughput traffic (pooling strategy, etc...)

It seems httplib has everything we need : https://github.com/yhirose/cpp-httplib#default-thread-pool-support and https://github.com/yhirose/cpp-httplib#override-the-default-thread-pool-with-yours

On the server side, nghttp2.org has always made great first impression to me.

It doesn't to me :) The C api seems really low level and the c++ api is experimental and depends on boost.

kelson42 commented 4 years ago

Pinging @legoktm as he might have a feedback about this ticket.

legoktm commented 4 years ago

I have a preference to any library that is already relatively well maintained in Debian :)

Both libcurl and libmicrohttpd are popular and used by other projects and have good track records behind them.

I didn't really give httplib a good look earlier since it was just being used in tests, I don't really know how widely it's used but I expect HTTP servers to be robust and have good security properties because of how often they're attacked. And then any HTTP client should properly use the system certificate store and crypto policies, etc. My quick search through the code indicates that httplib doesn't support stuff like HSTS, which is disappointing (but might not be a concern?).

kelson42 commented 4 years ago

My quick search through the code indicates that httplib doesn't support stuff like HSTS, which is disappointing (but might not be a concern?).

This is a quite interesting point because to support SW based ZIM file, we will need to support HTTPS. Not sure what is the value of HTTPS with HSTS?!

mgautierfr commented 4 years ago

Maybe we have here two problems: a lib for the client side and a lib for the server side.

Probably yes. We use libcurl mainly to pilot the aria2c tools (and remote favicon listed in catalog). If we change this, it would be even better to have a websocket client. It would allow us to be notified by aria2c instead of polling it. (And we use a subprocess because we fail to link/compile with libaria2 on windows. I don't remember why but we may rework on that and simply the client http library).

I didn't really give httplib a good look earlier since it was just being used in tests, I don't really know how widely it's used but I expect HTTP servers to be robust and have good security properties because of how often they're attacked. And then any HTTP client should properly use the system certificate store and crypto policies, etc. My quick search through the code indicates that httplib doesn't support stuff like HSTS, which is disappointing (but might not be a concern?).

On the server side, I would tend to say it (https) is not our problem. If you want https, use a reverse-proxy to do it (it is pretty easy with nginx, httpd or candy). It will be more secured, integrated with the distribution (and made by people who fully understand what they are doing with the certificate).

kelson42 commented 3 years ago

Related to https://github.com/kiwix/kiwix-tools/issues/416

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

kelson42 commented 1 year ago

Related to https://github.com/kiwix/kiwix-desktop/issues/957#issuecomment-1717362443.

I suspect that libmicrohhtpd and libcurl might be the final choices.