search

pulp / pulpcore

Pulp 3 pulpcore package https://pypi.org/project/pulpcore/
GNU General Public License v2.0
301 stars 116 forks source link

As a user I can set the max speed when synchronizing a repo #1893

Open pulpbot opened 2 years ago

pulpbot commented 2 years ago

Author: mped (mped)

Redmine Issue: 6700, https://pulp.plan.io/issues/6700

hi

In Pulp2 it is possible to set the max-speed when synchronizing a repo would it be possible to do this for Pulp3 too please?

Thanks

Matt

pulpbot commented 2 years ago

From: @dralley (dalley) Date: 2020-12-16T15:01:59Z

It is likely possible to use cgroups to accomplish this

bmbouter commented 2 years ago

We do not have a max speed in bits / sec, but we do have rate limiting in requests / sec. See the rate_limit attribute in the remote API docs. If having a bytes / second rate limiting would be useful please comment or reopen this ticket.

dralley commented 2 years ago

Awaiting more info for RFE https://bugzilla.redhat.com/show_bug.cgi?id=2035224

bmbouter commented 2 years ago

We can wait to hear more. In the places I've heard about this comparison to Pulp2 is that the bandwidth restricted is desired to be "from Pulp" not necessarily "Pulp syncing from repo X". The latter we could provide, but the former I don't think as a clustered application we're in a position to offer clustered byte-based rate limiting.

mdellweg commented 2 years ago

Firewall based traffic shaping comes to my mind (problem solved elsewhere...).

dralley commented 2 years ago

Yeah I'm still not seeing how doing this at the application layer is the appropriate solution. The more they explain about their requirements the more it seems that a network-level or firewall-level solution is what they ought to be using. But I guess they don't like that solution.

bmbouter commented 2 years ago

They could also use tc on a single machine with no extra hardware, or maybe cgroups.

ipanova commented 2 years ago

cgroups FTW

dralley commented 2 years ago

Agreed.

dralley commented 1 year ago

Reopening this to avoid fighting with the Bugzilla automation while a discussion is ongoing

bmbouter commented 1 year ago

If this is to be accomplished, the issue would need to be filed against aiohttp. I don't think Pulp can support this until aiohttp supports it.

ipanova commented 11 months ago

A user shared an example that limits public net to 250mbps, and leaves private at 10gbps

tc qdisc add dev eth0 root handle 1: htb default 20
modprobe ifb numifbs=1
ip link set dev ifb0 up
tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: protocol ip prio 2 u32 match ip src 0.0.0.0/0 action mirred egress redirect dev ifb0
tc qdisc add dev ifb0 root handle 2: htb
tc class add dev eth0 parent 1: classid 1:1 htb rate 10240000kbit
tc class add dev ifb0 parent 2: classid 2:1 htb rate 256000kbit
tc filter add dev ifb0 protocol ip parent 2: prio 10 u32 match ip src 0.0.0.0/0 flowid 2:1
tc filter add dev eth0 parent ffff: protocol ip prio 1 u32 classid 1:1 match ip src 192.168.0.0/16 flowid 1:1
tc filter add dev eth0 parent ffff: protocol ip prio 1 u32 classid 1:1 match ip src 172.16.0.0/16 flowid 1:1
tc filter add dev eth0 parent ffff: protocol ip prio 1 u32 classid 1:1 match ip src 10.0.0.0/8 flowid 1:1
ggainey commented 5 months ago

https://bugzilla.redhat.com/show_bug.cgi?id=2035224