search

metwork-framework / mfdata

metwork/mfdata module
http://metwork-framework.org/
BSD 3-Clause "New" or "Revised" License
6 stars 4 forks source link

httpsend plugin - send find to multiple servers #479

Open matthieumarrast opened 8 months ago

matthieumarrast commented 8 months ago

The idea is to update the plugin httpsend to allow sending a file through HTTP to multiple urls. With below config example:

# HTTP full URL to use for uploading files
# You can use following placeholders in this url:
# - {ORIGINAL_BASENAME}
# - {ORIGINAL_DIRNAME}
# - {RANDOM_ID}
# - {STEP_COUNTER}
# - {ORIGINAL_UID}
# - strftime placeholders like %Y%m%d
# Example for feeding another mfdata with HTTP
# http://FIXME:9091/incoming/{ORIGINAL_BASENAME}
# (coma separated list of urls)
http_url=http://server1:9091/incoming/{ORIGINAL_BASENAME},http://server2:9091/incoming/{ORIGINAL_BASENAME},http://server3:9091/incoming/{ORIGINAL_BASENAME}
matthieumarrast commented 8 months ago

the sending must be done as simultaneously as possible

thebaptiste commented 8 months ago

Maybe helpful : https://www.slingacademy.com/article/python-aiohttp-sending-multiple-requests-concurrently/?utm_content=cmp-true