64k limit problem for "tool fetch url"

eworm-de / routeros-scripts

a collection of scripts for MikroTik RouterOS

GNU General Public License v3.0

1.27k stars 285 forks source link

64k limit problem for "tool fetch url" #51

Closed katanatr closed 9 months ago

katanatr commented 9 months ago

If the size of the IP list in the URL for the fw-addr-lists list is larger than 64k, it cannot download all of it. It can download the first 64k size. Can you do a looped download in parts for this, @eworm-de?

For example, can you download the entire content in parts like this with a loop? :tool fetch url=$url http-header-field="Range: bytes=$start-$end" output=user defined

eworm-de commented 9 months ago

Yes, that is a limitation in RouterOS... Please make sure to open a support issue at Mikrotik and ask to address this. I did myself (SUP-132297) and answer was:

If there will be similar requests, we will consider increasing the output limit.

I am hesitating to do the loop thing... It makes the code fragile, and not all servers do support it. (In fact a lot of servers are configured not to support this, or do not support it if content is generated on request.)