[semver:minor] Use files instead of CLI arguments

[roopakv]

Using data directly doesnt work when the continue config is large.

This accounts for that and uses files instead

[sulami]

Hey,

thanks so much for submitting this. We've actually noticed this problem ourselves and are looking at fixing it, but you got there faster.

This solves the problem with arguments too large for curl, but there is actually a second problem once the config hits a certain size. Exactly the same thing, but $RAW_CONFIG is too large for jq. I've been trying to make jq read it from a file as well, but that turns out to be somewhat difficult. If you have some extra time and would like to look into that as well, we'd appreciate it.

The specific problem I've encountered was that --slurpfile cannot read a raw string into a value, only the lines into an array, which is not what we want.

[roopakv]

@sulami I think this should fix it :) lmk what you think

[sulami]

The problem with this is that argfile expects the file to contain some kind of valid JSON value, but it actually contains YAML. We can't just enclose it in quotation marks, because there can be quotation marks inside the YAML which we would have to espace then, which was the original reason for piping the config through jq.

[roopakv]

ahh @sulami that makes sense. let me think a bit more.

in the meanwhile can I undo my jq part of the commit and we can get the curl fixed?

[sulami]

I actually took your code and played a bit with that over in https://github.com/CircleCI-Public/continuation-orb/pull/5, have a look if that looks good to you, or even better, give it a try if you can.

[roopakv]

one tiny nit :)

On Tue, Mar 9, 2021 at 11:35 AM Robin Schroer notifications@github.com wrote:

I actually took your code and played a bit with that over in #5 https://github.com/CircleCI-Public/continuation-orb/pull/5, have a look if that looks good to you, or even better, give it a try if you can.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CircleCI-Public/continuation-orb/pull/4#issuecomment-794352068, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFFSX7IT4MC6TU7K6HUSEDTCZ2ALANCNFSM4Y3DD24A .

-- Roopak Venkatakrishnan

https://roopakv.com

Ph:+1 (347)-766-7252

[roopakv]

Superseded by #5