search

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
https://www.nomadproject.io/
Other
14.83k stars 1.95k forks source link

[feature request] Allow dispatch_payload larger than 16KB #3365

Closed dukeland9 closed 6 years ago

dukeland9 commented 6 years ago

Though artifact provides a convenient way to get the binary and/or data needed for a task, sometimes it's still useful to use some local binary or data to start a remote job. For example, one may want his/her java application tested on one machine before pushing the jar to the nexus repository where the jar can be used as an artifact for serious deployments later. The dispatch_payload stanza may be a way to solve this, however, a jar is very likely to be (if not always) larger than 16KB, it can't be used as a dispatch_payload (as stated in https://www.nomadproject.io/docs/job-specification/parameterized.html, "The maximum size of a payload is 16 KiB").

So shall we have dispatch_payloads larger than 16KB? Or it would be more convenient if the artifact stanza can handle this behind the scene: if the artifact address is a remote address, it works the same as now. If the artifact address is local, it automatically dispatches the local artifact to the clients.

dadgar commented 6 years ago

@dukeland9 Hey unfortunately we won't be increasing the allowed payload size since the payload is stored in memory and communicated using Raft across servers, we do not want to be committing jars over a consensus protocol :)

The solution is to provide the artifact source as metadata when dispatching and then interpolating it in the artifact block. That way you can pull in artifacts!

github-actions[bot] commented 1 year ago

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.