despiegk
commented
5 years ago check fix #109
Closed despiegk closed 5 years ago
despiegk
commented
5 years ago check fix #109
ranatrk
commented
5 years ago Implement sandbox functions for CoreDns and OpenResty: https://github.com/threefoldtech/jumpscaleX/commit/23131349b90a2241697c205f2dce1954e9f94498
ahmadnazih
commented
5 years ago Traefik and Caddy builders are finished with the creation of their sandbox and flist
https://github.com/threefoldtech/jumpscaleX/commit/346c3d158a3cbda3302fbe6d602e36c6a3e44d0c
https://github.com/threefoldtech/jumpscaleX/commit/d28ea1c64ceb8ed18cc20e8a9e9c87f023c0bfee
andrewayoub
commented
5 years ago A document about how to create a new builder and an example added here
andrewayoub
commented
5 years ago A pull request created here to track the progress
ahmadnazih
commented
5 years ago Humhub done - Startup toml file for nginx needs to be created https://github.com/threefoldtech/jumpscaleX/commit/ae1214b15d907cc815eb9ca83471bcc63a1820f6
0xIslamTaha
commented
5 years ago caddy test status:
j.builder.web.traefik.build() passed
j.builder.web.traefik.sandbox() passed
j.builder.web.traefik.start() passed
j.builder.web.traefik.stop() bug
I created its flist using the tar file and here are the steps to verify its container
node = j.clients.zos.get('ac1f6b4575a8', data={'host':'10.102.234.243', 'password_':'*************'})
cont = node.client.container.create("https://hub.grid.tf/xtremx/caddy.flist")
client = node.client.container.client(24)
client.filesystem.list('/')
client.system('/sandbox/bin/caddy -h').get()output
In [30]: cont_client.system("/sandbox/bin/traefik -h").get()
Out[30]:
STATE: 0 SUCCESS
STDOUT:
(default "0")
--rancher.trace Display additional provider logs (if available). (default "false")
--rancher.watch Watch provider (default "true")
--respondingtimeouts Timeouts for incoming requests to the Traefik instance (default "true")
--respondingtimeouts.idletimeout IdleTimeout is the maximum amount duration an idle (keep-alive) connection will (default "3m0s")
remain idle before closing itself. Defaults to 180 seconds. If zero, no timeout
is set
--respondingtimeouts.readtimeout ReadTimeout is the maximum duration for reading the entire request, including (default "0s")
the body. If zero, no timeout is set
--respondingtimeouts.writetimeout WriteTimeout is the maximum duration before timing out writes of the response. (default "0s")
If zero, no timeout is set
--rest Enable Rest backend with default settings (default "true")
--rest.entrypoint EntryPoint (default "traefik")
--retry Enable retry sending request if network error (default "true")
--retry.attempts Number of attempts (default "0")
--rootcas Add cert file for self-signed certificate
--sendanonymoususage send periodically anonymous usage statistics (default "false")
--servicefabric Enable Service Fabric backend with default settings (default "false")
--servicefabric.apiversion Service Fabric API version
--servicefabric.appinsightsbatchsize Number of trace lines per batch, optional (default "0")
--servicefabric.appinsightsclientname The client name, Identifies the cloud instance
--servicefabric.appinsightsinterval The interval for sending data to Application Insights, optional (default "0s")
--servicefabric.appinsightskey Application Insights Instrumentation Key
--servicefabric.clustermanagementurl Service Fabric API endpoint
--servicefabric.constraints Filter services by constraint, matching with Traefik tags. (default "[]")
--servicefabric.debugloggeneratedtemplate Enable debug logging of generated configuration template. (default "false")
--servicefabric.filename Override default configuration template. For advanced users :)
--servicefabric.refreshseconds Polling interval (in seconds) (default "0s")
--servicefabric.templateversion Template version. (default "0")
--servicefabric.tls Enable TLS support (default "false")
--servicefabric.tls.ca TLS CA
--servicefabric.tls.caoptional TLS CA.Optional (default "false")
--servicefabric.tls.cert TLS cert
--servicefabric.tls.insecureskipverify TLS insecure skip verify (default "false")
--servicefabric.tls.key TLS key
--servicefabric.trace Display additional provider logs (if available). (default "false")
--servicefabric.watch Watch provider (default "false")
--tracing OpenTracing configuration (default "false")
--tracing.backend Selects the tracking backend ('jaeger','zipkin', 'datadog'). (default "jaeger")
--tracing.datadog Settings for DataDog (default "false")
--tracing.datadog.debug Enable DataDog debug. (default "false")
--tracing.datadog.globaltag Key:Value tag to be set on all the spans.
--tracing.datadog.localagenthostport Set datadog-agent's host:port that the reporter will used. Defaults to (default "localhost:8126")
localhost:8126
--tracing.jaeger Settings for jaeger (default "false")
--tracing.jaeger.localagenthostport set jaeger-agent's host:port that the reporter will used. (default "127.0.0.1:6831")
--tracing.jaeger.samplingparam set the sampling parameter. (default "1")
--tracing.jaeger.samplingserverurl set the sampling server url. (default "http://localhost:5778/sampling")
--tracing.jaeger.samplingtype set the sampling type. (default "const")
--tracing.servicename Set the name for this service (default "traefik")
--tracing.spannamelimit Set the maximum character limit for Span names (default 0 = no limit) (default "0")
--tracing.zipkin Settings for zipkin (default "false")
--tracing.zipkin.debug Enable Zipkin debug. (default "false")
--tracing.zipkin.httpendpoint HTTP Endpoint to report traces to. (default "http://localhost:9411/api/v1/spans")
--tracing.zipkin.id128bit Use ZipKin 128 bit root span IDs. (default "true")
--tracing.zipkin.samespan Use ZipKin SameSpan RPC style traces. (default "false")
--traefiklog Traefik log settings (default "false")
--traefiklog.filepath Traefik log file path. Stdout is used when omitted or empty
--traefiklog.format Traefik log format: json | common (default "common")
--traefiklogsfile (Deprecated) Traefik logs file. Stdout is used when omitted or empty
--web (Deprecated) Enable Web backend with default settings (default "false")
--web.address (Deprecated) Web administration port (default ":8080")
--web.certfile (Deprecated) SSL certificate
--web.keyfile (Deprecated) SSL certificate
--web.metrics (Deprecated) Enable a metrics exporter (default "false")
--web.metrics.datadog DataDog metrics exporter type (default "false")
--web.metrics.datadog.address DataDog's address (default "localhost:8125")
--web.metrics.datadog.pushinterval DataDog push interval (default "10s")
--web.metrics.influxdb InfluxDB metrics exporter type (default "false")
--web.metrics.influxdb.address InfluxDB address (default "localhost:8089")
--web.metrics.influxdb.database InfluxDB database used when protocol is http
--web.metrics.influxdb.protocol InfluxDB address protocol (udp or http) (default "udp")
--web.metrics.influxdb.pushinterval InfluxDB push interval (default "10s")
--web.metrics.influxdb.retentionpolicy InfluxDB retention policy used when protocol is http
--web.metrics.prometheus Prometheus metrics exporter type (default "false")
--web.metrics.prometheus.buckets Buckets for latency metrics (default "[0.1 0.3 1.2 5]")
--web.metrics.prometheus.entrypoint EntryPoint (default "traefik")
--web.metrics.statsd StatsD metrics exporter type (default "false")
--web.metrics.statsd.address StatsD address (default "localhost:8125")
--web.metrics.statsd.pushinterval StatsD push interval (default "10s")
--web.path (Deprecated) Root path for dashboard and API
--web.readonly (Deprecated) Enable read only API (default "false")
--web.statistics (Deprecated) Enable more detailed statistics (default "false")
--web.statistics.recenterrors Number of recent errors logged (default "10")
--zookeeper Enable Zookeeper backend with default settings (default "false")
--zookeeper.constraints Filter services by constraint, matching with Traefik tags. (default "[]")
--zookeeper.debugloggeneratedtemplate Enable debug logging of generated configuration template. (default "false")
--zookeeper.endpoint Comma separated server endpoints (default "127.0.0.1:2181")
--zookeeper.filename Override default configuration template. For advanced users :)
--zookeeper.password KV Password
--zookeeper.prefix Prefix used for KV store (default "traefik")
--zookeeper.templateversion Template version. (default "0")
--zookeeper.tls Enable TLS support (default "false")
--zookeeper.tls.ca TLS CA
--zookeeper.tls.caoptional TLS CA.Optional (default "false")
--zookeeper.tls.cert TLS cert
--zookeeper.tls.insecureskipverify TLS insecure skip verify (default "false")
--zookeeper.tls.key TLS key
--zookeeper.trace Display additional provider logs (if available). (default "false")
--zookeeper.username KV Username
--zookeeper.watch Watch provider (default "true")
-h, --help Print Help (this message) and exit
STDERR:
DATA:
j.builder.web.caddy.build() - pass j.builder.web.caddy.sandbox() - pass j.builder.web.caddy.start() - parsing error j.builder.web.caddy.stop() - bug
cont = node.client.container.create("https://hub.grid.tf/xtremx/traefik.flist")
cont_client = node.client.container.client(cont.get())
cont_client.filesystem.list('/')
cont_client.system('/sandbox/bin/caddy -h').get()Result
In [21]: data
Out[21]:
STATE: 1002 ERROR
STDOUT:
STDERR:
Usage of sandbox/bin/caddy:
-agree
Agree to the CA's Subscriber Agreement
-ca string
URL to certificate authority's ACME server directory (default "https://acme-v02.api.letsencrypt.org/directory")
-caddyfile-to-json
From Caddyfile stdin to JSON stdout
-catimeout duration
Default ACME CA HTTP timeout (default 30s)
-conf string
Caddyfile to load (default "Caddyfile")
-cpu string
CPU cap (default "100%")
-default-sni string
If a ClientHello ServerName is empty, use this ServerName to choose a TLS certificate
-disable-http-challenge
Disable the ACME HTTP challenge
-disable-tls-alpn-challenge
Disable the ACME TLS-ALPN challenge
-disabled-metrics string
Comma-separated list of telemetry metrics to disable
-email string
Default ACME CA account email address
-env string
Path to file with environment variables to load in KEY=VALUE format
-grace duration
Maximum duration of graceful shutdown (default 5s)
-host string
Default host
-http-port string
Default port to use for HTTP (default "80")
-http2
Use HTTP/2 (default true)
-https-port string
Default port to use for HTTPS (default "443")
-json-to-caddyfile
From JSON stdin to Caddyfile stdout
-log string
Process log file
-log-roll-compress
Gzip-compress rolled process log files (default true)
-log-roll-mb int
Roll process log when it reaches this many megabytes (0 to disable rolling) (default 100)
-pidfile string
Path to write pid file
-plugins
List installed plugins
-port string
Default port (default "2015")
-quic
Use experimental QUIC
-quiet
Quiet mode (no initialization output)
-revoke string
Hostname for which to revoke the certificate
-root string
Root path of default site (default ".")
-type string
Type of server to run (default "http")
-validate
Parse the Caddyfile but do not start the server
-version
Show version
DATA:
Dinaamagdy
commented
5 years ago etcd test status:
j.builder.db.etcd.build() https://github.com/threefoldtech/jumpscaleX/issues/183
j.builder.db.etcd.sandbox() passed
j.builder.db.etcd.start() bug
j.builder.db.etcd.stop() bug
I created its flist using the tar file and here are the steps to verify its container
node = j.clients.zos.get('ac1f6b4575a8', data={'host':'10.102.234.243', 'password_':'*************'})
cont = node.container.create(root_url="https://hub.grid.tf/dina_magdy/etcd.flist", nics=[{'type':"default",'name':"def"}], port={2225:22},storage="zdb://hub.grid.tf:9900")
client = node.client.container.client(24)
client.filesystem.list('/')
client.system('/sandbox/bin/etcd -h').get() output
Out[19]:
STATE: 0 SUCCESS
STDOUT:
Member:
--name 'default'
Human-readable name for this member.
--data-dir '${name}.etcd'
Path to the data directory.
--wal-dir ''
Path to the dedicated wal directory.
--snapshot-count '100000'
Number of committed transactions to trigger a snapshot to disk.
--heartbeat-interval '100'
Time (in milliseconds) of a heartbeat interval.
--election-timeout '1000'
Time (in milliseconds) for an election to timeout. See tuning documentation for details.
--initial-election-tick-advance 'true'
Whether to fast-forward initial election ticks on boot for faster election.
--listen-peer-urls 'http://localhost:2380'
List of URLs to listen on for peer traffic.
--listen-client-urls 'http://localhost:2379'
List of URLs to listen on for client traffic.
--max-snapshots '5'
Maximum number of snapshot files to retain (0 is unlimited).
--max-wals '5'
Maximum number of wal files to retain (0 is unlimited).
--quota-backend-bytes '0'
Raise alarms when backend size exceeds the given quota (0 defaults to low space quota).
--backend-batch-interval ''
BackendBatchInterval is the maximum time before commit the backend transaction.
--backend-batch-limit '0'
BackendBatchLimit is the maximum operations before commit the backend transaction.
--max-txn-ops '128'
Maximum number of operations permitted in a transaction.
--max-request-bytes '1572864'
Maximum client request size in bytes the server will accept.
--grpc-keepalive-min-time '5s'
Minimum duration interval that a client should wait before pinging server.
--grpc-keepalive-interval '2h'
Frequency duration of server-to-client ping to check if a connection is alive (0 to disable).
--grpc-keepalive-timeout '20s'
Additional duration of wait before closing a non-responsive connection (0 to disable).
Clustering:
--initial-advertise-peer-urls 'http://localhost:2380'
List of this member's peer URLs to advertise to the rest of the cluster.
--initial-cluster 'default=http://localhost:2380'
Initial cluster configuration for bootstrapping.
--initial-cluster-state 'new'
Initial cluster state ('new' or 'existing').
--initial-cluster-token 'etcd-cluster'
Initial cluster token for the etcd cluster during bootstrap.
Specifying this can protect you from unintended cross-cluster interaction when running multiple clusters.
--advertise-client-urls 'http://localhost:2379'
List of this member's client URLs to advertise to the public.
The client URLs advertised should be accessible to machines that talk to etcd cluster. etcd client libraries parse these URLs to connect to the cluster.
--discovery ''
Discovery URL used to bootstrap the cluster.
--discovery-fallback 'proxy'
Expected behavior ('exit' or 'proxy') when discovery services fails.
"proxy" supports v2 API only.
--discovery-proxy ''
HTTP proxy to use for traffic to discovery service.
--discovery-srv ''
DNS srv domain used to bootstrap the cluster.
--discovery-srv-name ''
Suffix to the dns srv name queried when bootstrapping.
--strict-reconfig-check 'true'
Reject reconfiguration requests that would cause quorum loss.
--pre-vote 'false'
Enable to run an additional Raft election phase.
--auto-compaction-retention '0'
Auto compaction retention length. 0 means disable auto compaction.
--auto-compaction-mode 'periodic'
Interpret 'auto-compaction-retention' one of: periodic|revision. 'periodic' for duration based retention, defaulting to hours if no time unit is provided (e.g. '5m'). 'revision' for revision number based retention.
--enable-v2 'true'
Accept etcd V2 client requests.
Security:
--cert-file ''
Path to the client server TLS cert file.
--key-file ''
Path to the client server TLS key file.
--client-cert-auth 'false'
Enable client cert authentication.
--client-crl-file ''
Path to the client certificate revocation list file.
--trusted-ca-file ''
Path to the client server TLS trusted CA cert file.
--auto-tls 'false'
Client TLS using generated certificates.
--peer-cert-file ''
Path to the peer server TLS cert file.
--peer-key-file ''
Path to the peer server TLS key file.
--peer-client-cert-auth 'false'
Enable peer client cert authentication.
--peer-trusted-ca-file ''
Path to the peer server TLS trusted CA file.
--peer-cert-allowed-cn ''
Required CN for client certs connecting to the peer endpoint.
--peer-auto-tls 'false'
Peer TLS using self-generated certificates if --peer-key-file and --peer-cert-file are not provided.
--peer-crl-file ''
Path to the peer certificate revocation list file.
--cipher-suites ''
Comma-separated list of supported TLS cipher suites between client/server and peers (empty will be auto-populated by Go).
--cors '*'
Comma-separated whitelist of origins for CORS, or cross-origin resource sharing, (empty or * means allow all).
--host-whitelist '*'
Acceptable hostnames from HTTP client requests, if server is not secure (empty or * means allow all).
Auth:
--auth-token 'simple'
Specify a v3 authentication token type and its options ('simple' or 'jwt').
--bcrypt-cost 10
Specify the cost / strength of the bcrypt algorithm for hashing auth passwords. Valid values are between 4 and 31.
Profiling and Monitoring:
--enable-pprof 'false'
Enable runtime profiling data via HTTP server. Address is at client URL + "/debug/pprof/"
--metrics 'basic'
Set level of detail for exported metrics, specify 'extensive' to include histogram metrics.
--listen-metrics-urls ''
List of URLs to listen on for the metrics and health endpoints.
Logging:
--logger 'capnslog'
Specify 'zap' for structured logging or 'capnslog'.
--log-outputs 'default'
Specify 'stdout' or 'stderr' to skip journald logging even when running under systemd, or list of comma separated output targets.
--debug 'false'
Enable debug-level logging for etcd.
Logging (to be deprecated in v3.5):
--log-package-levels ''
Specify a particular log level for each etcd package (eg: 'etcdmain=CRITICAL,etcdserver=DEBUG').
v2 Proxy (to be deprecated in v4):
--proxy 'off'
Proxy mode setting ('off', 'readonly' or 'on').
--proxy-failure-wait 5000
Time (in milliseconds) an endpoint will be held in a failed state.
--proxy-refresh-interval 30000
Time (in milliseconds) of the endpoints refresh interval.
--proxy-dial-timeout 1000
Time (in milliseconds) for a dial to timeout.
--proxy-write-timeout 5000
Time (in milliseconds) for a write to timeout.
--proxy-read-timeout 0
Time (in milliseconds) for a read to timeout.
Experimental feature:
--experimental-initial-corrupt-check 'false'
Enable to check data corruption before serving any client/peer traffic.
--experimental-corrupt-check-time '0s'
Duration of time between cluster corruption check passes.
--experimental-enable-v2v3 ''
Serve v2 requests through the v3 backend under a given prefix.
--experimental-backend-bbolt-freelist-type
ExperimentalBackendFreelistType specifies the type of freelist that boltdb backend uses(array and map are supported types).
Unsafe feature:
--force-new-cluster 'false'
Force to create a new one-member cluster.
CAUTIOUS with unsafe flag! It may break the guarantees given by the consensus protocol!
STDERR:
Usage:
etcd [flags]
Start an etcd server.
etcd --version
Show the version of etcd.
etcd -h | --help
Show the help information about etcd.
etcd --config-file
Path to the server configuration file.
etcd gateway
Run the stateless pass-through etcd TCP connection forwarding proxy.
etcd grpc-proxy
Run the stateless etcd v3 gRPC L7 reverse proxy.
DATA:verifying openresty build:pass sandbox: pass flist verifying: I creates flist from tar.gz generated from sandbox and create container using it but it doesn't work correctly https://github.com/threefoldtech/jumpscaleX/issues/186
Dina-Abd-Elrahman
commented
5 years ago verifying coredns
build:pass sandbox: bug
despiegk
commented
5 years ago have created a checklist see main content
rkhamis
commented
5 years ago Closed in favor https://github.com/threefoldtech/jumpscaleX/issues/299
instructions in https://github.com/threefoldtech/jumpscaleX/blob/development/docs/flist/auto_generation.md are very wrong, DO NOT USE
we need to use language = DSL for specifiying how to sandbox/build/...
follow instructions do how to do builders https://github.com/threefoldtech/jumpscaleX/blob/development_types/docs/Internals/Builders.md
Current open issues:
branch
use branch development_types
move docs to docs/builder