search

apache / trafficserver

Apache Traffic Server™ is a fast, scalable and extensible HTTP/1.1 and HTTP/2 compliant caching proxy server.
https://trafficserver.apache.org/
Apache License 2.0
1.82k stars 806 forks source link

No Available PARENTS and ATS not adhering to parent configurations. #8914

Open amoghyermalkar123 opened 2 years ago

amoghyermalkar123 commented 2 years ago

Problem - 

[Jun 15 05:23:59.443] [ET_NET 11] WARNING: no available parents and the config proxy.config.http.no_dns_just_forward_to_parent, prevents origin lookups.

My Parent.config - 

dest_domain=. method=get parent="127.0.0.1:8000" scheme=http round_robin=false go_direct=false ignore_self_detect=true parent_is_proxy=false

My records.config - 

##############################################################################
# *NOTE*: All options covered in this file should be documented in the docs:
#
#    https://docs.trafficserver.apache.org/records.config
##############################################################################

##############################################################################
# Thread configurations. Docs:
#    https://docs.trafficserver.apache.org/records.config#thread-variables
##############################################################################
CONFIG proxy.config.exec_thread.autoconfig INT 1
CONFIG proxy.config.exec_thread.autoconfig.scale FLOAT 1.5
CONFIG proxy.config.exec_thread.limit INT 2
CONFIG proxy.config.accept_threads INT 1
CONFIG proxy.config.task_threads INT 2
CONFIG proxy.config.cache.threads_per_disk INT 8
CONFIG proxy.config.exec_thread.affinity INT 1

##############################################################################
# Specify server addresses and ports to bind for HTTP and HTTPS. Docs:
#    https://docs.trafficserver.apache.org/records.config#proxy.config.http.server_ports
##############################################################################
CONFIG proxy.config.http.server_ports STRING 80 443:proto=http:ssl 443:ipv6
CONFIG proxy.config.http.negative_caching_list STRING 301 302 204 305 403 404 414 500 501 502 503 504

##############################################################################
# Via: headers. Docs:
#     https://docs.trafficserver.apache.org/records.config#proxy-config-http-insert-response-via-str
##############################################################################
CONFIG proxy.config.http.insert_request_via_str INT 1
CONFIG proxy.config.http.insert_response_via_str INT 0

##############################################################################
# Parent proxy configuration, in addition to these settings also see parent.config. Docs:
#    https://docs.trafficserver.apache.org/records.config#parent-proxy-configuration
#    https://docs.trafficserver.apache.org/en/latest/admin-guide/files/parent.config.en.html
##############################################################################
CONFIG proxy.config.http.parent_proxy_routing_enable INT 1
CONFIG proxy.config.http.parent_proxy.retry_time INT 1
CONFIG proxy.config.http.parent_proxy.connect_attempts_timeout INT 30
CONFIG proxy.config.http.forward.proxy_auth_to_parent INT 0
CONFIG proxy.config.http.uncacheable_requests_bypass_parent INT 1
CONFIG proxy.config.http.forward_connect_method INT 1
##############################################################################
# HTTP connection timeouts (secs). Docs:
#    https://docs.trafficserver.apache.org/records.config#http-connection-timeouts
##############################################################################
CONFIG proxy.config.http.keep_alive_no_activity_timeout_in INT 120
CONFIG proxy.config.http.keep_alive_no_activity_timeout_out INT 120
CONFIG proxy.config.http.transaction_no_activity_timeout_in INT 900
CONFIG proxy.config.http.transaction_no_activity_timeout_out INT 900
CONFIG proxy.config.http.transaction_active_timeout_in INT 900
CONFIG proxy.config.http.transaction_active_timeout_out INT 0
CONFIG proxy.config.http.accept_no_activity_timeout INT 120
CONFIG proxy.config.net.default_inactivity_timeout INT 86400
CONFIG proxy.config.http.keep_alive_enabled_in INT 1
##############################################################################
# Origin server connect attempts. Docs:
#    https://docs.trafficserver.apache.org/records.config#origin-server-connect-attempts
##############################################################################
CONFIG proxy.config.http.connect_attempts_max_retries INT 3
CONFIG proxy.config.http.connect_attempts_max_retries_dead_server INT 1
CONFIG proxy.config.http.connect_attempts_rr_retries INT 3
CONFIG proxy.config.http.connect_attempts_timeout INT 30
CONFIG proxy.config.http.post_connect_attempts_timeout INT 1800
CONFIG proxy.config.http.down_server.cache_time INT 60
CONFIG proxy.config.http.down_server.abort_threshold INT 10
CONFIG proxy.config.cache.enable_read_while_writer INT 2
CONFIG proxy.config.http.background_fill_active_timeout INT 10
CONFIG proxy.config.http.background_fill_completed_threshold FLOAT 0.0

##############################################################################
# Negative response caching, for redirects and errors. Docs:
#    https://docs.trafficserver.apache.org/records.config#negative-response-caching
##############################################################################
CONFIG proxy.config.http.negative_caching_enabled INT 0
CONFIG proxy.config.http.negative_caching_lifetime INT 1800
CONFIG proxy.config.http.negative_caching_list STRING 301 302 204 305 403 404 414 500 501 502 503 504

##############################################################################
# Proxy users variables. Docs:
#    https://docs.trafficserver.apache.org/records.config#proxy-user-variables
##############################################################################
CONFIG proxy.config.http.insert_client_ip INT 1
CONFIG proxy.config.http.insert_squid_x_forwarded_for INT 1

##############################################################################
# Security. Docs:
#    https://docs.trafficserver.apache.org/records.config#security
##############################################################################
CONFIG proxy.config.http.push_method_enabled INT 0

##############################################################################
# Enable / disable HTTP caching. Useful for testing, but also as an
# overridable (per remap) config
##############################################################################
CONFIG proxy.config.http.cache.http INT 1

##############################################################################
# Cache control. Docs:
#    https://docs.trafficserver.apache.org/records.config#cache-control
#    https://docs.trafficserver.apache.org/en/latest/admin-guide/files/cache.config.en.html
##############################################################################
CONFIG proxy.config.http.cache.ignore_client_cc_max_age INT 0
CONFIG proxy.config.http.normalize_ae INT 1
CONFIG proxy.config.http.cache.cache_responses_to_cookies INT 1
CONFIG proxy.config.http.cache.cache_urls_that_look_dynamic INT 1
# https://docs.trafficserver.apache.org/records.config#proxy-config-http-cache-when-to-revalidate
CONFIG proxy.config.http.cache.when_to_revalidate INT 0
# https://docs.trafficserver.apache.org/records.config#proxy-config-http-cache-required-headers
CONFIG proxy.config.http.cache.required_headers INT 1

##############################################################################
# Heuristic cache expiration. Docs:
#    https://docs.trafficserver.apache.org/records.config#heuristic-expiration
##############################################################################
CONFIG proxy.config.http.cache.heuristic_min_lifetime INT 3600
CONFIG proxy.config.http.cache.heuristic_max_lifetime INT 86400
CONFIG proxy.config.http.cache.heuristic_lm_factor FLOAT 0.10

##############################################################################
# Network. Docs:
#    https://docs.trafficserver.apache.org/records.config#network
##############################################################################
CONFIG proxy.config.net.connections_throttle INT 30000
CONFIG proxy.config.net.max_connections_in INT 30000
CONFIG proxy.config.net.max_connections_active_in INT 10000

##############################################################################
# RAM and disk cache configurations. Docs:
#    https://docs.trafficserver.apache.org/records.config#ram-cache
#    https://docs.trafficserver.apache.org/en/latest/admin-guide/files/storage.config.en.html
##############################################################################
#8G RAM Cache
CONFIG proxy.config.cache.ram_cache.size INT 8589934592
CONFIG proxy.config.cache.ram_cache_cutoff INT 4194304
# https://docs.trafficserver.apache.org/records.config#proxy-config-cache-limits-http-max-alts
CONFIG proxy.config.cache.limits.http.max_alts INT 5
# https://docs.trafficserver.apache.org/records.config#proxy-config-cache-max-doc-size
CONFIG proxy.config.cache.max_doc_size INT 0
CONFIG proxy.config.cache.min_average_object_size INT 64000

##############################################################################
# Logging Config. Docs:
#    https://docs.trafficserver.apache.org/records.config#logging-configuration
#    https://docs.trafficserver.apache.org/en/latest/admin-guide/files/logging.yaml.en.html
##############################################################################
CONFIG proxy.config.log.logging_enabled INT 3
CONFIG proxy.config.log.max_space_mb_for_logs INT 20000
CONFIG proxy.config.log.max_space_mb_headroom INT 4000
CONFIG proxy.config.log.rolling_enabled INT 1
CONFIG proxy.config.log.rolling_interval_sec INT 21600
CONFIG proxy.config.log.rolling_size_mb INT 10
CONFIG proxy.config.log.auto_delete_rolled_files INT 1
CONFIG proxy.config.log.periodic_tasks_interval INT 5

##############################################################################
# These settings control remapping, and if the proxy allows (open) forward proxy or not. Docs:
#    https://docs.trafficserver.apache.org/records.config#url-remap-rules
#    https://docs.trafficserver.apache.org/en/latest/admin-guide/files/remap.config.en.html
##############################################################################
CONFIG proxy.config.url_remap.remap_required INT 1
# https://docs.trafficserver.apache.org/records.config#proxy-config-url-remap-pristine-host-hdr
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
# https://docs.trafficserver.apache.org/records.config#reverse-proxy
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.http_ui_enabled INT 3

##############################################################################
# SSL Termination. Docs:
#    https://docs.trafficserver.apache.org/records.config#client-related-configuration
#    https://docs.trafficserver.apache.org/en/latest/admin-guide/files/ssl_multicert.config.en.html
##############################################################################
CONFIG proxy.config.ssl.client.verify.server INT 0
CONFIG proxy.config.ssl.client.CA.cert.filename STRING NULL
CONFIG proxy.config.ssl.server.multicert.exit_on_load_fail INT 0
#CONFIG proxy.config.ssl.server.cert.path STRING /opt/sugarbox/ssl/sboxcdn.com
#CONFIG proxy.config.ssl.server.private_key.path STRING /opt/sugarbox/ssl/sboxcdn.com
CONFIG proxy.config.ssl.server.cipher_suite STRING ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-DSS-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA

##############################################################################
# Debugging. Docs:
#    https://docs.trafficserver.apache.org/records.config#diagnostic-logging-configuration
##############################################################################
CONFIG proxy.config.diags.debug.enabled INT 1
CONFIG proxy.config.diags.debug.tags STRING hostdb
CONFIG proxy.config.diags.output.debug STRING L
CONFIG proxy.config.diags.output.diag STRING L
# ToDo: Undocumented
CONFIG proxy.config.dump_mem_info_frequency INT 0
CONFIG proxy.config.http.slow.log.threshold INT 5000
CONFIG proxy.config.admin.user_id STRING trafficserver

CONFIG proxy.config.http.parent_proxy.per_parent_connect_attempts INT 20
CONFIG proxy.config.http.parent_proxy.mark_down_hostdb INT 0
CONFIG proxy.config.http.parent_proxy.self_detect INT 0
CONFIG proxy.config.http.no_dns_just_forward_to_parent INT 1

My remap.config - 

map https://example.com/ https://example.com/ \
@plugin=cachekey.so @pparam=--remove-all-params=true

Detailed Description of the Problem - We are using nginx as the parent server which runs on the same physical machine as ATS on port 8000. We are seeing failures where ATS tries to resolve the origin domain defined in remap.config, instead of going to the local nginx. Our expectation out of these configurations is that, remap should only be used for request transformation, we need the outbound host header as in this case is example.com to be served via the nginx that runs locally on port 8000.

What's currently happening is that ATS does not try to reach out to the local nginx in the first place instead, it DNS resolves example.com and requests that machine the domain points to, bypassing the parent configuration logic completely.

rob05c commented 2 years ago

map https://example.com/ https://example.com/

dest_domain=. method=get parent="127.0.0.1:8000" scheme=http

The scheme on the parent.config line needs to match the scheme of the remap target, not the source. The remap target here is https, so the parent.config line needs to be scheme=https to match.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. Marking it stale to flag it for further consideration by the community.

vkukk commented 9 months ago

map https://example.com/ https://example.com/

dest_domain=. method=get parent="127.0.0.1:8000" scheme=http

The scheme on the parent.config line needs to match the scheme of the remap target, not the source. The remap target here is https, so the parent.config line needs to be scheme=https to match.

Would be great time saver if this would be clearly stated in documentation. Documentation at https://docs.trafficserver.apache.org/admin-guide/files/parent.config.en.html only has following about the scheme.

scheme

    A request URL protocol: http or https.

No reference to remap dependency, not clear what request.