nginxinc / crossplane

Quick and reliable way to convert NGINX configurations into JSON and back.
Apache License 2.0
719 stars 87 forks source link

Parse nginx.conf lost the single quotes character #65

Closed lth2015 closed 5 years ago

lth2015 commented 5 years ago

crossplane is a great tool for me, It's very helpful.

In my case, we want translate the nginx.conf into golang object, and then rander it to nginx.conf with golang template. A program written in go can parse the json into golang object. I do it completely, but there is a problem for me, when a ' character occur in a directive, it will not be parsed into the result json.

for example:

log_format main '$remote_addr - $remote_user [$time_local] ' ' "$request" $status $bytes_sent $request_time ' ' "$http_referer" "$http_user_agent" ' ' "$http_x_forwarded_for" ' ' JSESSIONID=$cookie_JSESSIONID ' ' "$http_host" "$scheme" "$ssl_protocol" "$upstream_addr" "$upstream_response_time" ' ' "$http_x_yop_request_id" ';

log_format download '$remote_addr - $remote_user [$time_local] ' '"$request" $status $bytes_sent $request_time $request_length ' '"$http_referer" "$http_user_agent" ' '"$http_range" "$sent_http_content_range"';

......

ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128:AES256:AES:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK';  

when I used crossplane parse command, the result json losts the ' character:

{
              "line": 15,
              "args": [
                "main",
                "'$remote_addr - $remote_user [$time_local]  '",
                "\"$request\" $status $bytes_sent $request_time ",
                "\"$http_referer\" \"$http_user_agent\" ",
                "\"$http_x_forwarded_for\"",
                "JSESSIONID=$cookie_JSESSIONID ",
                "\"$http_host\" \"$scheme\" \"$ssl_protocol\" \"$upstream_addr\" \"$upstream_response_time\" '",
                "\"$http_x_yop_request_id\" "
              ],
              "directive": "log_format"
            },
            {
              "line": 16,
              "args": [
                "download",
                "$remote_addr - $remote_user [$time_local] ",
                "\"$request\" $status $bytes_sent $request_time $request_length",
                "\"$http_referer\" \"$http_user_agent\"",
                "\"$http_range\" \"$sent_http_content_range\""
              ],
              "directive": "log_format"
            },
}
{
                  "line": 221,
                  "args": [
                    "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128:AES256:AES:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK"
                  ],
                  "directive": "ssl_ciphers"
                },
}

How can I fix this problem?

Thank's for your help!

aluttik commented 5 years ago

When I run crossplane parse on a config with the directives you provided, I get this:

{
    "directive": "log_format",
    "line": 3,
    "args": [
        "main",
        "$remote_addr - $remote_user [$time_local] ",
        " \"$request\" $status $bytes_sent $request_time ",
        " \"$http_referer\" \"$http_user_agent\" ",
        " \"$http_x_forwarded_for\" ",
        " JSESSIONID=$cookie_JSESSIONID ",
        " \"$http_host\" \"$scheme\" \"$ssl_protocol\" \"$upstream_addr\" \"$upstream_response_time\" ",
        " \"$http_x_yop_request_id\" "
    ]
},
{
    "directive": "log_format",
    "line": 5,
    "args": [
        "download",
        "$remote_addr - $remote_user [$time_local] ",
        "\"$request\" $status $bytes_sent $request_time $request_length ",
        "\"$http_referer\" \"$http_user_agent\" ",
        "\"$http_range\" \"$sent_http_content_range\""
    ]
},
{
    "directive": "ssl_ciphers",
    "line": 7,
    "args": [
        "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128:AES256:AES:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK"
    ]
}

This is slightly different than what you provided. Your example has this argument in the first log_format:

"'$remote_addr - $remote_user [$time_local]  '",

What you should be getting is this:

"$remote_addr - $remote_user [$time_local] "

There is a reason for this. Nginx itself does not care about ' characters. Both ' characters and " are only used to escape whitespace and special characters like ; and {. They have no other purpose in the nginx syntax. The "args" list in the crossplane payload contains what nginx understands to be the separate arguments, and it doesn't care about what quotes are used.

If you want to keep the quotes for some reason, you could do something like this:

log_format main "'$remote_addr - $remote_user [$time_local] '" "' "$request" $status $bytes_sent $request_time '" ...;

That would make a really weird log format though.

If you are having problems because you're trying to write an nginx config using the JSON structure, then you're either going to have to quote the arguments manually or use crossplane build. Note that any argument or directive in an nginx config file can be quoted and it will work the same. For example, this config:

"events" {}
'http' {
    'log_format' "main" foo " " bar;
}

Is the same as this config:

events {}
http {
    log_format main 'foo ' 'bar';
}
aluttik commented 5 years ago

@lth2015 does that help?

lth2015 commented 5 years ago

@aluttik It's very helpful! thank you very much!

I want to convert the configuration files nginx.conf --> nginx.json --> nginx.conf smoothly,so I needed to keep the original file and coverted file are the same.

Under your help, the quote exist or not is Unnecessary,I will discuss with my team member and make it completely.

thank you for you help!

aluttik commented 5 years ago

Cool, glad everything worked out 🙂 I'm going to close this issue then.