Puny2idn

[jacobmealey]

This PR adds a few new features around converting punycode to IDN. specifically --as-idn and idn: option in the get syntax.

The development for this work was done using the latest version of curl from master. If you link against this version you should be able use these features (though trurl works with out them).

There is a large uptick in memory related issue - either double frees or curl_free not cleaning everything up. I addressed some of them but I am having trouble identifying the others, so this may take more investigation.

[jacobmealey]

@bagder Currently this branch returns trurl note: invalid url [Bad hostname] if the curl_url_get returns CURLUE_BAD_HOST. If the user enters trurl --url "http://xn-----/" --as-idn should it return http://xn-----/ or some error message like trurl note: invalid punycode ? The documentation says "If the host name is not using punycode then the original host name is used" so I think it makes sense to return http://xn-----/.

[bagder]

I think it makes sense to return http://xn-----/.

I agree. We can possibly output a warning (?) but I think it makes sense to treat --as-idn as convert-if-possible.

Does it make sense with how the libcurl API works or do we need to poke anything in that end as well?

[jacobmealey]

libcurl should be fine as is, I have a commit on my personal computer that behaves this way that I can push up when I get home. Currently libcurl reports an error on conversion which it should, and trurl (on the commit I haven't pushed up) is handling the error by simply returning original url. I can make sure it throws a warning as well.

[bagder]

I can make sure it throws a warning as well.

I'm not entirely sure on that warnings is a good idea, but I figure we can try it. I suspect that if we introduce "warnings" for some things we probably also need to provide a way to hush them up...

[jacobmealey]

hm... we already have warnf could we have some function like trurl_warnf(option *o, char *fmt, ...) and have an option for supressing warnings?

trurl_warnf(option *o, char *fmt, ...) 
{
  ... 
  if(!o->supress_warnings) {
    warnf(fmt, ap);
  }
}

[bagder]

Yes, I think that is very sensible.