domain expiration check

[ArturZurawski]

Describe the bug

It seems that DOMAIN_EXPIRATIONcheck fails whenever Expiry date is few years ahead:

output from whois:

    Relevant dates:
        ...
        Expiry date:  31-Jul-2031
        ...

check is failing:

What do you see?

No response

What do you expect to see?

No response

List the steps that must be taken to reproduce this issue

Use DOMAIN_EXPIRATION check on long lasting domain

Version

No response

Additional information

No response

[lefes]

Hello! @TwiN, can you assign this bug to me? I want to solve it and participate in the project as well.

[TwiN]

@lefes sure!

This may or may not involve another project of mine: github.com/TwiN/whois

[lefes]

@ArturZurawski Hello! I am here to help you! The issue lies with the WHOIS package (like @TwiN said). Unfortunately, due to variations in the formats of returned WHOIS dates, it is impossible to provide a parsing filter for all cases. However, we can add the format of individual domains and zones as needed. It appears that your domain is not in the supported list. To assist you, I need the domain you are trying to add or a list of domains so that I can implement filters for them.

[ArturZurawski]

Hello @lefes . I would prefer not to brag about my domains on the public Internet :). Above snippet is from the actual whois output, so my guess is that is enough information to be able to phrase it.

Just in case this is Registrar:

    Registrar:
        Tucows Inc t/a Tucows [Tag = TUCOWS-CA]
        URL: https://www.tucowsdomains.com

[lefes]

@ArturZurawski Totally understandable :) In this case, the provided information will be helpful, but a crucial piece to assist you will be the TLDs (top-level domains) of what you want to monitor with the Gatus tool (for example, .uk, .com, .th, etc.).

[lefes]

@ArturZurawski , sorry for disturbing you again, but to fix this problem, I need to know the Top-Level Domains (TLDs) you want to use (for example, .uk, .com, .th, etc.). Can you provide that information for me?

[ArturZurawski]

@lefes I honestly don't understand how giving specific domains is going to solve this problem. If you are parsing the output from whois then apparently the function/regexp used does not catch the expiry date in this case. The relevant part of output was provided by me and that is all I want to make public in this case.

[lefes]

@ArturZurawski , the issue is that there is no specific format for WHOIS information. The Gatus project uses github.com/TwiN/whois to parse this information. In the whois project, TLDs are used for filtering and applying the date format. The function responsible for this can be found in the following link: https://github.com/TwiN/whois/blob/029a9259cbe132dd38eea0e6935e2fbb253ad1ec/whois.go#L150

The date format can be as follows: 01/02/2016. However, depending on the TLD, the first digit can represent either the month or the day. It is impossible to parse this date without additional context.

I am relying on what has been previously implemented in the package used by Gatus for all other cases. If you have a better solution, I would be glad to hear it and will try to implement it.

[ArturZurawski]

@lefes According to https://github.com/TwiN/whois/blob/029a9259cbe132dd38eea0e6935e2fbb253ad1ec/whois.go#L153 it seems that there should be a match for 31-Jul-2031 as in output from my whois query. Why does it not match then?

[lefes]

@ArturZurawski , I don't know, and I cannot answer your question because I do not have enough information to debug the problem. If you see your TLDs in existing filters and this still doesn't work, then I assume we need the full domain name to fix it. If you don't want to share it in public, you can send it to me on Telegram (link in my profile). Alternatively, I don't see an opportunity to fix it.

[ArturZurawski]

@lefes I'm sorry but as mentioned I will not share the domain. I think that I provided enough information to replicate the issue. If there is no way to simulate parsing whois output of long lasting domain (10 yrs) then I guess we need to close the issue or wait for someone else who will have the same problem.

[lefes]

@ArturZurawski , I checked the case where the domain has "Expiry date: 15-Jul-2034," and there is no problem at all. It worked well and looks good. It seems like 10 years is not a problem at all. I'm sorry, but I can't provide any further assistance on this problem without more information.

@TwiN , it's your call then. Are we waiting for someone else with the same problem and more information, or should we just close the issue?

[roughnecks]

I believe I got the same issue and my domain is woodpeckersnest.eu

[lefes]

@roughnecks Hi there! The .eu domains are tricky. Due to GDPR, whois information is now private and does not provide expiry dates via default whois services. Looks like I can't help you there, sorry.

[ITechGeek81]

I have 3 domains (2 .bz & 1 .nu) all registered w/ Namecheap

This is what I got when I ran whois from Ubuntu command line: .bz: "Registrar Registration Expiration Date: 2025-03-17T03:50:42.99Z" & "Registry Expiry Date: 2024-05-06T06:50:04Z" .nu: "expires: 2024-05-01" (This one is saying eNom is the registrar)

[Jeppedy]

Currently receiving a "Success" response, but with the following text: 'Get "http://mydomain.com": context deadline exceeded (Client.Timeout exceeded while awaiting headers)'

The error message seems to suggest the check was NOT as successful check. Granted, the error is with it's ability to query IANA, but shouldn't that be giving me some sort of error?

YAML Config snippet:

  - name: Domain Name Expiration
    url: "http://mydomain.com"
    interval: 6h
    client:
      timeout: 10s   <---  **_HERE'S THE ISSUE_**
    conditions:
      - "[DOMAIN_EXPIRATION] > 720h"

Partially RESOLVED: I extended the timeout to 60s and it got a successful result in ~30s. But the question remains... How do I confirm both it could contact IANA successfully AND the expiration date is not too near?

[TwiN]

@Jeppedy Have you tried playing with the [CONNECTED] placeholder?

Looking at https://github.com/TwiN/gatus/blob/1eba430797fe331aeaf4b4cc2a9dfeaff1ff47ff/core/dns.go#L53-L58

it's possible that if connection to IANA fails, the [CONNECTED] == true condition will fail, but I'm not sure.

[TwiN]

For anybody who wants to support additional domains, as I mentioned in an earlier comment, it goes through https://github.com/TwiN/whois, so I'd suggest submitting a PR if you can, but you can also create an issue on said repo 🙏

[liamjones-pw]

Bouncing off https://github.com/TwiN/gatus/issues/635#issuecomment-1901918986, any interest in supporting domains with private expiration by querying providers directly where APIs are available? E.g. AWS Go SDK Route53 GetDomainDetail.