Attempt to download rfcxxxx state ... error

[mjethanandani]

A run of idnits with the latest version results in the following error for all RFC cited in the draft. In this case it was being run against draft-ietf-idr-bgp-model-15.txt

Attempted to download rfc8277 state... The downloaded file seems to be corrupt, proceeding with outdated information.

Removing the .idnits directory in home does not help.

[rjsparks]

I'm not able to reproduce with idnits 2.17.1

Could you verify that wget on your affected system behaves this way?

wget --timeout=2 -q -O - --no-check-certificate https://datatracker.ietf.org/doc/rfc8277/idnits2-state/
Doc-tag: draft-ietf-mpls-rfc3107bis;datatracker
Doc-rfcnum: 8277
Doc-created: 2016-09-16;datatracker
Doc-deststatus: Proposed Standard;datatracker
Doc-rev: 04;datatracker

[mjethanandani]

I get the same output of the above command. So do not know what is happening. Here is some more info.

Version check.

$ ../idnits/idnits --version
idnits  2.17.1

Non-debug run after removing .idnits folder in home directory.

   Attempted to download rfc0000 state...
   Failure fetching the file, proceeding without it.
   Attempted to download rfc159 state...
   Failure fetching the file, proceeding without it.
   Attempted to download rfc201 state...
   Failure fetching the file, proceeding without it.
   Attempted to download rfc428 state...
   Failure fetching the file, proceeding without it.
   Attempted to download rfc5800 state...
   Failure fetching the file, proceeding without it.
   Attempted to download rfc665 state...
   Failure fetching the file, proceeding without it.

Debug run after removing the .idnits folder in home directory. The problem seems to be with some specific RFCs.

+ for ref in '$rfcrefs'
echo $ref | tr -d "\r"
++ echo 201
++ tr -d '\r'
+ ref=201
+ name=rfc201
+ num=201
+ '[' '!' rfc201 = rfc ']'
+ download 'rfc201 state' https://datatracker.ietf.org/doc/rfc201/idnits2-state/ /Users/mahesh/.idnits/rfc201.state 'Doc-rfcnum: 201'
+ name='rfc201 state'
+ url=https://datatracker.ietf.org/doc/rfc201/idnits2-state/
+ file=/Users/mahesh/.idnits/rfc201.state
+ test='Doc-rfcnum: 201'
+ cd /Users/mahesh/.idnits
+ '[' /Users/mahesh/.idnits/rfc201.state ']'
$FDATE $file.date 2>/dev/null
++ stat -f %Sm -t %Y-%m-%d /Users/mahesh/.idnits/rfc201.state.date
+ filedate=
tmpfile
++ tmpfile
basename $0
+++ basename ../idnits/idnits
++ prefix=idnits
++ for tmpdir in '$TMPDIR' '$TMP' '$TEMP' /tmp .
++ '[' -d /var/folders/gl/sn4vgdl1119900k_2gn21ps0000ghf/T/ -a -w /var/folders/gl/sn4vgdl1119900k_2gn21ps0000ghf/T/ ']'
++ tmpfn=/var/folders/gl/sn4vgdl1119900k_2gn21ps0000ghf/T//idnits-57611.tmp
++ '[' -f /var/folders/gl/sn4vgdl1119900k_2gn21ps0000ghf/T//idnits-57611.tmp ']'
++ rm /var/folders/gl/sn4vgdl1119900k_2gn21ps0000ghf/T//idnits-57611.tmp
++ echo /var/folders/gl/sn4vgdl1119900k_2gn21ps0000ghf/T//idnits-57611.tmp
++ exit
+ tempfile=/var/folders/gl/sn4vgdl1119900k_2gn21ps0000ghf/T//idnits-57611.tmp
+ debug 'Status file /Users/mahesh/.idnits/rfc201.state is dated .'
+ '[' '' ']'
+ '[' '' = 2023-01-19 ']'
+ note ''
+ '[' '' ']'
+ '[' -f /Users/mahesh/.idnits/rfc201.state ']'
+ note ' - No rfc201 state file. Attempting to download it...'
+ '[' '' ']'
+ /usr/local/bin/wget --timeout=2 -q -O - https://datatracker.ietf.org/doc/rfc201/idnits2-state/
+ '[' -z '' ']'
+ '[' '' ']'
+ echo '   Attempted to download rfc201 state...'
   Attempted to download rfc201 state...
+ echo '   Failure fetching the file, proceeding without it.'
   Failure fetching the file, proceeding without it.
+ touch /Users/mahesh/.idnits/rfc201.state.date

[rjsparks]

The failure at rfc201 is expected - there is no state to give you for that RFC. Same for all in the list you show in the above comment. See, e.g., https://www.rfc-editor.org/info/rfc201

When you reran this, though, you didn't get the following that you originally reported? If there's an issue to fix here, that's the exercise point to focus on.

Attempted to download rfc8277 state...
The downloaded file seems to be corrupt, proceeding with outdated information.

If you didn't, and the only complaints you see are for the 6 you listed above, this is expected, and you should close the issue.

[mjethanandani]

Hi @rjsparks, we could be dealing with two different issues here.

The first one being the six RFCs that have no state information. I know my document does not reference RFC201, so not sure why it is being downloaded. I can close it, but idnits will continue to give these errors unless either we do not download them, or have some dummy download file for them.

To the second error about file being corrupt, I have seen that error more than once. It was probably not in the last run. I can keep an eye out, but suspect the bug to be transient. Could it be the datatracker is not able to keep up with all the requests coming at it?

[rjsparks]

The first one being the six RFCs that have no state information. I know my document does not reference RFC201, so not sure why it is being downloaded. I can close it, but idnits will continue to give these errors unless either we do not download them, or have some dummy download file for them.

Yes - we have no plan to change that for this implementation of idnits. We will look to make the next re-implementation behave differently.

Could it be the datatracker is not able to keep up with all the requests coming at it?

We have no indication of the datatracker not being able to keep up, but the script is running as an anonymous user and has a short timeout (2), so there's a chance that if cloudflare (or the datatracker if the edge closest to you doesn't have the document in cache) isn't responsive in that short timeout that you could be running into issues.