Closed meriouma closed 7 years ago
Unfortunately, you're running into #186 -- the Federal Register hasn't converted many of their pre-2000 documents into XML yet. We should probably be issuing a warning and skipping these.
If you're up for it, I think the best place to do this would be when we generate the initial list of versions from the Federal Register's API. We can probably request the full_text_xml_url
field (if we don't already) and filter the results to include only notices where that's present. I think that logic would be somewhere inside this function: https://github.com/eregs/regulations-parser/blob/47c8648fb0ea1f19bba40dcab327b7c98db78275/regparser/commands/versions.py#L15
Ah thanks for the pointer, I didn't look for related issues. I'll try to see what I can achieve!
Here's what I tried. I commented changes from 1 to 13 in the commit (in the order I added the changes). At the moment it doesn't yield errors anymore, but it seems to end up in an infinite loop. I guess my changes created inconsistent data. https://github.com/navigo/regulations-parser/commit/881df6fff5d561831bd5ce815fbd41ceb04323a1
It looks like @cmc333333 addressed the initial subject of this issue in this commit [ 7bb4092 ] which was merged into eregs/regulations-parser
master
as part of #186. Shall we close this issue and create new issues to track the additional challenges @meriouma has identified in attempting to parse 37 CFR Part 1? Is there a protocol eregs
likes to follow in this circumstance?
I'd be for closing this issue and creating another for the other 37 CFR 1 issues. That said, there's not a whole lot of traffic in this repo given that the 18F team is focused on OMB; having an extra issue doesn't hurt :)
@gregoryfoster's been using other issues to track 37 CFR 1 errors, so I'm going to close this one.
I've tried running
eregs clear && eregs pipeline 37 1 http://localhost:8080
and end up with this stacktrace :I've tracked it down to
meta['full_text_xml_url']
returningNone
. The log entry I have just before isAttempting to resolve dependency: .eregs_index/notice_xml/95-6377
.Should I try to skip that notice?