search

blakearchive / erdman

0 stars 0 forks source link

items not showing up in TOC #41

Closed ghost closed 7 years ago

ghost commented 7 years ago

poems whose titles are the first lines (in quotation marks) aren't showing up in the TOC. for example:

"Are not the joys of morning sweeter"

and many others

queryluke commented 7 years ago

they aren't showing up in your current implementation of erdman http://www.blakearchive.org/blake/erdman.html, I don't see it under SONGS AND BALLADS

ghost commented 7 years ago

yeah, it's wrong on our current site. the actual erdman printed book has them in the TOC. do we need to edit the XML?

ghost commented 7 years ago

here's the relevant XML:

<div2 page="471" type="poem" id="b5.15"><lg><head type="incipits">"Are not the joys of morning sweeter"</head>

ghost commented 7 years ago

and the first couple poems from Satiric Verses:

<supplied>[SATIRIC VERSES AND EPIGRAMS]</supplied></head>
        <lb rend="hardrule"/>
        <div2 id="b6.1" page="499" type="section">
            <div3 id="b6.1.1" page="499" type="poem">
                <milestone unit="[From Blake's Notebook]"/>
                <head rend="center">Motto to the Songs of Innocence &amp; of Experience</head>
                <lg>
                    <l n="1">The Good are attracted by Mens perceptions</l>
                    <l n="2" rend="1">And Think not for themselves</l>
                    <l n="3" rend="1">Till Experience teaches them to catch</l>
                    <l n="4" rend="1">And to cage the Fairies &amp; Elves</l>
                </lg>
                <lg>
                    <l n="5">And then the Knave begins to snarl         </l>
                    <l n="6">And the Hypocrite to howl</l>
                    <l n="7">And all his good Friends shew their private ends</l>
                    <l n="8">And the Eagle is known from the Owl</l>
                </lg>
            </div3>
            <div3 id="b6.1.2" page="499" type="poem">
                <lg>
                    <head type="incipits">&quot;Let the Brothels of Paris be opened&quot;</head>
                    <l n="1">Let the Brothels of Paris be opened
                        <ptr target="tn499.2"/></l>
                    <l n="2">With many an alluring dance</l>
                    <l n="3">To awake the Physicians thro the city 
                        <ptr target="tn499.3"/></l>
                    <l n="4">Said the beautiful Queen of France</l>
                </lg>
                <lg>
                    <l n="5">Then old Nobodaddy aloft                 </l>
                    <l n="6">Farted &amp; belchd &amp; coughd</l>
                    <l n="7">And said I love hanging &amp; drawing &amp; quartering</l>
                    <l n="8">Every bit as well as war &amp; slaughtering
                        <ptr target="tn499.4"/></l>
                </lg>
                <lg>
                    <l n="9">Then he swore a great &amp; solemn Oath 
                        <ptr target="tn499.5"/></l>
                    <l n="10">To kill the people I am loth               </l>
                    <l n="11">But If they rebel they must go to hell</l>
                    <l n="12">They shall have a Priest &amp; a passing bell</l>
                </lg>
                <pb n="500"/>
                <lg>
                    <l n="13">The King awoke on his couch of gold</l>
                    <l n="14">As soon as he heard these tidings told</l>
                    <l n="15">Arise &amp; come both fife &amp; drum         </l>
                    <l n="16">And the 
                        <del>
                            [
                            <hi rend="i">Famine</hi>
                            ]</del>shall eat both crust
&amp; crumb
                        <ptr target="tn500.1"/></l>
                </lg>
                <lg>
                    <l n="17">The Queen of France just touchd this Globe</l>
                    <l n="18">And the Pestilence darted from her robe
                        <ptr target="tn500.2"/></l>
                    <l n="19">But our good Queen quite grows to the ground</l>
                    <l n="20">And a great many suckers grow all around 
                        <ptr target="tn500.3"/></l>
                </lg>
            </div3>
queryluke commented 7 years ago

It may be as simple as removingtype="incipits". I don't know why the import scripts are ignoring them, or maybe it's the XSLT doesn't have a template for them.

ghost commented 7 years ago

well, but this isn't showing in the TOC either:

<head rend="center">Motto to the Songs of Innocence &amp; of Experience</head>

ghost commented 7 years ago

i think the problem is that it's a div within a div, and this part of the import code seems to be relevant:

def get_titles(tree):
    heads = tree.xpath("//head")
    seen_pages = set()

    def should_add(head):
        parent = head.getparent()
        if parent.tag in {"div1", "div2", "div3", "div4"}:
            id_ = parent.attrib["id"]
            if id_ not in seen_pages:
                seen_pages.add(id_)
                return True
        return False

    return {
            head.getparent().attrib["id"]: {
                'heading': head.xpath("string()").strip(),
                'page': head.getparent().attrib["page"] if head.getparent().attrib["page"] else ""
            } for head in heads if should_add(head)
        }
ghost commented 7 years ago

i take that back. others like that are working:

<div3 id="b8.3.3" page="530">
                <milestone n="1" rend="left" unit="page"/>
                <head rend="center">DESCRIPTIVE CATALOGUE, 
                    <lb/>
                    &amp;C. &amp;C.</head>
                <lb rend="hardrule"/>

that's in a div2

queryluke commented 7 years ago

yeah and I tried removing the type attribute, no luck

ghost commented 7 years ago

hmm, maybe because those headings are also in <supplied>. i mean the topmost heading, Satiric Verses

ghost commented 7 years ago

i don't have my environment set up properly right now. tomorrow, could you try taking out <supplied> from the Satiric Verses heading. if that fixes things, i can fix the rest of the <supplied> tags. gonna stop working for the night

queryluke commented 7 years ago

I don't think it's the supplied. I think it's that some <divX> elements don't have headers

Look a The Pickering Manuscript, it opens a <div2> with <head>The Pickering Manuscript</head> It closes after "William Bond" (A <div3>). Then a <div2> opens, then a <div3>. No <head> element exists between the <div2> and <div3>. Which results in "Mr Blake's Nursery Rhyme" not showing up.

Follow that logic for the code you posted above and it matches. So there are some <divX> that don't have head tags. If they don't have <head> tags there is nothing to create a TOC for.

So it looks like this might require some major BAD editing.

ghost commented 7 years ago

yeah, i see, but shouldn't that just mean Mrs. Blake's Record should come up after The Pickering Manuscript?

ghost commented 7 years ago

would the solution be to move the <head> tags coming after <div3> to between <div2> and <div3>?

ghost commented 7 years ago

hmm, that doesn't seem right, either. i feel like the import script should be able to deal with this structure, but if we can figure out a fix in the XML, that would be fine.

queryluke commented 7 years ago

Think about it in non-programmatic terms:

Are "The Pickering Manuscript" and "Mr Blake's Nursery Rhyme" on the same hierarchical level? I.e., Should the headers under Songs and Ballads be:

If Mr Blake's Nursery Rhyme is on the same hierarchical level, it should be it's own div2.

From:
<div2 page="498" type="section" id="b5.45">
  <milestone unit="[Mrs Blake's record]"/>
  <p>
    <ptr target="tn498.1"/>
  </p>
  <div3 page="498" type="poem" id="b5.45.1">
    <head rend="center">Mr Blake's Nursery Rhyme</head>
    ...
  </div3>
</div2>

To:
<div2 page="498" type="poem" id="b5.45">
  <milestone unit="[Mrs Blake's record]"/>
  <p>
    <ptr target="tn498.1"/>
  </p>
  <head rend="center">Mr Blake's Nursery Rhyme</head>
    ...
</div2>

Or, is "Mr Blake's Nursery Rhyme" part of the section "Mrs Blake's Record"? Should the TOCs look like:

If that's the case, it's a matter of giving the <div2> a <head> value:

From:
<div2 page="498" type="section" id="b5.45">
  <milestone unit="[Mrs Blake's record]"/>
  <p>
    <ptr target="tn498.1"/>
  </p>
  <div3 page="498" type="poem" id="b5.45.1">
    <head rend="center">Mr Blake's Nursery Rhyme</head>
    ...
  </div3>
</div2>

To:
<div2 page="498" type="section" id="b5.45">
  <milestone unit="[Mrs Blake's record]"/>
  <head rend="center">Mrs. Blake's Record</head>
  <p>
    <ptr target="tn498.1"/>
  </p>
  <div3 page="498" type="poem" id="b5.45.1">
    <head rend="center">Mr Blake's Nursery Rhyme</head>
    ...
  </div3>
</div2>

In this latter example, if you don't want the the <head> tag to render on the actual page, we can add some additional xsl templates and use a rend="none" to indicate the element should be hidden. But the <head> tag would still be parsed and propagate a TOC.

ghost commented 7 years ago

yes, my point was that it seems like it should've been showing up, incorrectly but still showing up.

[Mrs. Blake's record] should just be like [Songs 1] in the Songs of Innocence and of Experience. it's just a side heading, if you will. "Mr. Blake's Nursery Rhyme" should be a poem under the Pickering Manuscript--the two aren't on the same level.

so:

Several Questions Answered The Pickering Manuscript [indent] ... [indent] William Bond [indent] Mrs. Blake's Nursery Rhyme

actually, now I've noticed that "My Spectre around me night & day" is wrong. it should be under Several Questions Answered.

have a look at the TOC in the actual book:

https://books.google.com/books?id=p-yaJajW3kEC&printsec=frontcover&dq=the+complete+poetry+%26+prose+of+william+blake+pdf&hl=en&sa=X&ved=0ahUKEwib65bBruLQAhXJKCYKHW1pApoQ6wEIHDAA#v=onepage&q&f=false

ghost commented 7 years ago

[Mrs. Blake's Record] is a <milestone>, which has an entry in the XSL

ghost commented 7 years ago

in the XSL:

<xsl:template match="milestone">
        <div class="plate">
            [
            <span class="plate-unit-name">
                <xsl:value-of select="@unit"></xsl:value-of>
            </span>
            <span class="plate-unit-number">
                <xsl:value-of select="@n"></xsl:value-of>
            </span>
            ]
            <xsl:apply-templates/>
        </div>
    </xsl:template>
ghost commented 7 years ago

just made a change that i think might fix the pickering manuscript. but i tried to deploy and i got that same python26 error:

EBUG [78577729] Command: cd ../erdman; git checkout master; git pull origin master; git checkout python26; git merge --no-ff -m 'Merge master into python26' master; git push origin python26; git checkout master; DEBUG [78577729] error: you need to resolve your current index first DEBUG [78577729] scripts/transform.py: needs merge DEBUG [78577729] error: Pull is not possible because you have unmerged files. DEBUG [78577729] hint: Fix them up in the work tree, and then use 'git add/rm ' DEBUG [78577729] hint: as appropriate to mark resolution and make a commit. DEBUG [78577729] fatal: Exiting because of an unresolved conflict. DEBUG [78577729] error: you need to resolve your current index first DEBUG [78577729] scripts/transform.py: needs merge DEBUG [78577729] error: merge is not possible because you have unmerged files. DEBUG [78577729] hint: Fix them up in the work tree, and then use 'git add/rm ' DEBUG [78577729] hint: as appropriate to mark resolution and make a commit. DEBUG [78577729] fatal: Exiting because of an unresolved conflict. DEBUG [78577729] To https://github.com/blakearchive/erdman.git DEBUG [78577729] ! [rejected] python26 -> python26 (non-fast-forward) DEBUG [78577729] error: failed to push some refs to 'https://github.com/blakearchive/erdman.git' DEBUG [78577729] hint: Updates were rejected because the tip of your current branch is behind DEBUG [78577729] hint: its remote counterpart. Integrate the remote changes (e.g. DEBUG [78577729] hint: 'git pull ...') before pushing again. DEBUG [78577729] hint: See the 'Note about fast-forwards' in 'git push --help' for details. DEBUG [78577729] error: you need to resolve your current index first DEBUG [78577729] scripts/transform.py: needs merge error: you need to resolve your current index first

ghost commented 7 years ago

i changed it to read:

<div3 id="b5.45.1" page="498" type="poem">
                <milestone unit="[Mrs Blake's record]"/>
                <p>
                    <ptr target="tn498.1"/>
                </p>
                <head rend="center">Mr Blake's Nursery Rhyme</head>
                <lg>
                    <l n="1">The sow came in with the saddle,</l>
                    <l n="2">The little pig rocked the cradle,</l>
                    <l n="3">The dish jumped o' top of the table</l>
                    <l n="4">To see the brass pot swallow the ladle.</l>
                    <l n="5">The old pot behind the door                     </l>
                    <l n="6">Called the kettle a blackamoor.</l>
                    <l n="7">'Odd bobbs' said the gridiron, 'can't you agree?</l>
                    <l n="8">I'm the head constable, bring them to me.'</l>
                </lg>
            </div3>
ghost commented 7 years ago

just made a similar change to fix "My Spectre around me night & day"

queryluke commented 7 years ago

The change to "my sepctre" worked. But you also will need to update your div ID's. See my updated ID commit for Mr Blake's Nursery Rhyme.

ghost commented 7 years ago

ok, all set. i think i've fixed the Pickering Manuscript, too. but i'm still getting the deploy error. could you look into that?

ghost commented 7 years ago

just pushed a change for Satiric Verses. i think they're fixed now

queryluke commented 7 years ago

sorry I pushed a fix to the deploy but forgot to tell you. You should be able to do a git pull in your local erdman-deploy directory.

And all those fixes seemed to have worked. I've deployed the code to qa.

ghost commented 7 years ago

ok, i've just pushed a big fix to Songs and Ballads

ghost commented 7 years ago

did the deploy, but nothing seems to have changed. you mentioned modifying data.js after making changes?

queryluke commented 7 years ago

yeah, I'm going to have to make notes on how to do this. In the normal development cycle data.js is createded when you run transform.py. But we can't do transform.py locally (becasue of OSX sierra). So currently I have to pull down data.js from the server, run npm run-script watch then redeploy. On the second redeploy I don't seed the db.

If the OSX sierra ever gets fixed this won't be an issue

ghost commented 7 years ago

i got ya. thanks, luke.

ok, let me know when you do another round. i think i've made all the fixes needed. Songs and Ballads should show a whole bunch of items in the TOC, especially under 'To Go on I Plate' under Songs and Ballads

queryluke commented 7 years ago

ok, changes should be up now