3M doesn't provide author for some books. We could find this info but don't.

[ghost]

From @leonardr on August 20, 2015 15:53

A large number of 3M books (at least 170 in NYPL collection) have no author listed. This is because 3M doesn't list an author. Some of these are compilations, but a lot of them are just regular books with no author listed. This makes them show up first in lists of books sorted by author.

The metadata wrangler can fix this. We have an ISBN for most of these books. We can use that to look up author information from OCLC.

A band-aid is to set Edition.author to 'Unknown' whenever we would otherwise set it to the empty string. That would hide the authorless books deep in the list.

Copied from original issue: NYPL/Simplified-iOS#210

[ghost]

From @leonardr on August 20, 2015 15:55

fv1nfr9 9781466870482 The Year's Best Science Fiction: Thirty-Second Annual Collection d65xmr9 9781466866676 Home Made Simple ehazzz9 9781466873223 Classical Chinese Poetry d65zd89 9781442499942 Two to Kung Fu essvb89 9781481437295 Penguins of Madagascar Movie Novelization erme4r9 9781416550914 Starcraft: War Stories fd7ps89 9780374713409 City by City dhkc9r9 9780770432409 Fodor's Costa Rica 2014 dxhobr9 9780698141117 Diaries of an Unfinished Revolution eswenz9 9781466860391 Ireland: A Luminous Beauty d75rdr9 9781609452209 The Cemetery of Swallows dvazor9 9781466859517 Modern Classics of Science Fiction apf8f89 9781455510603 Blame It on Texas anegnr9 9781451693270 The Art of the Epigraph d4fr589 9780143179993 The Poisoned Pawn ds81kz9 9781476735320 Reimagining India emt1dr9 9781481405133 Kung Fu to the Rescue! fqrqvz9 9781439103340 The Complete Annotated Grateful Dead Lyrics ep7h8g9 9781476778037 World of Warcraft: Paragons 3x44g9 9781466826120 My Sherlock Holmes aghp289 9781101602829 Why We Write andstg9 9781416566205 Star Trek: Mirror Universe: Shards and Shadows apfuf89 9780316216135 Mad Science c7n2fr9 9780698136267 xo Orpheus andqmr9 9781416553069 The Library of Congress World War II Companion andqa89 9781416549925 Star Trek:Corps of Engineers: Aftermath dpr8589 9781429903868 The Year's Best Science Fiction: Twenty-Second Annual Collection dpruog9 9780312703721 The Year's Best Science Fiction: Eighteenth Annual Collection d6umr9 9781101544020 Vampires, Zombies, Werewolves and Ghosts d7o2r9 9781590209028 Mother Russia dptfoz9 9781429952545 Under African Skies and7og9 9781439148730 Last Lion dprt289 9780312207533 The Year's Best Science Fiction: Fifteenth Annual Collection apf7bz9 9781455510504 One Good Friend Deserves Another ammf589 9781439117668 Closest Companion dpuenz9 9781429978835 New Skies d5d4r9 9781101160596 The Complete Dead Sea Scrolls in English andohr9 9781416525455 Star Trek: The Original Series: Constellations Anthology esbuhz9 9781455580255 Disquiet Time d4tfz9 9781101046630 Passages d6h7r9 9781101486177 The Penguin Book of Victorian Women in Crime eruc8g9 9781476750057 Doghouse e256xr9 9781466887541 On Directing emttdg9 9781481417051 The Furry and the Furious amm1mz9 9781476702810 My Heart is Boundless erubm89 9781466876163 Women on the Verge eswd4r9 9781137386991 In the Shadow of the Rising Dragon dpugvr9 9781429983716 The Year's Best Science Fiction: Twenty-Fifth Annual Collection d6ry89 9781101533277 Hot and Steamy dpruaz9 9780312274139 Heights of the Marvelous ehaz1z9 9781466873292 I hope you are all happy now dpr85r9 9781429903844 The Year's Best Science Fiction: Twenty-First Annual Collection eqrvk89 9781250029386 The Insider's Guide to the Colleges, 2014 ammcxr9 9781416586500 College Boy dydnxg9 9780316322720 Find It in Everything ect4989 9781466870338 Making Callaloo dvaxfg9 9781466827745 Counting One's Blessings ammv3g9 9781451658897 Best of the Best American Poetry eswnhg9 9781250037473 November 22, 1963: Ordinary and Extraordinary People Recall Their Reactions When They Heard the News… ammvc89 9781451649840 Chorus esvdg89 9781466848344 50 Successful Harvard Application Essays dptg8g9 9781429954310 Dark Eros eswke89 9781429956413 Lowball anefdz9 9781442460409 Swan Sister aghumg9 9781101617441 An African Quilt eswefr9 9781466848351 The Insider's Guide to the Colleges, 2015 amm3sg9 9781476728414 Hugs Daily Inspirations for Grandmas dhkepz9 9780770432461 Fodor's Beijing dpuhqg9 9781429983068 The Year's Best Science Fiction: Twenty-Eighth Annual Collection and7n89 9781439148723 Ted Kennedy dpr8489 9781429903820 The Year's Best Science Fiction: Nineteenth Annual Collection bygyx89 9781429971133 My Big Fat Supernatural Honeymoon amk68r9 9780743270847 Cold Blooded apft4g9 9780316212878 Hop: The Chapter Book apfnmg9 9780316152099 Gossip Girl, The Carlyles #4: Love the One You're With ammzwr9 9781451691139 God's Help for Your Every Need andqyz9 9781416554707 Star Trek: The Next Generation: The Sky's the Limit dvazog9 9781466859500 The Good Old Stuff d4oxr9 9781101029121 The Penguin Book of Gaslight Crime frfuv89 9781627794336 Pinocchio 3xzmz9 9781429959742 Killer Year dhkc9g9 9780770432386 Fodor's New Orleans 2014 ewp4289 9781466884472 Modern Classic Short Novels Of Science Fiction ec2ve89 9780316287364 The Moth esvt8g9 9781250047823 The Book of Men etaksr9 9780698170094 Bohemians, Bootleggers, Flappers, and Swells c35w8z9 9780345803269 The O. Henry Prize Stories 2013 q6rp89 9781101559789 All There Is eswgg89 9780374711450 Labor Day: True Birth Stories by Today’s Best Women Writers apf82g9 9781455518586 Jewish Jocks esvxk89 9781627791465 By the Book dps6789 9781429942041 Betty Page Confidential aghqbg9 9781101601037 Fairy Tales from the Brothers Grimm dqehwz9 9780316255097 Rags & Bones cmvsor9 9780698136243 Troubled Daughters, Twisted Wives cavad89 9781455579174 This Is How You Die dvazr89 9781466859739 Pulp Friction andhar9 9780743449168 The Amazing Stories dqepuz9 9781455552085 The Book of Jezebel exs7ar9 9781466885028 Gilgamesh andkdz9 9780743480161 Star Trek: New Frontier: No Limits Anthology ammu7z9 9781451645484 Be Fruitful d71wkg9 9781481412278 If I Ran the School esst5r9 9781481426954 El libro de la vida: La novelización (The Book of Life Movie Novelization) aghu5r9 9781101619353 How to Travel the World on $50 a Day 3xqvr9 9780312207137 The Confidential Casebook of Sherlock Holmes esvgp89 9781466847590 55 Successful Harvard Law School Application Essays dvav3r9 9780316250665 Bartlett's Familiar Black Quotations beuaz9 9780316167857 Despicable Me: The Junior Novel andt7r9 9781416571544 Other People's Skin d7nesz9 9781101638873 Games Creatures Play d7ne9r9 9781455551309 Mystery Writers of America Presents Ice Cold dpsksz9 9781429918459 This I Believe beuve89 9780316280389 The Adventures of Tintin: A Novel edu21r9 9781466872059 Which Way to the Vomitorium? andmwr9 9781416510321 Star Trek: Voyager: Distant Shores Anthology andmez9 9781416503644 Star Trek:The Next Generation: Tales of the Dominion War anefdr9 9781442460393 A Wolf at the Door ew6h8g9 9781476799858 Reach d6xng9 9781101552766 Agony/Ecstasy etkrug9 9781481426091 Home dpsbcg9 9781429905695 The Year's Best Science Fiction: Twenty-Seventh Annual Collection dprxw89 9780312271626 The Year's Best Science Fiction: Seventeenth Annual Collection dpsykg9 9781429934831 Letters from Black America andcq89 9780743400817 The Lives Of Dax apf7r89 9781455522682 Mystery Writers of America Presents The Mystery Box ammbn89 9781416571445 Scouts in Bondage aefd3r9 9780807069134 The Long Walk to Freedom dpsksr9 9781429918404 20 Under 40 andmvr9 9781416510284 Star Trek: Tales From the Captain's Table d6d889 9781101462287 The Tibetan Book of the Dead dpv18r9 9781466844315 55 Successful Harvard Law School Application Essays dtn2dr9 9781439123423 Star Trek: Seven Deadly Sins d4os89 9781101024645 Other Earths aghp4r9 9781101602942 The Portable Malcolm X Reader etksnz9 9781476784434 Never Can Say Goodbye esbg2g9 9781250028044 The Year's Best Science Fiction: Thirtieth Annual Collection ewp43g9 9781466884489 Modern Classics of Fantasy esw8m89 9781466865297 The Year's Best Science Fiction: Thirty-First Annual Collection ando1r9 9781416545057 Star Trek: Strange New Worlds X eswdrg9 9781466880665 I Am the Beggar of the World c34xz89 9781455551156 The Uncensored Truth Bible for New Beginnings dhkc889 9780770432362 Fodor's San Francisco andh9g9 9780743476072 Star Trek: Deep Space Nine: Prophecy and Change Anthology esswv89 9781476708188 The Best American Poetry 2014 etk3989 9780698141841 The Penguin Book of Witches andbtz9 9780743282543 Children Playing Before a Statue of Hercules edb5zz9 9781466871113 How They Got into Harvard apf7qz9 9781455521623 The Family Corleone essvsr9 9781481423526 The Book of Life Movie Novelization 3x44r9 9781466826137 Sherlock Holmes aghprr9 9781101601099 The Tale of the Heike dpsma89 9781429916158 The Year's Best Science Fiction: Twenty-Fourth Annual Collection bhwhdg9 9781101614082 Clockwork Fairy Tales: A Collection of Steampunk Fables ev9z6z9 9781466851948 The Doll Collection eur5mz9 9781476772530 Faith f7112g9 9781476768502 Kids Pick The Funniest Poems gq8wdz9 9780316306621 Birdie's Big-Girl Shoes fmeytg9 9781621452195 1801 Home Remedies extudg9 9781614292128 Mind of Mahamudra eur7cg9 9781466838680 The Uncanny Reader fmeysg9 9781621452164 Keep This Toss That ev936r9 9781614291060 Twenty-First-Century Buddhists in Conversation fxster9 9781466891531 Teen Angst gcw6ur9 9781476768519 Miles of Smiles ausaz89 9780316246309 Despicable Me 2: The Junior Novel gzra189 9780861718948 The Lives and Liberation of Princess Mandarava f4y3hg9 9781466892637 A Galaxy Not So Far Away anecx89 9781442420311 Under the Moons of Mars essvcg9 9781481437387 Skipper's Tale

[leonardr]

The underlying problems should be double-fixed at this point, but I'll need some time to make sure that the problem goes away naturally as our scripts run.

[leonardr]

This problem is not fixed, because the data just isn't there. These books are anthologies or other books that don't have an "author" role, and rather than giving us the roles (as Overdrive does) or treating the editor of an anthology as the "author" (as we present it to patrons), 3M just says the book has no author.

There is a path to getting accurate information and it goes through the metadata wrangler. 3M gives us an ISBN. When we ask the metadata wrangler about a book, the metadata wrangler will look up that ISBN. We can have the metadata wrangler fill in author information from OCLC Linked Data if that data is missing from the primary data source for the Edition (3M in this case).

As an example, the ISBN for "Dark Eros" (dptg8g9) is 9781429954310. Looking up that ISBN in OCLC Linked Data will let us get Reginald Martin as the author, which is not 100% accurate but is good enough for our purposes, and certainly better than nothing.

There's a slight problem in that the contributions table maps Contributor to Edition. I think it would be more accurate to map Contributor to Identifier (as we do with Subject, Equivalency, Measurement, etc.) and then figure everything out when we're deciding what string to display as Work.author and Work.sort_author. But we can fix the immediate problem without making this large change.

[leonardr]

I'm reopening this due to some problems which I think need to be addressed by the refactoring and which prevent the script from working.

The immediate problem is that LinkedDataCoverageProvider.process_item() calls metadata.apply(edition) even when metadata is None. LDCP.process_oclc_edition() returns None for metadata when it believes there's no work to do. It does this "If this OCLC Number didn't tell us about any ISBNs we didn't already know, and there is no description."

The first time we look up a book, it's true that a lack of new ISBNs means there's no work to do. But when we come back and redo the work to fix something (as we're doing here), we don't expect any of the ISBNs to be new, but we want the data anyway.

I think the determination of "is there work to do" should happen within process_item(), after process_oclc_edition() returns metadata. If there is no existing edition, and the metadata is basically empty, then don't create a new edition just to hold no information. If there is an existing edition, we should always reapply the metadata, even if it doesn't seem to hold anything new.

[leonardr]

I put my change to the 3M fix script in the branch https://github.com/NYPL-Simplified/metadata_wrangler/tree/fix-3m-author-fix-script

[jce1028]

I know this is not the right place to introduce this but going forward, we should consider reporting the metadata that is actually packaged in the ePub spine item with the book as a source of info. Luckily, as readers read a book we could post the books metadata from the spine item of the book. This can literally be gathered in the background by the client (albeit with some work) sense is has the only functionality to decrypt the book and parse the spine item. Not something we want to do as part of this particular ticket but I thought relevant to the discussion. See spine item specs below.

http://www.idpf.org/epub/20/spec/OPF_2.0.1_draft.htm#Section2.2.2

http://www.idpf.org/epub/20/spec/OPF_2.0.1_draft.htm

[leonardr]

James, I filed your comment as a standalone issue, https://github.com/NYPL-Simplified/Simplified/issues/47