deanishe / zothero

Rapidly search and cite Zotero entries from Alfred
MIT License
480 stars 31 forks source link

Juris-M Support #25

Closed yilu1015 closed 5 years ago

yilu1015 commented 5 years ago

I use Juris-M, a variant of Zotero, to manage my bibliography. Its database is structured just like Zotero's, but it's stored at ~/Juris-M as jurism.sqlite. I tried updating ZOTERO_DIR to ~/Juris-M, but it appears that ZotHero still tries to look for zotero.sqlite there. How do I make it look for jurism.sqlite instead?

Thanks in advance for your help!

deanishe commented 5 years ago

You'd have to change the few places in the code where zotero.sqlite is hard-coded:

That said, it's likely the workflow isn't set up to use the fields you'd want (see #16).

yilu1015 commented 5 years ago

Thank you for your reply. I modified the two files per your instruction and can confirm that the directory is pointing correctly to ~Juris-M/jurism.sqlite, but still received the following error message:

08:35:58 core.py:197 INFO     [core] searching for "cultural revolution" ...
08:35:58 index.py:245 INFO     [index] 0 result(s) for u'cultural revolution'
08:35:58 workflow.py:2080 ERROR    Zotero database not found: u'~/Juris-M/jurism.sqlite'
Traceback (most recent call last):
  File "./lib/workflow/workflow.py", line 2073, in run
    func(self)
  File "./zh", line 669, in main
    return do_reindex()
  File "./zh", line 575, in do_reindex
    app.update_index(force=False)
  File "./lib/zothero/core.py", line 166, in update_index
    self.index.update(self.zotero, force)
  File "./lib/zothero/core.py", line 125, in zotero
    raise ValueError('Zotero database not found: %r' % original)
ValueError: Zotero database not found: u'~/Juris-M/jurism.sqlite'

I understand the workflow is not set up for variants of Zotero, but if there might be another workaround, please kindly let me know. Many thanks.

deanishe commented 5 years ago

You found a bug. Fixed in v1.2.2.

Grab the new workflow from the above link or enter zot workflow:update in Alfred.

You'll have to change the DB names again, I'm afraid.

yilu1015 commented 5 years ago

Thank you. Having modified the DB names, I confirm that the workflow supports Juris-M.

deanishe commented 5 years ago

Does it find what you need?

I don't know anything about citing legal cases, and #16 indicated that the workflow doesn't do a good job of it. Or does Juris-M rejigger the regular fields to put case name in the normal title field?

deanishe commented 5 years ago

Also, I'll look into adapting the workflow so it can handle differently-named databases as well as different directories.

yilu1015 commented 5 years ago

Yes. The workflow has successfully tapped into Juris-M's database, but after some experiments, I found that there're two additional issues.

  1. Some items in non-Western languages are picked by Zotero/Juris-M, but not by the workflow. I cannot quite establish a pattern based on my few searches, but it appears that theses (as item type) are particularly affected. I have three theses in Chinese my database; none of them appears in the search result (regardless of general search or search by field).
Screen Shot 2019-09-03 at 4 50 54 PM

The screenshot shows me trying to search the first result in ZotHero. Below shows another example, showing that the workflow fails to capture the selected entry (highlighted in blue) in Juris-M:

Screen Shot 2019-09-03 at 5 02 16 PM

I don't think it's an issue with encoding; after all, the entire database, I believe, is in UTF-8. I also don't think this is a general issue against Chinese (or other non-Western languages); most of them show up just fine. So far, I have only found consistent missing results from theses; English-language theses appear correctly in the result.

  1. The second issue is more specific to Juris-M: the reason why I use it rather than vanilla Zotero is that it supports multilingual fields. More specifically, it allows users to add transliterations and translations to bibliography entries in non-Western languages. In Chicago, the proper citation for a work in Chinese should include transliterations for author, title, and publisher; the title additionally needs to an English translation.

The following is a correct citation: Chen Zhongyuan 陈重远, Liulichang shihua 琉璃厂史话 [History of Liulichang] (Beijing chubanshe, 2015). However, when I use ZotHero to generate a bibliography, the variants are all gone; what remains is simply the citation in the original language: 陈重远, 琉璃厂史话 (北京出版社, 2015).

This is irrespective of the CSL I use: after all, Juris-M uses the same CSL but is able to render the citation correctly. What went wrong in the style translation?

Both issues I raise here are related to non-Western languages; however, I do not know what exactly causes them. If there is any additional information I could provide, please let me know. I'm also happy to export some entries and send them to you for testing.

Once again, thank you for your wonderful workflow and generous help.

deanishe commented 5 years ago

fails to capture the selected entry

Try inserting an asterisk in front of your query, i.e. title:*<ideograms>

If that works, I can make the workflow insert it automatically.

This is irrespective of the CSL I use

I don’t think CSL has any notion of transliterations. Juris-M is presumably combining the “real” field and its transliteration before passing it to the stylesheet. Or maybe I’m wrong…

If you could provide me with a small sample database that exhibits the issue—and tell me the expected output—I’ll see what I can do.

yilu1015 commented 5 years ago

I created a test library and exported it in CSL JSON format, which captures all the multilingual fields that Juris-M has created. Two of them are theses in Chinese language; as control group, I have also created two entries using their translated titles and author names. (So they appear like English theses). To these four, I have also included a book that shows up consistently and correctly in my workflow searches.

Here's the raw csl json. Note that pny stands for pinyin; it is a transliteration of Chinese.

[
    {
        "id": "zhen2018",
        "type": "thesis",
        "multi": {
            "main": {
                "title": "zh"
            },
            "_keys": {
                "title": {
                    "en": "Evolution of Beijing Antique Culture: From Liulichang to Panjiayuan",
                    "pny": "beijing wenwan wenhua de yanjing: cong liulichang dao panjiayuan"
                },
                "publisher": {
                    "en": "Chongqing Normal University"
                }
            }
        },
        "title": "北京文玩文化的演进—从琉璃厂到潘家园",
        "publisher": "重庆师范大学",
        "genre": "硕士",
        "source": "CNKI",
        "abstract": "",
        "URL": "http://eng.oversea.cnki.net/kcms/detail/detail.aspx?recid=&FileName=GHJS200801036&DbName=CJFD2008&DbCode=CJFD&uid=WEEvREcwSlJHSldRa1Fhb09jT0pjbHAxcWg5ZllMUWphSkY3YUx1MGFRND0=$9A4hF_YAuvQ5obgVAqNKPCYcEjKensW4IQMovwHtwkF4VYPoHbKxJw!!",
        "author": [
            {
                "family": "甄",
                "given": "宏达",
                "multi": {
                    "_key": {
                        "en": {
                            "family": "Zhen",
                            "given": "Hongda"
                        }
                    }
                }
            }
        ],
        "issued": {
            "date-parts": [
                [
                    "2018"
                ]
            ]
        },
        "accessed": {
            "date-parts": [
                [
                    "2019",
                    5,
                    27
                ]
            ]
        }
    },
    {
        "id": "liu2010",
        "type": "thesis",
        "multi": {
            "main": {
                "title": "zh",
                "publisher": "zh"
            },
            "_keys": {
                "title": {
                    "en": "\"Internal Reference\": A Unique Cultural Phenomenon in the Cold War, 1951-1978",
                    "pny": "\"neibu faxing\": lengzhan beijingxia de yizhong teshu wenhua xianxiang"
                },
                "publisher": {
                    "en": "East China Normal University"
                }
            }
        },
        "title": "“内部发行”:冷战背景下的一种特殊文化现象(1951-1978)",
        "publisher": "华东师范大学",
        "source": "CNKI",
        "URL": "http://eng.oversea.cnki.net/kcms/detail/detail.aspx?recid=&FileName=2010197736.nh&DbName=CMFD2011&DbCode=CMFD&uid=WEEvREcwSlJHSldRa1Fhb09jT0pjbHAxcWg3MElkdnlzWHdmNkpIWXV5Yz0=$9A4hF_YAuvQ5obgVAqNKPCYcEjKensW4IQMovwHtwkF4VYPoHbKxJw!!",
        "language": "zh",
        "author": [
            {
                "family": "刘",
                "given": "阳",
                "multi": {
                    "_key": {
                        "pny": {
                            "family": "Liu",
                            "given": "Yang"
                        }
                    }
                }
            }
        ],
        "issued": {
            "date-parts": [
                [
                    "2010"
                ]
            ]
        },
        "accessed": {
            "date-parts": [
                [
                    "2019",
                    5,
                    27
                ]
            ]
        }
    },
    {
        "id": "chen2015e",
        "type": "book",
        "multi": {
            "main": {
                "title": "zh",
                "publisher": "zh"
            },
            "_keys": {
                "title": {
                    "en": "History of Liulichang",
                    "pny": "Liulichang shihua"
                },
                "publisher": {
                    "pny": "Beijing chubanshe"
                }
            }
        },
        "title": "琉璃厂史话",
        "publisher": "北京出版社",
        "number-of-pages": "198",
        "ISBN": "978-7-200-11112-5",
        "language": "zh",
        "author": [
            {
                "family": "陈",
                "given": "重远",
                "multi": {
                    "_key": {
                        "pny": {
                            "family": "Chen",
                            "given": "Zhongyuan"
                        }
                    },
                    "main": "zh"
                }
            }
        ],
        "issued": {
            "date-parts": [
                [
                    "2015"
                ]
            ]
        }
    },
    {
        "id": "zhen2018a",
        "type": "thesis",
        "multi": {
            "main": {
                "title": "en"
            },
            "_keys": {}
        },
        "title": "Evolution of Beijing Antique Culture: From Liulichang to Panjiayuan",
        "publisher": "Chongqing Normal University",
        "genre": "Masters",
        "source": "CNKI",
        "URL": "http://eng.oversea.cnki.net/kcms/detail/detail.aspx?recid=&FileName=GHJS200801036&DbName=CJFD2008&DbCode=CJFD&uid=WEEvREcwSlJHSldRa1Fhb09jT0pjbHAxcWg5ZllMUWphSkY3YUx1MGFRND0=$9A4hF_YAuvQ5obgVAqNKPCYcEjKensW4IQMovwHtwkF4VYPoHbKxJw!!",
        "language": "en",
        "author": [
            {
                "family": "Zhen",
                "given": "Hongda",
                "multi": {
                    "_key": {}
                }
            }
        ],
        "issued": {
            "date-parts": [
                [
                    "2018"
                ]
            ]
        },
        "accessed": {
            "date-parts": [
                [
                    "2019",
                    5,
                    27
                ]
            ]
        }
    },
    {
        "id": "liu2010g",
        "type": "thesis",
        "multi": {
            "main": {},
            "_keys": {}
        },
        "title": "\"Internal Reference\": A Unique Cultural Phenomenon in the Cold War, 1951-1978",
        "publisher": "East China Normal University",
        "source": "CNKI",
        "URL": "http://eng.oversea.cnki.net/kcms/detail/detail.aspx?recid=&FileName=2010197736.nh&DbName=CMFD2011&DbCode=CMFD&uid=WEEvREcwSlJHSldRa1Fhb09jT0pjbHAxcWg3MElkdnlzWHdmNkpIWXV5Yz0=$9A4hF_YAuvQ5obgVAqNKPCYcEjKensW4IQMovwHtwkF4VYPoHbKxJw!!",
        "author": [
            {
                "family": "Liu",
                "given": "Yang",
                "multi": {
                    "_key": {}
                }
            }
        ],
        "issued": {
            "date-parts": [
                [
                    "2010"
                ]
            ]
        },
        "accessed": {
            "date-parts": [
                [
                    "2019",
                    5,
                    27
                ]
            ]
        }
    }
]

It might be difficult to test multilingual citation rendering without Juris-M, but here's my configuration and the standard citation format. Juris-M is able to output the following citations correctly:

Chen Zhongyuan 陈重远, Liulichang shihua 琉璃厂史话 [History of Liulichang] (Beijing chubanshe, 2015); 

Liu Yang 刘阳, “‘neibu faxing’: lengzhan beijingxia de yizhong teshu wenhua xianxiang” “内部发行”:冷战背景下的一种特殊文化现象(1951-1978) [“Internal Reference”: A Unique Cultural Phenomenon in the Cold War, 1951-1978] (华东师范大学, 2010); 

甄宏达 Hongda Zhen, “beijing wenwan wenhua de yanjing: cong liulichang dao panjiayuan” 北京文玩文化的演进—从琉璃厂到潘家园 [Evolution of Beijing Antique Culture: From Liulichang to Panjiayuan] (硕士, 重庆师范大学, 2018).

I think you are right to say that Juris-M did something to the fields before translating the database into citation formats. This hypothesis is confirmed by the fact that pandoc cannot render the citations correctly even with the same CSL, either. Please let me know if you have anything to add to this diagnosis so that I may pursue this further with developers of Juris-M.