Development environment (for Zotero 5.x extensions)

[ghost]

Can you pelase provide some info on how to setup a development environment on Linux? i.e. specifically which build tools are you using... If I understand correctly, Zotero 5.x is still a XUL application and extensions like Zutilo are old-style (pre jetpacks and pre web-extensions) extensions? What is a possible workflow for creating an extension from scratch etc.

Thanks

[wshanks]

Sorry, I haven't done much development since the recent changes, so I'm not the best person to give advice. Zotero 5.0 is basically Firefox with Zotero preinstalled. I'm not sure which version of Firefox or how it deviates from stock Firefox.

Zutilo is an old-style XUL add-on. It is bootstrapped (restartless). Any of the older style XUL add-ons should work with Zotero -- XUL wth a manifest, XUL bootstrapped, or the SDK (you would need to require('chrome') to access Zotero). You don't really need any particular build tools to create these add-ons. They are just zip files. One nice aspect of bootstrapped and SDK add-ons is that they don't require restarting Firefox so you can change things, uninstall the add-on and then install it again. If you look at Zutilo specifically, it is built with GNU make via a Makefile. Besides make and zip, the build process uses pandoc, sed, and some Python scripts.

For a long time, I used Pentadactyl to debug Zutilo because I could run arbitrary javascript and inspect everything in the DOM. In recent versions of Firefox this has been possible iwth the Browser Console which also includes a debugger that can step through add-on code. So it would be worth looking into if it is possible to activate the Browser Console from Zotero 5.0, or to install Zotero 5.0 into Firefox (which version?).

If you are familiar with XUL add-on development and have a project with a limited scope in mind, then go ahead and work on it. Otherwise, i would caution about making a new Zotero add-on at this time. Zotero 5.0 is a transitional release away from Firefox. Really it still uses Firefox internally, but the next major release will likely use Electron. The developers have said the new version will support add-ons but it is too early to know what that will look like and how much work it will be to port from XUL to Electron. If you don't need integration into the Zotero UI, I'd recommend researching the Zotero API. I believe the API can be used to work zotero.org and with Zotero running locally. If you do need to integrate into Zotero, I'd recommend keeping your core code cleanly separated from anything specific to Firefox.

Let me know if there is something else you wanted to know about. You can also ask on the zotero-dev mailing list: https://groups.google.com/forum/#!forum/zotero-dev

[ghost]

Thanks so much for taking the time. It clarified many questions I had and couldn't find the answers to. Saw some mention for Node transition but realized it wasn't 5.x. So I understand that sticking to the online API is probably the safest at this this stage, depending on what the extension your developing is trying to do. I guess Zutillo must integrate with the GUI but would stuff like copying tags or relating items can be done with the web API only?

[wshanks]

Sorry, it looks like I exaggerated the current state of the client API. This post summarizes the current state:

https://forums.zotero.org/discussion/comment/274341#Comment_274341

Basically, a client API is planned but not implemented yet. I know I have seen some references on the zotero-dev mailing list to using the client API, but I don't recall what the context was. Zutilo actually interacts with the API a tiny bit -- the "Copy select links" function generates zotero:// links that select items in the UI when clicked on from an external program if you have the Zotero protocol handler set up (see https://forums.zotero.org/discussion/20040/using-zotero-select-for-external-program-links for some links and discussion). For me, the zotero:// links just work on Linux without any more set up. My guess is that people saying they are doing things with the client API are using the Zotero protocol handler.

In the first link above, Dan says that an initial version of a client API would likely be read-only, so copying tags seems more likely to work. Relating items would require writing to the database so I am not sure about that. The web API does support modifying the database though:

https://www.zotero.org/support/dev/web_api/v3/write_requests

We would need to test what the current protocol handler is capable of (or look at the code or ask on zotero-dev, though Dan would probably just say it isn't officially supported yet). The code is here:

https://github.com/zotero/zotero/blob/9652b9f553eb87acf027caa2fe7cbc0f9e0e856c/components/zotero-protocol-handler.js

It looks like it only has support for selecting items, going fullscreen, receiving url's from the connector, generating reports and timelines, and debugging, so probably not enough to do what you want unless I am missing something.

[ghost]

Thank you. I will start following the mailin list more closely. Very helpful intrdoduction.

[ghost]

specifically I would like to be able to mostly retrieve items, but understanding the limitations of the client before starting is clearly essential.

[wshanks]

If syncing your data to zotero.org and using the web api is okay, that would probably be the most stable option. If you want something to work locally, you probably have to create a XUL add-on. If you do make a XUL add-on with a decent amount of code that you want to be maintainable for more than a year, then you probably want to think carefully about how you use the Zotero and Firefox api's, possibly putting those into wrapper functions so it is easy to change them when Zotero moves to Electron.

[wshanks]

By the way, see: https://www.zotero.org/support/dev/client_coding/building_the_standalone_client i just started using that method to build the client locally so I could turn on remote debugging and connect to it with the Firefox developer tools.

I'll close this now since it seems like your questions were answered.

[diegodlh]

@wshanks, your suggestion to build the Zotero client locally to turn on remote debugging is great! Do you know of a way to wait until the debugger has connected before starting? I would like to debug processes occurring when Zotero starts, and by the time I connect with the debugger they have finished already. Thanks!

[wshanks]

Hmm, I haven't tried to do that myself. I found the place where the debugger is started here:

https://github.com/zotero/zotero/blob/d4f682aa8894275d6fe92fb4bf6bd02bb4158ea7/components/zotero-service.js#L572

If you set a break point and restart with the debugger, it doesn't remember and stop there? If not, I wonder if you could hack in setting a break point in code in that section where it starts the debugger. Also, I see that Firefox itself has a --wait-for-jsdebugger option. Maybe whatever that option does could be added to the Zotero code in that section?

If you want to debug the start up of an addon, you could consider making it bootstrapped (like Zutilo is), so that you can enable it after Zotero has started up.

[diegodlh]

Thanks for your reply!

If you set a break point and restart with the debugger, it doesn't remember and stop there?

Problem is that when I restart Zotero, the debugger gets disconnected and shuts down.

I wonder if you could hack in setting a break point in code in that section where it starts the debugger.

I'm not sure how to do this, since neither setTimeout() nor alert() seem to be defined in the scope of that function, but it seems to be the easier way to go! Thanks

I see that Firefox itself has a --wait-for-jsdebugger option. Maybe whatever that option does could be added to the Zotero code in that section?

Yes, I had thought of that. But I tried it with Firefox itself and it doesn't seem to work. That is, I tried debugging a Firefox v60.9.0 instance --started with --jsdebugger --wait-for-jsdebugger or --wait-for-jsdebugger command line options-- and wanted to connect remotely to it, but it doesn't seem to wait for a connection.

If you want to debug the start up of an addon, you could consider making it bootstrapped (like Zutilo is)

Yes! That's what I'm doing in fact. But I am having an issue where the addon wouldn't load on Zotero startup but rather I have to disable it and enable it again to load (whereas Zutilo does load on startup) and I wanted to figure out what was going on. But I'm sure I'll be able to sort it out somehow else.

Thank you very much for your reply again!