Prob with cite-keys when used with Eratosthenes

RustyRiley commented 10 years ago

got following comment from developer of Eratosthenes app for Android --

"It looks like this is not an eratosthenes problem though. It appears that Better BibTeX is putting spaces in the cite-keys in your bibtex output, which is not valid. It is also putting ' characters in the cite-keys, which I believe is also not valid under the BibTeX standard.

You'll need to contact the dev for better bibtex about this issue. In the mean time, I've fixed your .bib file. It is attached. The next version of eratosthenes will ignore these errors, but you will lose the affected entries. -Matt Matlock P.S. I did find an incidental bug when the last ; is missing from the groupstree, I have fixed this in your file, but I will need to publish an update to the play store to prevent future problems with this."

RustyRiley commented 10 years ago

OK, thanks Russell J. Wilson Dunedin Aotearoa / New Zealand sent via AquaMail for Android

On 2 August 2014 6:45:49 pm AllThatIsTheCase notifications@github.com wrote:

No, I'm going to release the 11th. I'll be offline for a full week starting this afternoon, I don't like to put out new stuff when I'll be unreachable.

Reply to this email directly or view it on GitHub: https://github.com/ZotPlus/zotero-better-bibtex/issues/83#issuecomment-50955785

RustyRiley commented 10 years ago

tried out the new version, and the attempt failed -- don't know if it was due to the "showstopper" -- trying to export the files was a VERY lengthy process, so I left it overnight -- didn't get any message, other than the export "failed" -- but had proceeded a considerable way -- the resulting output was over 2Gb in size (don't know how big it should be, but something over three I think)

best Russell

From: AllThatIsTheCase notifications@github.com To: ZotPlus/zotero-better-bibtex zotero-better-bibtex@noreply.github.com Cc: RustyRiley russwilsonau@yahoo.com.au Sent: Saturday, 2 August 2014 6:45 PM Subject: Re: [zotero-better-bibtex] Prob with cite-keys when used with Eratosthenes (#83)

No, I'm going to release the 11th. I'll be offline for a full week starting this afternoon, I don't like to put out new stuff when I'll be unreachable. — Reply to this email directly or view it on GitHub.

retorquere commented 10 years ago

It could well be -- I'm working on a fix.

retorquere commented 10 years ago

Did you have 'journal' in the cite key format? I've found that the abbreviatior from Zotero is very slow -- about 2 seconds per entry. I am caching the results now, so at least it shouldn't do twice for the same journal, but I'm also looking at a better abbreviator based on the LTWI.

RustyRiley commented 10 years ago

I'll check later

I find I tend to make lots of "excuses" for it's slowness - "slow processor" (AMD a6 6310), "only 4Gig RAM" - - but straight copy paste of bigger directory doesn't take as long. I'm more concerned at "showstoppers"

why it aborted "halfway through". Is new version up now? Haven't checked for couple of days.

On 15 August 2014 5:45:31 am AllThatIsTheCase notifications@github.com wrote:

Did you have 'journal' in the cite key format? I've found that the abbreviatior from Zotero is very slow -- about 2 seconds per entry. I am caching the results now, so at least it shouldn't do twice for the same journal, but I'm also looking at a better abbreviator based on the LTWI.

Reply to this email directly or view it on GitHub: https://github.com/ZotPlus/zotero-better-bibtex/issues/83#issuecomment-52216993

RustyRiley commented 10 years ago

Btw sfaik the usual citekey is just "author year"

On 15 August 2014 5:45:31 am AllThatIsTheCase notifications@github.com wrote:

Did you have 'journal' in the cite key format? I've found that the abbreviatior from Zotero is very slow -- about 2 seconds per entry. I am caching the results now, so at least it shouldn't do twice for the same journal, but I'm also looking at a better abbreviator based on the LTWI.

Reply to this email directly or view it on GitHub: https://github.com/ZotPlus/zotero-better-bibtex/issues/83#issuecomment-52216993

RustyRiley commented 10 years ago

Btw the A6 6310 is like a fast i3, but not even i5

On 15 August 2014 5:45:31 am AllThatIsTheCase notifications@github.com wrote:

Did you have 'journal' in the cite key format? I've found that the abbreviatior from Zotero is very slow -- about 2 seconds per entry. I am caching the results now, so at least it shouldn't do twice for the same journal, but I'm also looking at a better abbreviator based on the LTWI.

Reply to this email directly or view it on GitHub: https://github.com/ZotPlus/zotero-better-bibtex/issues/83#issuecomment-52216993

retorquere commented 10 years ago

The new version is ready, but I'm awaiting confirmation from the user that reported #88 -- probably tomorrow. The new version is available at https://dl.dropboxusercontent.com/u/5300806/zotero-better-bibtex-0.6.4.xpi if you're curious. "authoryear" exports should be fast -- can you create a Zotero TestCase export of the failing BibTeX export and upload it to http://dbinbox.com/allthatisthecase ?

retorquere commented 10 years ago

The #88 showstopper is unlikely to be related to your midway export failure, so I'd like to look into that separately.

RustyRiley commented 10 years ago

file sent to dbinbox,

but problems experienced with files attached,

BUT as importantly, with details only

there's something still wrong, obviously, with the original Zotero database that I've never gotten around to "fixing"

From: AllThatIsTheCase notifications@github.com To: ZotPlus/zotero-better-bibtex zotero-better-bibtex@noreply.github.com Cc: RustyRiley russwilsonau@yahoo.com.au Sent: Friday, 15 August 2014 6:57 AM Subject: Re: [zotero-better-bibtex] Prob with cite-keys when used with Eratosthenes (#83)

The new version is ready, but I'm awaiting confirmation from the user that reported #88 -- probably tomorrow. The new version is available at https://dl.dropboxusercontent.com/u/5300806/zotero-better-bibtex-0.6.4.xpi if you're curious. "authoryear" exports should be fast -- can you create a Zotero TestCase export of the failing BibTeX export and upload it to http://dbinbox.com/allthatisthecase ? — Reply to this email directly or view it on GitHub.

RustyRiley commented 10 years ago

mind you, having said that, checking with Zotero's own database integrity checker says there are no problems with database

still the difference in formats across items -- e.g., PDF files with no other details -- could well create some problems?

From: Russell Wilson russwilsonau@yahoo.com.au To: ZotPlus/zotero-better-bibtex reply@reply.github.com Sent: Friday, 15 August 2014 11:39 AM Subject: Re: [zotero-better-bibtex] Prob with cite-keys when used with Eratosthenes (#83)

file sent to dbinbox,

but problems experienced with files attached,

BUT as importantly, with details only

there's something still wrong, obviously, with the original Zotero database that I've never gotten around to "fixing"

From: AllThatIsTheCase notifications@github.com To: ZotPlus/zotero-better-bibtex zotero-better-bibtex@noreply.github.com Cc: RustyRiley russwilsonau@yahoo.com.au Sent: Friday, 15 August 2014 6:57 AM Subject: Re: [zotero-better-bibtex] Prob with cite-keys when used with Eratosthenes (#83)

The new version is ready, but I'm awaiting confirmation from the user that reported #88 -- probably tomorrow. The new version is available at https://dl.dropboxusercontent.com/u/5300806/zotero-better-bibtex-0.6.4.xpi if you're curious. "authoryear" exports should be fast -- can you create a Zotero TestCase export of the failing BibTeX export and upload it to http://dbinbox.com/allthatisthecase ? — Reply to this email directly or view it on GitHub.

retorquere commented 10 years ago

The export in dbinbox was 7.23 MB; it imported & exported cleanly to a 4.16 MB bib file in around a minute, so this isn't capturing the problem you saw. Could you please do the following:

Go into the Zotero "gear" menu, open "preferences", go to the "advanced" tab, and enable debug output logging.
Export your library using BBT and have it generate the error
Go back into the "advanced" tab and click "disable", then "view output"
In the resulting window, right-click the text, select "select all", then right-click again and select "copy"
Open a text editor (Notepad will do, but not Word for example) and paste the log
Save and upload to dbinbox

RustyRiley commented 10 years ago

file sent, thanks for persisting with this

From: AllThatIsTheCase notifications@github.com To: ZotPlus/zotero-better-bibtex zotero-better-bibtex@noreply.github.com Cc: RustyRiley russwilsonau@yahoo.com.au Sent: Friday, 15 August 2014 6:57 AM Subject: Re: [zotero-better-bibtex] Prob with cite-keys when used with Eratosthenes (#83)

The new version is ready, but I'm awaiting confirmation from the user that reported #88 -- probably tomorrow. The new version is available at https://dl.dropboxusercontent.com/u/5300806/zotero-better-bibtex-0.6.4.xpi if you're curious. "authoryear" exports should be fast -- can you create a Zotero TestCase export of the failing BibTeX export and upload it to http://dbinbox.com/allthatisthecase ? — Reply to this email directly or view it on GitHub.

RustyRiley commented 10 years ago

just as a matter of curiousity, what kind of system re yu using -- it takes me 6 minutes to get an export (unsuccessful) of just the item details -- no notes, or files -- not just the one minute you mentioned

From: AllThatIsTheCase notifications@github.com To: ZotPlus/zotero-better-bibtex zotero-better-bibtex@noreply.github.com Cc: RustyRiley russwilsonau@yahoo.com.au Sent: Friday, 15 August 2014 5:27 PM Subject: Re: [zotero-better-bibtex] Prob with cite-keys when used with Eratosthenes (#83)

The export in dbinbox was 7.23 MB; it imported & exported cleanly to a 4.16 MB bib file in around a minute, so this isn't capturing the problem you saw. Could you please do the following:

Go into the Zotero "gear" menu, open "preferences", go to the "advanced" tab, and enable debug output logging.

Export your library using BBT and have it generate the error

Go back into the "advanced" tab and click "disable", then "view output"

In the resulting window, right-click the text, select "select all", then right-click again and select "copy"

Open a text editor (Notepad will do, but not Word for example) and paste the log

Save and upload to dbinbox — Reply to this email directly or view it on GitHub.

retorquere commented 10 years ago

I should thank you for persisting. This debug log was created with 0.6.0 or 0.6.1. The pending release has much better handling of the keys. I'll cut a release tonight, please try with that.

The system that I'm using BBT on is a pretty well-specced laptop; an i7@2.67Ghz with 8GB RAM. My dev system is much more modest, an Atom D525 @ 1.80GHz with 5Gb RAM. but even there it only takes a few minutes. Still looking into that.

retorquere commented 10 years ago

You could try with https://dl.dropboxusercontent.com/u/5300806/zotero-better-bibtex-0.6.5.xpi, or wait out the release of tonight

RustyRiley commented 10 years ago

Thanks, it's pretty late here, I'll await the release

Some people have "all the luck" -- i7 with 8Gb (and graphics card), for what? the cost of a new battery? ;-D

experimenting with trying to "resurrect" an old netbook -- Atom N2600, with only a Gig of RAM, wondering if I can get Linux to run on it -- learning a lot

From: AllThatIsTheCase notifications@github.com To: ZotPlus/zotero-better-bibtex zotero-better-bibtex@noreply.github.com Cc: RustyRiley russwilsonau@yahoo.com.au Sent: Friday, 15 August 2014 11:32 PM Subject: Re: [zotero-better-bibtex] Prob with cite-keys when used with Eratosthenes (#83)

You could try with https://dl.dropboxusercontent.com/u/5300806/zotero-better-bibtex-0.6.5.xpi, or wait out the release of tonight — Reply to this email directly or view it on GitHub.

retorquere commented 10 years ago

No, the charging circuit is broken. It won't charge at all, the keyboard is flaky, and the screen is stained from the inside. But nothing that a docking station, a USB mouse & keyboard, and two external monitors can't fix, most of which I had lying about anyhow. The graphics card is appropriate for a laptop -- nothing too fancy. But it's a good desktop replacement for a non-gamer like myself. My son has very little interest in the thing.

My Atom system is a linux system. I am more comfortable with developing on Linux, I'm totally a command-line/scripting geek. The atom system did not in fact finish in minutes, it failed in minutes, I'm running a fixed bulk test using your library now.

retorquere commented 10 years ago

I tried on the atom system, and that is ridiculously slow. I'll try to make performance improvements.

RustyRiley commented 10 years ago

I used to do everything on the Atom notebook, but recently have used the A6 6310 - - still ridiculously slow - - even creating new entries in Z is painful. Looking at activities in Task Manager, size of Ff often more than doubles, when creating new entry, before returning to "normal"; Windows search indexer also seems to go crazy with each new item - - in the past, the Atom used Win7; now the A6 6310 uses Win8. Just "idle observations",
don't know enough to understand all this; or even if these are related to the slowness of the export, or if the "solution" is faster processor or more RAM for the A6 system.

always thought the Atom struggled with either Win7, or my use of multitasking - - Win7 should be pretty quick though. So not sure if changing Atom to Linux Mint Xfce (lower overhead?) would speed it up that much. Best Russell J. Wilson Dunedin, Aotearoa / New Zealand sent via AquaMail for Android

On 16 August 2014 6:43:35 am AllThatIsTheCase notifications@github.com wrote:

I tried on the atom system, and that is ridiculously slow. I'll try to make performance improvements.

Reply to this email directly or view it on GitHub: https://github.com/ZotPlus/zotero-better-bibtex/issues/83#issuecomment-52343002

retorquere commented 10 years ago

After more testing, the export seems fairly reasonable. Could be better, of course, but I'm averaging 22 items/second on the Atom system. The import of your library takes 15 minutes on that system; the export a tad over 2.

On Fri, Aug 15, 2014 at 9:58 PM, RustyRiley notifications@github.com wrote:

I used to do everything on the Atom notebook, but recently have used the A6 6310 - - still ridiculously slow - - even creating new entries in Z is painful. Looking at activities in Task Manager, size of Ff often more than doubles, when creating new entry, before returning to "normal"; Windows search indexer also seems to go crazy with each new item - - in the past, the Atom used Win7; now the A6 6310 uses Win8.

You don't want the windows indexer on that system. If you need search on such a constrained machine, and you are content with filename-only search, get everything search from http://www.voidtools.com/.

Just "idle observations", don't know enough to understand all this; or even if these are related to the slowness of the export, or if the "solution" is faster processor or more RAM for the A6 system.

That should be easy to verify: in your task manager, is the processor load consistently over 50%? You want a faster processor. Is "physical memory usage history" consistently pegged close to 100? You want more RAM. These are not mutually exclusive -- you may need more ram and a faster processor. But the A6 ought to beat my Atom hands down ( http://www.cpubenchmark.net/compare.php?cmp[]=2290&cmp[]=611), and you have an equal amount of RAM, so something else may be the culprit. This system I'm testing on differs from yours in 3 aspects:

It's headless (no graphics, I'm testing using a graphics-card emulator)
It's running Linux (Ubuntu)
It's a desktop system, running 7200rpm disks

always thought the Atom struggled with either Win7, or my use of

multitasking - - Win7 should be pretty quick though. So not sure if changing Atom to Linux Mint Xfce (lower overhead?) would speed it up that much.

I think Windows 7 was pretty well optimized, so I don't think there are inherent speed benefits between the two, but with some work you can usually safely dress down a linux system much further than you can a windows system. Xfce is a good choice I think. Slimming down your windows install with tools such as http://www.makeuseof.com/tag/tiny-ssd-slim-windows-7-minimal-installs/ could be another option, and you might want to benchmark your disk, as on netbooks these are usually very slow because they're frugal with power (and cheap). Replacing it with an SSD (not too expensive these days) could make all the difference. On my previous netbook, the slow drive was the real performance killer.

RustyRiley commented 10 years ago

I'll let you know - - big difference between your Atom system and mine - - you have 5? gig RAM - - I have only 1 (one!) Gig , and you have a faster processor in yours, too - - see results on CpuBoss -- I'll let you know - - I've retained a copy of the same database

Russell J. Wilson Dunedin, Aotearoa / New Zealand sent via AquaMail for Android

On 16 August 2014 8:50:22 am AllThatIsTheCase notifications@github.com wrote:

After more testing, the export seems fairly reasonable. Could be better, of course, but I'm averaging 22 items/second on the Atom system. The import of your library takes 15 minutes on that system; the export a tad over 2.

On Fri, Aug 15, 2014 at 9:58 PM, RustyRiley notifications@github.com wrote:

I used to do everything on the Atom notebook, but recently have used the A6 6310 - - still ridiculously slow - - even creating new entries in Z is painful. Looking at activities in Task Manager, size of Ff often more than doubles, when creating new entry, before returning to "normal"; Windows search indexer also seems to go crazy with each new item - - in the past, the Atom used Win7; now the A6 6310 uses Win8.

You don't want the windows indexer on that system. If you need search on such a constrained machine, and you are content with filename-only search, get everything search from http://www.voidtools.com/.

Just "idle observations", don't know enough to understand all this; or even if these are related to the slowness of the export, or if the "solution" is faster processor or more RAM for the A6 system.

That should be easy to verify: in your task manager, is the processor load consistently over 50%? You want a faster processor. Is "physical memory usage history" consistently pegged close to 100? You want more RAM. These are not mutually exclusive -- you may need more ram and a faster processor. But the A6 ought to beat my Atom hands down ( http://www.cpubenchmark.net/compare.php?cmp[]=2290&cmp[]=611), and you have an equal amount of RAM, so something else may be the culprit. This system I'm testing on differs from yours in 3 aspects:

It's headless (no graphics, I'm testing using a graphics-card emulator)

It's running Linux (Ubuntu)

It's a desktop system, running 7200rpm disks

always thought the Atom struggled with either Win7, or my use of

multitasking - - Win7 should be pretty quick though. So not sure if changing Atom to Linux Mint Xfce (lower overhead?) would speed it up that much.

I think Windows 7 was pretty well optimized, so I don't think there are inherent speed benefits between the two, but with some work you can usually safely dress down a linux system much further than you can a windows system. Xfce is a good choice I think. Slimming down your windows install with tools such as http://www.makeuseof.com/tag/tiny-ssd-slim-windows-7-minimal-installs/ could be another option, and you might want to benchmark your disk, as on netbooks these are usually very slow because they're frugal with power (and cheap). Replacing it with an SSD (not too expensive these days) could make all the difference. On my previous netbook, the slow drive was the real performance killer.

Reply to this email directly or view it on GitHub: https://github.com/ZotPlus/zotero-better-bibtex/issues/83#issuecomment-52356169

RustyRiley commented 10 years ago

You were also going to set up a new version, which I've heard nothing about

Russell J. Wilson Dunedin, Aotearoa / New Zealand sent via AquaMail for Android

On 16 August 2014 8:50:22 am AllThatIsTheCase notifications@github.com wrote:

After more testing, the export seems fairly reasonable. Could be better, of course, but I'm averaging 22 items/second on the Atom system. The import of your library takes 15 minutes on that system; the export a tad over 2.

On Fri, Aug 15, 2014 at 9:58 PM, RustyRiley notifications@github.com wrote:

I used to do everything on the Atom notebook, but recently have used the A6 6310 - - still ridiculously slow - - even creating new entries in Z is painful. Looking at activities in Task Manager, size of Ff often more than doubles, when creating new entry, before returning to "normal"; Windows search indexer also seems to go crazy with each new item - - in the past, the Atom used Win7; now the A6 6310 uses Win8.

You don't want the windows indexer on that system. If you need search on such a constrained machine, and you are content with filename-only search, get everything search from http://www.voidtools.com/.

Just "idle observations", don't know enough to understand all this; or even if these are related to the slowness of the export, or if the "solution" is faster processor or more RAM for the A6 system.

That should be easy to verify: in your task manager, is the processor load consistently over 50%? You want a faster processor. Is "physical memory usage history" consistently pegged close to 100? You want more RAM. These are not mutually exclusive -- you may need more ram and a faster processor. But the A6 ought to beat my Atom hands down ( http://www.cpubenchmark.net/compare.php?cmp[]=2290&cmp[]=611), and you have an equal amount of RAM, so something else may be the culprit. This system I'm testing on differs from yours in 3 aspects:

It's headless (no graphics, I'm testing using a graphics-card emulator)

It's running Linux (Ubuntu)

It's a desktop system, running 7200rpm disks

always thought the Atom struggled with either Win7, or my use of

multitasking - - Win7 should be pretty quick though. So not sure if changing Atom to Linux Mint Xfce (lower overhead?) would speed it up that much.

I think Windows 7 was pretty well optimized, so I don't think there are inherent speed benefits between the two, but with some work you can usually safely dress down a linux system much further than you can a windows system. Xfce is a good choice I think. Slimming down your windows install with tools such as http://www.makeuseof.com/tag/tiny-ssd-slim-windows-7-minimal-installs/ could be another option, and you might want to benchmark your disk, as on netbooks these are usually very slow because they're frugal with power (and cheap). Replacing it with an SSD (not too expensive these days) could make all the difference. On my previous netbook, the slow drive was the real performance killer.

Reply to this email directly or view it on GitHub: https://github.com/ZotPlus/zotero-better-bibtex/issues/83#issuecomment-52356169

retorquere commented 10 years ago

It's already up. Firefox should auto-update as usual.

RustyRiley commented 10 years ago

Thanks I'll check it later - - I'll be happy with two hours, let alone two minutes Russell J. Wilson Dunedin, Aotearoa / New Zealand sent via AquaMail for Android

On 16 August 2014 9:53:31 am AllThatIsTheCase notifications@github.com wrote:

It's already up. Firefox should auto-update as usual.

Reply to this email directly or view it on GitHub: https://github.com/ZotPlus/zotero-better-bibtex/issues/83#issuecomment-52362151

RustyRiley commented 10 years ago

btw -- unlike some extensions, that advise they've automatically updated themselves, I've never had that with yours, I've always needed to ask "Check for Updates"

From: AllThatIsTheCase notifications@github.com To: ZotPlus/zotero-better-bibtex zotero-better-bibtex@noreply.github.com Cc: RustyRiley russwilsonau@yahoo.com.au Sent: Saturday, 16 August 2014 9:53 AM Subject: Re: [zotero-better-bibtex] Prob with cite-keys when used with Eratosthenes (#83)

It's already up. Firefox should auto-update as usual. — Reply to this email directly or view it on GitHub.

retorquere commented 10 years ago

That is odd. I'll look into that.

retorquere commented 10 years ago

I think I have found the problem and fixed it, but I commonly release more often than the check interval looks (see http://zotplus.github.io/)

RustyRiley commented 10 years ago

Thanks - - per second update IS a little bit too much 😁

What else would the computer be doing

😂

Russell J. Wilson Dunedin Aotearoa / New Zealand sent via MailDroid Pro

-----Original Message----- From: AllThatIsTheCase notifications@github.com To: ZotPlus/zotero-better-bibtex zotero-better-bibtex@noreply.github.com Cc: RustyRiley russwilsonau@yahoo.com.au Sent: Mon, 18 Aug 2014 10:42 am Subject: Re: [zotero-better-bibtex] Prob with cite-keys when used with Eratosthenes (#83)

I think I have found the problem and fixed it, but I commonly release more often than the check interval looks (see http://zotplus.github.io/)

Reply to this email directly or view it on GitHub: https://github.com/ZotPlus/zotero-better-bibtex/issues/83#issuecomment-52437944

retorquere / zotero-better-bibtex

Prob with cite-keys when used with Eratosthenes #83