How to restore previous back-up file into new version of Qiqqa (v79 -> v82)?

[Mocabl3nd]

Hi! I recently upgraded my Qiqqa from v79 into v82, but I can't seem to import my library from the previous version. It keeps on crashing. I need to restore my library since I have many pdfs with notes too and I am in the middle of writing my paper. :( I also moved to a new computer. Is there a way for me to fix this?

Here's the images of the steps that I did to restore my library, but failed:

1. I opened qiqqa app, clicked on the restore button and then yes
2. Chose the back-up file
3. Pressed A for Always
4. Clicked start Qiqqa (Esc)
5. Then this happened:
6. And qiqqa app stuck in the launching mode:

Please let me know any additional details that might help to fix this issue. Thank you so much!

[GerHobbelt]

Whoa! 😨

Ok, in rapid-fire what I see here and thoughts as-I-write:

S:\Users\.... 🤔

First screenshot (BTW, thanks for documenting your problem with such detail, thanks! 👍 ) mentions a "Qiqqa base directory" where all its libraries are kept on the "S:" drive.

This is unusual so I have to ask: this is a local disk, right? I have to ask to make absolutely sure. Because if it isn't... then we have another layer of potential problem causes (using SQLite databases, etc. right on top of network storage -- should work, but mark the should there, as it comes with its own set of requirements and problems; TL;DR: let's not go there, so I hope your S:\ drive is as local as is my own "W:\" drive, which is my second SSD in the development machine: in my case, my dev box has three local drives: C:, D: and W:. The rest of 'em are network storage and USB bays/drives. I'm running Qiqqa off D:\Qiqqa\base myself.)

Access Denied: C:\Users 🤔

Don't know where that one came from. 😮

Luckily there's a stacktrace included, which tells us it's Qiqqa telling us it happens during the startup phase when it has discovered there's no default "Guest" library present and attempts to create one.

My reaction here is 😮 because:

• my expectation is Qiqqa would attempt some Guess lib creation in the S:\Users\Policina\... directory tree instead of anywhere else: after all the directory reported in that startup screen (first screenshot) is the base for all Qiqqa libraries, so I am very curious how this could happen!
• second, assuming Qiqqa went wrong somehow there with the base path, then the default-if-all-else-fails-horribly is C:\Users\\Quantisle\Qiqqa (forgive any typo in this path as type it off the top of my head) and *certainly not in C:\Users\ itself as that one is for machine-wide purposes, rather than user-specific*.

Command line: S:\Program Files\....

OK, so it looks from this distance like your S:\ drive is local for you have installed Qiqqa there. Shouldn't be an issue, but good to note in case.

Attack Plan

Assumption: you have your backups. Good.

Step 1: see if there's anything in the S:\Users\Polinica\... directory tree

That's the place Qiqqa has been told (via a Windows registry key) to look for its libraries.

What we're looking for are files called "Qiqqa.library". See example screenshot of my test environment on D:\Qiqqa\base:

The other files in there are far less relevant for initial diagnosis: it's the Qiqqa.library file that contains the SQLite database carrying all your library's metadata and if that one isn't present, Qiqqa doesn't deem that directory a 'library folder'.

Step 2: just in case: any Qiqqa.library files anywhere?

Since your situation is pretty darn FUBAR, we need to check the very basics and discover where we have data lying around:

See if you can locate any other "Qiqqa.library" files anywhere. Of course prime candidates for this scan are the S:\Users and C:\Users directory trees. Please write down where you have found Qiqqa.library files; if those locations also have a 'documents' directory alongside the "Qiqqa.library" file, we have at least some libraries we can more-or-less recover manually with reasonable probability: the 'documents' directory tree should then contain the PDF documents. Not important right now, but so you get a bit of an idea what that might look like, here's a screenshot of a *.pdf search in the directory where I have a Qiqqa.library file:

Step 3: set up a place where we can recover your goods.

As your current state as reported is way outside the expected, first rule of engagement is not touching the current data files until we have full control over the situation and matching expectations.

I am treading very very carefully here, hence the extreme caution which would otherwise not be required.

This means: pick a spot on your local drive(s) where you have plenty space and easily copy any data to as we go. To prevent any potential hazard, that would preferably be a directory with a short name and no spaces, e.g. S:\Qiqqa\base

Since you're running build 29227, IIRC you can already use an undocumented feature: base directory override per command line.

If you're fine with using a commandline interface, such as CMD, PowerShell or bash, then it's as simple as the screenshot below, but in case you're not comfortable with that I'll write a different approach in the next message so this one can get out the door.

Note in the screenshot that my Qiqqa is installed in C:\ProgramFiles(x86)\Qiqqa and the shell used is bash (from Git For Windows, hence the / forward slashes in the path and /d/ instead of D:\)

(to be continued...)

[GerHobbelt]

@Mocabl3nd : there's a new (test) release published at https://github.com/GerHobbelt/qiqqa-open-source/releases/tag/v83.0.7649.30836 ; see description there. You can simply installed it over your existing Qiqqa; if you want to revert to another Qiqqa version, you can install that version over the new one without trouble.

About your particular issue:

Since we (I) intended to play it very safe, we should create a new (empty) base directory someplace and then you can point the new qiqqa to that directory AFTER having copied your libraries to the new directory -- that bit I'll describe later, but first the new startup screen of latest Qiqqa https://github.com/GerHobbelt/qiqqa-open-source/releases/tag/v83.0.7649.30836 :

Note that the startup screen has a new "Change this path" button, which is much easier to use than some commandline hacking: point Qiqqa at the new base directory and it'll look there from that moment on; the path will be remembered and recalled when you restart Qiqqa later, so only need to point at the new spot once.

Now about copying your directories

... so we don't work on the originals yet.

Every Qiqqa library has a set files, one of the most important ones called Qiqqa.library: *every library has its own Qiqqa.library metadata database.

Now you can easily copy a library over to the new place by **copying the ENTIRE folder which contains the Qiqqa.library file and documents directory and several other files (like the set in the screenshot in Step 1 in previous comment) to the new base directory. Of course, when you have many libraries, each library has its own folder (with the Qiqqa.library in it, etc.etc.) and you should each such folder to the new place.

Qiqqa (since v80, but improved in the v82/v83 series) performs auto-discovery, so anything that's anywhere near viable as a library in the eyes of Qiqqa is discovered by Qiqqa and added to the list.

Now if there's really a serious mess (which happens very rarely but just in case; I have to guess a little here), say when you find Qiqqa.library-containing folders anywhere and those folders happen to have the same name as folders you already copied to the new place, you can rename the folder which contains the Qiqqa.library file so any copy action will not yak about 'overwriting' any stuff.

There's MAY be some cryptic folder names in there when you have so old Commercial Qiqqa cloud libraries and Qiqqa went haywire somewhere, but this is surely survivable: here's a screenshot of my own Qiqqa base directory -- note the Guest library in there, which everyone has one of, plus loads of cryptic-named folder names: those are the result of my own folder-rename actions as I collected all crashed Commercial Qiqqa library instances from 2016/2017? till 2019 (and I had a lot of crashes back then!):

which isn't the entire list. Anyway, here's what you then get in Qiqqa:

and that's just a part too (check the scrollbar at right: it's only a small part, but you can see the Guest1, Guest2, etc. directories = libraries show up here.

The key take-away here

You don't have to "convert" any library; Qiqqa v82/83 does this automatically for you.

The other thing to keep in mind specifically about Commercial Qiqqa Cloud libraries is: Commercial Qiqqa always kept a local copy of those: 'Sync' ensured that the cloud stored copy matches your local copy.

Now what Open Source Qiqqa does it simply 'discover' your local copy of such libraries (as they sit in the same base directory as the Guest library and any old 'Intranet Libraries' you might have created back then) and list those.

What the latest release mentioned at the top of this message does extra vs the older releases of Open Source Qiqqa is this:

v83 'ignores' the commercial contract-based flags about being permitted to edit/write/do-whatever with these libraries and (since the commercial cloud storage is inaccessible to the open source Qiqqa versions) re-label any such library as a "Intranet Library" (possibly while 'augmenting' the library name with "Legacy bla bla' (as seen in the screenshot above) and thus finally allowing you everything like a 'paid in full' Commercial Qiqqa copy would have -- except of course without cloud access.

So, in case you believe your local copy is out of sync, you install commercial v79 to log into the cloud store (for as long as that's still available!!), do a 'Sync' so your local copy of the cloud libs are up-to-date, then install v83 over the existing Qiqqa install and let it auto-discover those as 'Intranet Libraries'.

Since you reported serious problems, I added the steps above to ensure that your local copy is not touched/modified by Qiqqa by you copying all libraries to a new location on your local disk (so you now have each library in the new place and somewhere else (old spot) on your disk) and then pointing Qiqqa to the new location, so it can go and 'auto discover' everything there and you can then check if everything is al right.

What if I ran Qiqqa and I happen to copy yet another library into the base directory?

OK, can happen. Restart Qiqqa and it will auto-discover the library during the next start.

What if it doesn't?!

That's where the copying of this stuff comes in handy: in the new location, look for any files named Qiqqa.known_web_libraries and delete or rename those. This will ensure that Qiqqa cannot recall which libraries it has found previously and thus will force it to perform a full scan of the base directory, listing every library in there that it can find.

WARNING: deleting that 'known_web_libraries' file will also lose you your library names, so use this only in case you know you want the rescan to happen for sure. Thus renaming the file (to, say, Qiqqa.known_web_libraries.BAK) is more pudent than simply deleting it, but sometimes we want to live dangerously. 😉

---

Hope this helps you fix your situation; please report what's happening and we'll see how we can resolve this in a satisfactory way.

Feedback appreciated.

[Mocabl3nd]

Hi Ger!

Thanks for the response! I really appreciate for helping usout whenever we encountered an issue and find solutions to the problem.

 Let me answer some ofyour questions

 

Q1: Is the S:\ drive my local drive disk?

A: Nope, it’s another ssd drive for storing files and whatnot.C:\ is my local disk.  

Q2: Any Qiqqa.library files

A: Is this the once you’re referring to? I have saved them inmy GDrive.

  Here are some of the pdfs:

Update 1:

I uninstalled the Qiqqa app and reinstalled it in my C:/ drive,only for find the same pop-up dialog with the same issue. This made me confused.So I uninstalled it again AND removed all files related to Qiqqa that might notbeen removed in the uninstallation (like the .pf files, located in C:\Windows\Prefetch).

After that, I installed it again for the 3rdtime, and it worked! The pop-up dialog didn’t show anymore.

It only happens again when I try to “restore” the previousback-up files (the ones shown in the image in the previous msg.)

This leads me that I can no longer use the previous back-upfiles. ☹ Now, all I can do is import the pdf files and do theBibtex sniffer, which is grueling, since I frequently cannot bypass the “Please verifythat you are not a robot” warning.  

However, I am still hoping that I can use the back-up filesthat I have. Do u think is there a fix for this?

 

Update 2:

After reading your 2nd email, I decided to followthe instructions and install the latest unpublished version (v83.0.7649.30836).

I changed the path to the desired location.

These are the changes that occurred.

I was able to retrieve the libraries but they’re empty! Nodocuments in the libraries. I don’t know if I missed something. But I stillhave all of the pdfs tho.

What can you say about these changes? I would love to knowyour feedback and fix the issue too. :

 

Thanks!

 

On Wednesday, January 13, 2021, 09:55:00 AM GMT+8, Ger Hobbelt <notifications@github.com> wrote:  

@Mocabl3nd : there's a new (test) release published at https://github.com/GerHobbelt/qiqqa-open-source/releases/tag/v83.0.7649.30836 ; see description there. You can simply installed it over your existing Qiqqa; if you want to revert to another Qiqqa version, you can install that version over the new one without trouble.

About your particular issue:

Since we (I) intended to play it very safe, we should create a new (empty) base directory someplace and then you can point the new qiqqa to that directory AFTER having copied your libraries to the new directory -- that bit I'll describe later, but first the new startup screen of latest Qiqqa https://github.com/GerHobbelt/qiqqa-open-source/releases/tag/v83.0.7649.30836 :

Note that the startup screen has a new "Change this path" button, which is much easier to use than some commandline hacking: point Qiqqa at the new base directory and it'll look there from that moment on; the path will be remembered and recalled when you restart Qiqqa later, so only need to point at the new spot once.

Now about copying your directories

... so we don't work on the originals yet.

Every Qiqqa library has a set files, one of the most important ones called Qiqqa.library: *every library has its own Qiqqa.library metadata database.

Now you can easily copy a library over to the new place by **copying the ENTIRE folder which contains the Qiqqa.library file and documents directory and several other files (like the set in the screenshot in Step 1 in previous comment) to the new base directory. Of course, when you have many libraries, each library has its own folder (with the Qiqqa.library in it, etc.etc.) and you should each such folder to the new place.

Qiqqa (since v80, but improved in the v82/v83 series) performs auto-discovery, so anything that's anywhere near viable as a library in the eyes of Qiqqa is discovered by Qiqqa and added to the list.

Now if there's really a serious mess (which happens very rarely but just in case; I have to guess a little here), say when you find Qiqqa.library-containing folders anywhere and those folders happen to have the same name as folders you already copied to the new place, you can rename the folder which contains the Qiqqa.library file so any copy action will not yak about 'overwriting' any stuff.

There's MAY be some cryptic folder names in there when you have so old Commercial Qiqqa cloud libraries and Qiqqa went haywire somewhere, but this is surely survivable: here's a screenshot of my own Qiqqa base directory -- note the Guest library in there, which everyone has one of, plus loads of cryptic-named folder names: those are the result of my own folder-rename actions as I collected all crashed Commercial Qiqqa library instances from 2016/2017? till 2019 (and I had a lot of crashes back then!):

which isn't the entire list. Anyway, here's what you then get in Qiqqa:

and that's just a part too (check the scrollbar at right: it's only a small part, but you can see the Guest1, Guest2, etc. directories = libraries show up here.

The key take-away here

You don't have to "convert" any library; Qiqqa v82/83 does this automatically for you.

The other thing to keep in mind specifically about Commercial Qiqqa Cloud libraries is: Commercial Qiqqa always kept a local copy of those: 'Sync' ensured that the cloud stored copy matches your local copy.

Now what Open Source Qiqqa does it simply 'discover' your local copy of such libraries (as they sit in the same base directory as the Guest library and any old 'Intranet Libraries' you might have created back then) and list those.

What the latest release mentioned at the top of this message does extra vs the older releases of Open Source Qiqqa is this:

v83 'ignores' the commercial contract-based flags about being permitted to edit/write/do-whatever with these libraries and (since the commercial cloud storage is inaccessible to the open source Qiqqa versions) re-label any such library as a "Intranet Library" (possibly while 'augmenting' the library name with "Legacy bla bla' (as seen in the screenshot above) and thus finally allowing you everything like a 'paid in full' Commercial Qiqqa copy would have -- except of course without cloud access.

So, in case you believe your local copy is out of sync, you install commercial v79 to log into the cloud store (for as long as that's still available!!), do a 'Sync' so your local copy of the cloud libs are up-to-date, then install v83 over the existing Qiqqa install and let it auto-discover those as 'Intranet Libraries'.

Since you reported serious problems, I added the steps above to ensure that your local copy is not touched/modified by Qiqqa by you copying all libraries to a new location on your local disk (so you now have each library in the new place and somewhere else (old spot) on your disk) and then pointing Qiqqa to the new location, so it can go and 'auto discover' everything there and you can then check if everything is al right.

What if I ran Qiqqa and I happen to copy yet another library into the base directory?

OK, can happen. Restart Qiqqa and it will auto-discover the library during the next start.

What if it doesn't?!

That's where the copying of this stuff comes in handy: in the new location, look for any files named Qiqqa.known_web_libraries and delete or rename those. This will ensure that Qiqqa cannot recall which libraries it has found previously and thus will force it to perform a full scan of the base directory, listing every library in there that it can find.

WARNING: deleting that 'known_web_libraries' file will also lose you your library names, so use this only in case you know you want the rescan to happen for sure. Thus renaming the file (to, say, Qiqqa.known_web_libraries.BAK) is more pudent than simply deleting it, but sometimes we want to live dangerously. 😉

Hope this helps you fix your situation; please report what's happening and we'll see how we can resolve this in a satisfactory way.

Feedback appreciated.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[GerHobbelt]

Hi,

[Edit: cut off the replied chunk at the bottom; github issue tracker doesn't show it, but clutters the search. Spellchecking the red-wave typo flags. Retransmitted this "cleaned up" response. In a bit of hurry there.]

See also at github in the issue tracker here: https://github.com/jimmejardine/qiqqa-open-source/issues/288#issuecomment-759260205-permalink

Somehow I think you included images with your email response, but they didn't come through. I also looked up your email in my Inbox and they weren't visible there either; don't know what happened. From personal experience: it might be easier to paste/attach images when you write thee response in the github issue tracker as a comment to the issue (as linked above; when you're logged in into github, you can scroll down and edit/enter a comment in the form at the bottom of the page)

Re your trouble and Update 1 + 2:

While there's plenty in there that's currently still inexplicable for me, it sure sounds like trouble. I could try to reproduce the issue here locally in the developer machine if you are willing / can send me the backup file you mentioned as one causing trouble: if you're okay with sharing the data in there with me, you could share the backup archive via, for example, a link to a Google Drive entry or Dropbox file or other means (chances are high it's too large for attachment to an email so a download link has a far better chance of working there; see also other issues in the tracker where folks have bundled Qiqqa logfiles in ZIP archive files and sent me download links to those: we're used to this approach)

If you could ZIP your log files and send them that way, that would be really helpful too, as understanding and observing your problems helps me greatly in finding ways to fix this and get you back where you should be, with a complete library. If you need a bit of guidance how to do that, give a holler and I'll help. (Have to chose between writing docu vs. working on software right now, choosing software as I'm in the middle of migrating the PDF viewer core away from the old SORAX lib. ;-) )

As I have some trouble following due to the missing screenshots there's a couple of questions, which we'll go through the next couple of days if you're okay with that. I may be pestering you with new test releases (which I'll call "unpublished" as they are not announced on the qiqqa mailing list for Qiqqa releases -- that mailing list is solely meant for folks to hear about Qiqqa software releases for general use; as I mentioned to a few others, Qiqqa "unpublished" releases are meant to check on specific issues reported by one or more people; they are meant to be installed, live a short life while folks see what happens and report back, and then they can decide to keep them for now or roll back to any previous version by installing that version right over the existing one as usual when upgrading/downgrading Qiqqa.

Plan of Attack

1: me getting a clearer picture of what's happening in detail. Logfiles help, more is better than less. (Simply zip the entire log directory and send a download link to that; I've no trouble "wading through". As I recently wrote to someone else: the logfiles are a huge help generally as there's often plenty things going on that make me go "ho-hum, why does Qiqqa do THAT?!" even when it may only be sideways related to the observed issue. Most problems are compound ones and lifelong experience has shown me that finding something that appears unrelated and is somehow obscure/hidden, is a hint to other trouble far more often than you'ld expect.

2: getting your problem to become reproducible on the development machine, or at least close enough that things go b0rk b0rk b0rk. Once I get it to reproduce, it becomes much easier for me to diagnose. For pessimists: much less difficult :-)

Hence getting a copy of your backup(s), which seem to at least trigger the problems, is helping. Re security: they don't get published; that stuff remains on my dev box only. If there's interesting stuff in there for future (regression/stability) testing, I'll ask separately for permission for that and which files are considered for that, then.

Sorry for the delay; I believe we can restore your library/libraries fine once we're through this ordeal. It's a matter of finding out what's happening exactly that's a little difficult now.

3. As a general mindset in a failure situation, mine is: everything is untrustworthy until proven otherwise. That means we have to check all our assumptions as we go along, ticking off items of possible concern until we've narrowed down to a small zone of probable cause. One such item to verify is software versions:

• do you remember which software version of Qiqqa you used to create those backups? Was it a Commercial Qiqqa version? v79 or earlier? Is it Open Source v80 or a v82 release? If you don't know, no sweat, then the date/time stamp of the archive tells us when it was created and we can guess-timate the version.

4. Next steps (to be addressed in follow-up messages here): analyze backup contents, check for possible "corruptions" (Qiqqa before v82 would act pretty oddly if you had fed it badly formatted BibTeX data, for example. Very, ahhh, "creative" individuals like myself could get metadata which would "hide" the entire record ("hey!? my whole document is gone! WUT?!"), etc. Hence: analysis needed of actual libraries; maybe further augmenting diagnostic reporting in Qiqqa to help us find out. After that: see if we need to improve recovery methods in Qiqqa and enhance those. (Each of which would mean, if necessary, pushing a new test release, maybe once, maybe a couple, for you to install and run to see what happens over on your side.)

Met vriendelijke groeten / Best regards,

Ger Hobbelt

[Hamihema]

I could successfully restore the entire library from one PC to another one, the qiqqa opened and everything was well, however, it was not logged in to my account, so I closed it and opened it again using my login details, this time non of the restored libraries were shown. So this is my question, how we can restore the libraries and then log in to our account and still be able to see the libraries?

[GerHobbelt]

"log into my account" sounds very much like you were using commercial qiqqa (v79). None of the open source qiqqa versions can access the qiqqa cloud (see also main README of the github repository). Open source qiqqa does not require any account / login.

MIGHT BE that you can reinstall the v79 commercial version afterwards and thusly login and use your cloud account.

Please read that README as that qiqqa cloud service is long overdue to be cancelled!

https://github.com/jimmejardine/qiqqa-open-source/blob/master/README.md

On Fri, Jul 23, 2021, 16:03 Hamihema @.***> wrote:

I could successfully restore the entire library from one PC to another one, the qiqqa opened and everything was well, however, it was not logged in to my account, so I closed it and opened it again using my login details, this time non of the restored libraries were shown. So this is my question, how we can restore the libraries and then log in to our account and still be able to see the libraries?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jimmejardine/qiqqa-open-source/issues/288#issuecomment-885661687, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADCIHRDB4BHBTVITF5YNILTZFZD3ANCNFSM4V4FRGUQ .