Optional 'Hard' link for goodreads 'read' shelf

[Grunthos]

Currently we use the the goodreads 'read' shelf to set the 'read' flag on first import only. Goodreads uses exclusive shelves (can not be on 'read' and 'to-read' or 'currently-reading') which is incompatible with our system of shelves.

The following request was received from a user:

If syncing with Goodreads, it would be nice to see an option to link the Goodreads "Read" bookshelf and the "Have you read this book field?" Another option is to disable the "Have you read this book?" and have the green tick show if a book is in "Read" bookshelf, no matter which bookshelf is being used.

Which would, more or less, require we implement shelves with special meaning. It's an interesting idea that needs more analysis, though I largely hate the 'exclusive' shelves model of goodreads.

[Imkal]

Hi,

On tuesday, I sent all my book to Goodreads to get the new books on my goodreads profile. Normally non of my books in Book Catalogue are registered on multiple shelves identified as "exclusive" in Goodreads but maybe one was (should have moved from the "wishlist" to the "to read" but was registered on both). It ended up with all my books on exclusive shelves other than the "Read" or "To Read" to be moved to the "To Read" one. Some of those new books were not available in Goodreads

I can understand you don't like the exclusive shelves mechanism and don't want to implement it in Book Catalogue though we need to make sure the update is done correctly.

I would suggest that a book registered on 2 Goodreads exclusive shelves is not updated and reported as an error.

I'm not sure about the update of the flag in case the book is on the Read shelves but I would definitely update the flag "Have you read This book? when the Read End date is defined and vice versa updating the Read End date with the current date when the flag is set.

Thanks and have a nice day, Patrice

[Grunthos]

It ended up with all my books on exclusive shelves other than the "Read" or "To Read" to be moved to the "To Read" one

This worried me: does this mean that books that should have gone on completely different shelves (like 'Reference' or 'Fiction') did not go to those shelves but ended up on only the read/to-read shelves?

Or was this a problem with the import? Or both?

[Imkal]

I only sent my books to Goodreads and did not import them back into Book Catalogue. When conntecting to Goodreads, I realized that all my books that were normally on the "Whislist" or on the "BD_Wishlist"shelve were moved to the "To Read" one. This was for all my books not just the new ones. I will try to reproduce if I can.

[Imkal]

Hi Philip,

I could reproduce the issue. In Bookcatalogue, I had a book in the "Currently reading" library and in the Read library at the same time. This situation has moved all the books (except 1) I had in other exclusive libraries to the "Ro-Read" library.The Read library was not affected. I will send you my DB so you can compare with my current Goodread situation.

Thanks and have a nice day, Patrice

[Grunthos]

Looking at the CSV file, it looks like there are 143 on to-read, but in goodreads you have 170 on to-read. Is that right?

It's hard for me to compare the two. Is it possible for you to point me to one that is 'to-read' in goodreads but should NOT be? Other than the 'currently-reading' ones?

[Imkal]

Hi Philip,

The situation in BC is the right one but the book of Mathieu Gaborit is on the "Read" and on the "Currently reading" exclusive shelves. Before I sent the books to goodreads the situation was the same in Goodreads but once sent the situation in Goodrfeads is as you see it now except for the Default shelf which I deleted in Goodreads and in BC as well but not in your export.

The Read, To-Read and Currently reading shelves cannot be modified in Goodreads, so removing the exclusive flag is not an option for those 3. I would suggest to use the Have you read this book flag to manage those 3 shelves in Goodreads and in BC.

For the others, having the constraint that the exclusive shelves will be set as unexclusive when you send your book to Goodreads is quite heavy. Is there no mean to get the info from Goodreads that a shelf is exclusive or not?

[Imkal]

Hi Philip,

The situation in BC is the right one but the book of Mathieu Gaborit is on the "Read" and on the "Currently reading" exclusive shelves. Before I sent the books to goodreads the situation was the same in Goodreads but once sent the situation in Goodrfeads is as you see it now except for the Default shelf which I deleted in Goodreads and in BC as well but not in your export.

The Read, To-Read and Currently reading shelves cannot be modified in Goodreads, so removing the exclusive flag is not an option for those 3. I would suggest to use the Have you read this book flag to manage those 3 shelves in Goodreads and in BC.

For the others, having the constraint that the exclusive shelves will be set as unexclusive when you send your book to Goodreads is quite heavy. Is there no mean to get the info from Goodreads that a shelf is exclusive or not?

[Grunthos]

I may be missing something, but "Chronique Du Soupir" by "Gaborit, Mathieu" is in goodreads, but not in the CSV file you sent me...at least as far as I could tell.

[Grunthos]

Recent csv has it, thanks.

Can you try something for me:

• go to a list of books in Book Catalogue with this book in it
• long-click on the book
• select 'Send to goodreads'
• tell me what happens on the goodreads site

It may be as simple as the book not being sent to goodreads. I just put a book on 'currently-reading' and 'read', then sent it and it ended up on 'read', which I what I would expect.

[Imkal]

The book is sent and moved to the "To-read" shelf and not the Read one.

If it was only that book it would have been a minor issue but all other book that are in wishlist and bd_wishilist exclusive shelves were all moved to To-Read shelf while there was no reason to do so.

[Grunthos]

Hmmm...I think I can now reproduce it. In my previous test, the book was on 'read', 'currently-reading' and was also on the 'SciFi/Fantasy' shelf...and it went to 'read' + SF/F.

There seems to be some kind of weird ordering and/or string problem. When I send the same book multiple times I I get the following results:

In BC: 'currently-reading', 'read', 'to buy'

-> 'to-read', 'to-buy' -> 'read', 'to-buy' -> 'currently-reading', 'to-buy'

so it looks like the exclusive shelves are just being really messed up.

I will hit this in the debugger tomorrow, but would be interested to know if you see a similar set of random results when sending multiple times.

[Imkal]

I tried several times and it's switching from Read to To-read but never on Currently reading.

[Grunthos]

OK, looking at the goodreads API code in Book Catalogue, I can see how this happens. The code also makes some nasty assumptions about shelves (eg. it assumes only the 3 standard exclusive shelves), and it assumes in another place that a book is either on the 'read' shelf or the 'to-read' shelf.

My thinking at the current time is that I'd much prefer the goodreads guys make exclusivity optional, since I view exclusive shelves as a very bad design choice and more of a bug than a feature. As a result I am rather disinclined to spend much time replicating and supporting it in Book Catalogue.

I the mean time, I propose we do the following:

When Sending Books

• get an initial list of all bookshelves in goodreads (one more API call)
• if a book is not an any exclusive shelves, but it marked as 'read', add it (for the purpose of the update) to the 'read' shelf
• if a book is on more than one exclusive shelf, do not set any exclusive shelves, but update the other details (just ignore them silently, or perhaps log it as an event).
• if a book is on exactly one exclusive shelf, set it

When retrieving books

• get an initial list of all bookshelves in goodreads (one more API call)
• if a book is currently on zero or one goodreads exclusive shelves in the local BC database, then update based on goodreads
• if a book is currently on more than one goodreads exclusive shelf in the local BC database, then ignore the goodreads shelves but update the other details

Question:

Does this cover all the bases?

[Grunthos]

I'm not sure I really need to worry about this:

if a book is currently on more than one goodreads exclusive shelf in the local BC database, then ignore the goodreads shelves but update the other details

Perhaps on import it is OK to overwrite with GR shelves?

[Imkal]

I don't believe exclusive shelves are a bug, it's a feature and I find it usefull somehow. On Babelio (French site) they have 2 fields to implement this a status which is about read to read, currently reading and which are exclusive and "stamps" which are not exclusive and you can use to classify your books. But I can understand your point of view if you look at how collections are managed in Librarything.

In Goodread, there is no mean to have a book registered on zero exclusive shelf, there must be one.

I would if feasible add a parameter that would allow the user to preserve the shelves from being updated from and into Goodreads.

With the flag on, when updating Goodreads, it would follow the statements here bleow and when retrieving from Goodreads, GR shelves should overwirte BC for what concerns shelves they have in common. I can imagine people who have extra shelves in BC they don't have in Goodreads.

With the flag off, there should not be any update at all.

To be honest, before the integration with Goodreads, I had a few shelves in BC and I find it quite boring to manage GR bokkshelves within BC so I would the the flag off for what concern my library ;)

When sending Books:

•get an initial list of all bookshelves in goodreads (one more API call) •if a book is not an any or on multiple exclusive shelves, but it marked as 'read', add it (for the purpose of the update) to the 'read' shelf, if not set it in the To-Read one •if a book is on exactly one exclusive shelf, set it

Now we also have to check the particular situation of that book in GR before updating. I don't knwo how tha API will react in case you want to add an exclusive shelf whil the book is already registered on another and as an exclusive shelf is mandatory, you just can't remove them all.

[Grunthos]

I don't believe exclusive shelves are a bug, it's a feature and I find it usefull somehow

I open to suggestions of how it is useful. The main problem is that a sect of two exclusive shelves is just a boolean: "have read"/"have not read". A set of more than two should be implemented as a list of options, much like a colour picker, or book format. Exclusive shelves is just a poor way of implementing a state field.

What makes it worse is that exclusive fields should be genuinely exclusive. I have no problem with the concept of a drop-down list of unique items, but "have read", "to read" and "currently reading" all have genuine logical overlap.

I would if feasible add a parameter that would allow the user to preserve the shelves from being updated

This is non-trivial and falls into the category of writing code and UI to support a broken feature, IMO.

•if a book is not an any or on multiple exclusive shelves, but it marked as 'read', add it (for the purpose of the update) to the 'read' shelf, if not set it in the To-Read one

This is actually existing logic that breaks your update. It tries to put them on "read" or "to read", thereby losing all "currently-reading". Also it would have the effect of losing the status of books in GR on user-defined exclusive shelves. This is why I wrote my version above.

[Grunthos]

FWIW, it might be worth considering the possible states and how they should be applied when importing from GR or exporting to GR (need to bear in mind special shelves and user-defined exclusive shelves)...some cases are trivial:

Book in BC, not in GR (new book)

Book is 'read' in BC

On 0 to 1 exclusive shelves

On 2 or more exclusive shelves

Book is NOT 'read' in BC

On 0 to 1 exclusive shelves

On 2 or more exclusive shelves

Book in BC, and in GR

Book is 'read' in BC

On 0 to 1 exclusive shelves in BC

On 2 or more exclusive shelves in BC

Book is NOT 'read' in BC

On 0 to 1 exclusive shelves in BC

On 2 or more exclusive shelves in BC

[Imkal]

I've run some tests and results might interest you

• Book in BC on the Read and Currently reading shelves flag Read is false: When sending to GR, the book switches from Read to To read shelves
• Book in BC on the Read and Currently reading shelves flag Read is true: When sending to GR, the book switches from Read to Currently reading shelves
• Book in BC on the Read, To Read and Currently reading shelves flag Read is false: When sending to GR, the book is set to the Read shelf
• Book in BC on the Read, To Read and Currently reading shelves flag Read is false: When sending to GR, the book is set to the Read shelf

I've run some more and came to the conclusion that it is the usage of your Read flag which is making things difficult.

I would say that you just updates the Goodreads shelves with what you find in BC except for books that are not in GR yet and for which you need to identify the exclusive shelf the book is on. The state table then becomes

Books in BC not in GR

• On 0 exclusive shelf => Use the Read flag to add it on the Read or on the To Read shelf and add to all other shelves the book is on in BC
• On 1 or more exclusive shelves => Add the book to all shelves the latest exclusive to be updated being the one that will be effective

Books in BC and in GR

• In all cases, add the book to all shelves the latest exclusive to be updated being the one that will be effective. If there is no exclusive shelf there is one in GR and that's enough.

[Grunthos]

the latest exclusive to be updated being the one that will be effective

Unfortunately, we do not know this, so we are back to the full state diagram. Even if we did know this, it is not clear where a book on the 'foobar' user-defined exclusive shelf and with the 'read' flag set should go.

It is worth filling out the full list of states bearing in mind the possibility of user-defined exclusive shelves etc.

[Grunthos]

To be clear, the reason the state table is important is that:

• we only know the state at GR and BC, not the history
• we only know the dates that some aspect of the book was updated at each site (eg. author may have been changed)

In these cases we need to know how to sensibly copy data. Objectives of such a system should be:

• doing the same operation twice results in the same data
• copying A->B then B->A should result in A being unchanged (this is quite hard in the current setup)

Edit: One possible solution (which is looking good!) is to completely ignore all exclusive shelves...

[Imkal]

Unfortunately, we do not know this, ....

I knwo but we should not care.

doing the same operation twice results in the same data

As long as we always add the shelves in the same order, it will.

One possible solution (which is looking good!) is to completely ignore all exclusive shelves...

This is what I meant with the latest exclusive to be updated being the one effective. Sorry, if it was not clear. But as GR requires one exclusive shelf and I don't know how GR will react if we don't provide one, I wold suggest that for the new book we use the Read flag.