Automatic cover feature.

[meteos77]

Could we have a functionality to download the covers without having to open 1 by 1 the documents ?

We use a menu or button to ask BiblioteQ to complete if the covers are empty to fetch them from amazon.

[textbrowser]

No. There is too much connected to the item's type. There are errors to report, proxies to configure, download type to select. BQ is not wget.

[textbrowser]

And if this is implemented, you'll progress into asking to download and populate all information for 20 items and automatically parse and populate the items. There are also things like interrupting requests which need to be considered. Thought about adding this to Batch Activities and then I stopped thinking about it because it becomes yet another massive amount of work for little return. I could write a script to do it instead.

[meteos77]

I thought it wasn't easy :(

With another tool, I can obtain images with the file format name "isbn13_0.jpg". Could BiblioteQ display its images external to the base without too much development? If we place them in a directory like .biblioteq/images_front_cover/, for example.

[textbrowser]

Download the images, convert them into Base-64, and update the relevant rows. You can work outside of BQ to do this.

[meteos77]

Download the images -> OK Convert them into Base-64 -> ?? python should be able to help Update the relevant rows -> ?? SQL?? I'm always afraid of breaking my base by writing directly into it.

You can work outside of BQ to do this. -> We love BiblioteQ so we want to work with him :-)

[FriedrichFroebel]

Convert them into Base-64 -> ?? python should be able to help

You already have a code snippet for converting base64 to plain bytes (#333), thus it should be rather easy to implement the other direction. The base64 module is part of the standard library and has online documentation as well as documentation inside the interactive interpreter (help(base64)). When writing code/developing, you should get familiar with reading the official documentation and doing further research if required, which will often provide you with suitable solutions.

Update the relevant rows -> ?? SQL?? I'm always afraid of breaking my base by writing directly into it.

If you are afraid of breaking anything, just create a backup copy of your database or use a dedicated copy during development until you are sure that everything works correctly. SQLite3 has standalone files which are very easy to backup.

[textbrowser]

Magic panel will be included in the last release of 2023. It's better if one learns blending separate processes instead of a single process. Benefits exploration.

[meteos77]

I think I've made good progress, even if it doesn't work yet :-)

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

"""Script pour stocker les images dans la base BiblioteQ."""
import base64
import sys
import sqlite3
import os

###################
if __name__ == "__main__":
    sourcefile = (sys.argv[1])
    print("sourcefile : BiblioteQ sqlite" + sourcefile )
    connection = sqlite3.connect(sourcefile)
    cursor = connection.cursor()
    sql="SELECT isbn13 FROM book"
    cursor.execute(sql)
    listeisbn13 = cursor.fetchall()
    for isbn13 in listeisbn13:
        isbnbook = isbn13[0]
        imagefile = isbnbook + "_0.jpg"

        # test si le fichier image existe
        fichierexist = os.path.isfile(imagefile)
        if fichierexist is True:
            print("image file " + imagefile + " FOUND")
            # encodage en base64
            imagedata = base64.b64encode(open(imagefile, "rb").read())

            sqlrequest="UPDATE book SET front_cover='%s' WHERE isbn13=%s"%('imagedata',isbnbook)
            cursor.execute(sqlrequest)
        else:
            print("image file " + imagefile + " NOT FOUND")
    cursor.close
    connection.close()

[meteos77]

The Christmas version is going to be great!

[meteos77]

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

"""Script pour stocker les images dans la base BiblioteQ.
BQimage_Ajoute-Images-dans-base-BQ_sqlite.py : Importer les images dans la base sqlite de BiblioteQ.
les images se trouvent dans le répertoire de la variable : IMAGESREP avec le nom isbn13.jpg pour chaque images

Remplir variable IMAGESREP
Prise en compte de la variable BIBLIOTEQ_DATABASE_NAME
"""

import base64
import sys
import sqlite3
import os

IMAGESREP = "/home/BiblioteQ_Images/"

def traitement(sourcefile):
    print("sourcefile : BiblioteQ sqlite" + sourcefile )
    connection = sqlite3.connect(sourcefile)
    cursor = connection.cursor()
    sql="SELECT isbn13 FROM book"
    cursor.execute(sql)
    listeisbn13 = cursor.fetchall()
    for isbn13 in listeisbn13:
        isbnbook = isbn13[0]
        imagefile = isbnbook + ".jpg"

        # test si le fichier image existe
        cheminfichier = os.path.join(IMAGESREP, imagefile)
        fichierexist = os.path.isfile(cheminfichier)
        if fichierexist is True:
            print("Le fichier image pour l'isbn13 : " + isbnbook + " TROUVÉE : "+ imagefile)
            # encodage en base64
            imagedata = base64.b64encode(open(cheminfichier, "rb").read())

            sqlrequest="""UPDATE book SET front_cover= ? WHERE isbn13= ?"""
            data = (imagedata, isbnbook)
            cursor.execute(sqlrequest, data)
            connection.commit()
        else:
            print("Le fichier image pour l'isbn13 : " + isbnbook + " INTROUVABLE")
    cursor.close
    connection.close()

    ###################
if __name__ == "__main__":
    sourcefile = ""
    print("BQimageAjouter pour importer les couvertures dans la base BiblioteQ")
    # Test si la variable BIBLIOTEQ_CURRENT_SQLITE_FILE est définie
    try:
        envfname = ""
        envfname = os.environ['BIBLIOTEQ_DATABASE_NAME']
    except:
        pass
    if len(envfname) > 0:
        sourcefile = envfname
        print(f"la valeur de sourcefile ={sourcefile}")
        traitement(sourcefile)
    else:
        try:
            sourcefile = (sys.argv[1])
        except:
            exit
        if len(sourcefile) > 0:
            traitement(sourcefile)
        else:
            print("Une base BiblioteQ est nécessaire")

[textbrowser]

New script committed. Tested with sample SQLite database and Open Library. Magic is not necessary. This is much easier and smooth. Magic is much work.

[textbrowser]

https://github.com/textbrowser/biblioteq/blob/master/Scripts/download-openlibrary-images-sqlite.sh

[textbrowser]

Done!

[meteos77]

Super, The scripts are easier to adapt to our needs and they do the job well.

It also shows me the shell procedure for other fields if we want to update them by script. It's great.

One more question: is there a "magic sql query" to create a book (not an update, but a creation)?

[textbrowser]

I may add an option to avoid books which already have front cover images. The PQ script is incomplete.

[textbrowser]

Once you understand the book table's definition, creating a book is easy. After a book entry is created, you will probably need a book-copy entry. The field types provide guidance.

[meteos77]

For the Bnf service (France) modify your amazon script. The difficulty was to transform isbn13 into ark ; (allows you to have images in 979-xxxxxx)

French only :-)

download-bnf-images-sqlite-isbn13.sh.zip

Note : BnF provides covers for books less than 6 months old and books to be published in a maximum of 6 months.

so amazon is our friend.

[meteos77]

The creation process still seems complicated to me:it has to generate the myoid; accession_number; the copy; there's the book_sequence ????

I'm not quite there yet :-)