search

bayang / jelu

Self hosted read and to-read list book tracker
MIT License
313 stars 12 forks source link

import from goodreads not importing, missing isbn #81

Closed reitenth closed 7 months ago

reitenth commented 8 months ago

image

bayang commented 8 months ago

Hi, the goodreads import expects isbn to be present.

Is there a reason none of your books has either isbn10 or isbn13 filled ?

Or it can be that goodreads changed (again) their export format. I can have a look at your file if you can share it to me (or even juste some lines if you want). You can paste the lines here or send me your file at stls@ik.me

reitenth commented 8 months ago

goodreads_library_export.csv

bayang commented 8 months ago

Ok, the file is fine. So according to the logs the import should have added 132 books to your jelu library and ignored a bunch of lines without isbn.

This is actually the standard behaviour of the goodreads csv import.

Maybe I should state it clearly in the docs.

You can import those missing books using the jelu built in metadata fetching.

Or manage to get th isbn added through goodreads and re-export a fuller csv.

KucharczykL commented 7 months ago

Hello, not the same person, but I'm also having the same problem after a fresh installation using Docker. I've just exported my Goodreads data, and after import only 1 book has been added.

In the logs it says "parsing finished, 473 entries recorded" though I'm not sure what that means.

Find attached my Goodreads export: goodreads_library_export.csv

bayang commented 7 months ago

@KucharczykL the logs mean that 473 lines from your csv have been kept, waiting for the metadata retrieval.

Import happens in 2 steps, first the csv is read and all relevant data is kept in database and then the second step happens, which tries to fetch metadata online (depending if you chose this option on the import page). The second step can be quite long and happens in the background. You can see the beginning and end of your import in your dashboard, in the messages section. I suggest you wait a bit and see if books are added. I tested your file and at least some files are added.

KucharczykL commented 7 months ago

There must be some difference between our setups then because since I wrote my previous message nothing else has happened:

image

I've checked the logs and the last message is: "csv parsing of $FILE ended after: 15 seconds" and that's it.

Just for completion's sake, this is my Docker compose file:

---
services:
  jelu:
    image: wabayang/jelu
    container_name: jelu
    volumes:
      - ${DOCKER_STORAGE_PATH}/jelu/config:/config
      - ${DOCKER_STORAGE_PATH}/jelu/database:/database
      - ${DOCKER_STORAGE_PATH}/jelu/files/images:/files/images
      - ${DOCKER_STORAGE_PATH}/jelu/files/imports:/files/imports
      - /etc/timezone:/etc/timezone:ro
    networks:
      - public
    environment:
      SERVER_PORT: 1111
      SPRING_DATASOURCE_USERNAME: lukas
      SPRING_DATASOURCE_PASSWORD: Q^k5i2^hN!wmEr6JLkYP9ME
      JELU_CORS_ALLOWED-ORIGINS: https://jelu.${DOMAIN}
    profiles:
      - base
    restart: unless-stopped

I'm serving it via Caddy reverse proxy:

jelu.${DOMAIN} {
    handle {
        reverse_proxy jelu:1111
    }
}

This is my whole log after restarting the container:

Show log ``` jelu | jelu | ,--. ,------. ,--. ,--. ,--. jelu | | | | .---' | | | | | | jelu | ,--. | | | `--, | | | | | | jelu | | '-' / | `---. | '--. ' '-' ' jelu | `-----' `------' `-----' `-----' jelu | jelu | version : 0.45.0 jelu | jelu | 2023-12-02T20:22:06.400+01:00 INFO 1 --- [ main] i.github.bayang.jelu.JeluApplicationKt : Starting JeluApplicationKt v0.45.0 using Java 17.0.9 with PID 1 (/app/BOOT-INF/classes started by root in /app) jelu | 2023-12-02T20:22:06.403+01:00 DEBUG 1 --- [ main] i.github.bayang.jelu.JeluApplicationKt : Running with Spring Boot v3.1.5, Spring v6.0.13 jelu | 2023-12-02T20:22:06.404+01:00 INFO 1 --- [ main] i.github.bayang.jelu.JeluApplicationKt : No active profile set, falling back to 1 default profile: "default" jelu | 2023-12-02T20:22:08.158+01:00 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JDBC repositories in DEFAULT mode. jelu | 2023-12-02T20:22:08.232+01:00 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 68 ms. Found 0 JDBC repository interfaces. jelu | 2023-12-02T20:22:09.134+01:00 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 1111 (http) jelu | 2023-12-02T20:22:09.145+01:00 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat] jelu | 2023-12-02T20:22:09.145+01:00 INFO 1 --- [ main] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.15] jelu | 2023-12-02T20:22:09.242+01:00 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext jelu | 2023-12-02T20:22:09.243+01:00 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 2746 ms jelu | 2023-12-02T20:22:10.637+01:00 INFO 1 --- [ main] o.s.b.a.w.s.WelcomePageHandlerMapping : Adding welcome page: class path resource [public/index.html] jelu | 2023-12-02T20:22:10.873+01:00 INFO 1 --- [ main] o.s.s.web.DefaultSecurityFilterChain : Will secure Or [Mvc [pattern='/api/**']] with [org.springframework.security.web.session.DisableEncodeUrlFilter@236b4a44, org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@35f760a4, org.springframework.security.web.context.SecurityContextHolderFilter@7ec508e6, org.springframework.security.web.header.HeaderWriterFilter@1a67f8b4, org.springframework.web.filter.CorsFilter@2ac05a33, org.springframework.security.web.authentication.logout.LogoutFilter@158e6fc2, org.springframework.security.web.authentication.www.BasicAuthenticationFilter@6d6f6ca9, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@30e2016a, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@61d09475, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@35af52dc, org.springframework.security.web.session.SessionManagementFilter@4e52d2f2, org.springframework.security.web.access.ExceptionTranslationFilter@6ae6d078, org.springframework.security.web.access.intercept.AuthorizationFilter@45c408a4] jelu | 2023-12-02T20:22:11.002+01:00 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting... jelu | 2023-12-02T20:22:11.175+01:00 INFO 1 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Added connection org.sqlite.jdbc4.JDBC4Connection@11170228 jelu | 2023-12-02T20:22:11.178+01:00 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed. jelu | 2023-12-02T20:22:11.881+01:00 INFO 1 --- [ main] liquibase.changelog : Reading from DATABASECHANGELOG jelu | 2023-12-02T20:22:11.893+01:00 INFO 1 --- [ main] liquibase.executor : Changelog query completed. jelu | Database is up to date, no changesets to execute jelu | 2023-12-02T20:22:12.341+01:00 INFO 1 --- [ main] o.s.l.c.support.AbstractContextSource : Property 'userDn' not set - anonymous context will be used for read-write operations jelu | 2023-12-02T20:22:12.437+01:00 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 1111 (http) with context path '' jelu | 2023-12-02T20:22:12.466+01:00 INFO 1 --- [ main] i.g.b.jelu.service.AppLifecycleAware : Lucene index not found, trigger rebuild jelu | 2023-12-02T20:22:12.479+01:00 INFO 1 --- [ main] i.g.b.jelu.service.SearchIndexService : Rebuild index for: [book, author] jelu | 2023-12-02T20:22:12.481+01:00 INFO 1 --- [ main] i.g.b.jelu.service.SearchIndexService : Rebuilding index for Book jelu | 2023-12-02T20:22:12.646+01:00 INFO 1 --- [ main] i.g.b.jelu.service.SearchIndexService : Number of entities: 2 and pages : 1 jelu | 2023-12-02T20:22:12.845+01:00 INFO 1 --- [ main] i.g.b.jelu.service.SearchIndexService : Processing page 1 of 1 (5000 elements) jelu | 2023-12-02T20:22:12.964+01:00 INFO 1 --- [ main] i.g.b.jelu.service.SearchIndexService : Wrote Book index in 123.985403ms jelu | 2023-12-02T20:22:13.125+01:00 DEBUG 1 --- [ main] i.g.b.jelu.service.SearchIndexService : no authors index yet jelu | 2023-12-02T20:22:13.231+01:00 INFO 1 --- [ main] i.g.bayang.jelu.search.LuceneHelper : Lucene index version: 1 jelu | 2023-12-02T20:22:13.247+01:00 INFO 1 --- [ main] i.github.bayang.jelu.JeluApplicationKt : Started JeluApplicationKt in 7.46 seconds (process running for 7.955) jelu | 2023-12-02T20:22:25.320+01:00 INFO 1 --- [nio-1111-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet' jelu | 2023-12-02T20:22:25.321+01:00 INFO 1 --- [nio-1111-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet' jelu | 2023-12-02T20:22:25.323+01:00 INFO 1 --- [nio-1111-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 2 ms jelu | 2023-12-02T20:22:25.349+01:00 WARN 1 --- [nio-1111-exec-1] o.s.web.servlet.PageNotFound : No mapping for GET /profile/messages jelu | 2023-12-02T20:22:38.496+01:00 DEBUG 1 --- [nio-1111-exec-1] i.g.b.jelu.controllers.ImportController : target import file at /files/imports/goodreads_library_export.csv jelu | 2023-12-02T20:22:38.588+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 18373, title Flowers for Algernon and author Daniel Keyes, import it yourself manually jelu | 2023-12-02T20:22:38.605+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 63198504, title Valuable Humans in Transit and Other Stories and author qntm, import it yourself manually jelu | 2023-12-02T20:22:38.606+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 13453029, title Wool Omnibus (Silo, #1) and author Hugh Howey, import it yourself manually jelu | 2023-12-02T20:22:38.606+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 17306293, title Shift (Silo, #2) and author Hugh Howey, import it yourself manually jelu | 2023-12-02T20:22:38.607+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10294362, title The Rise of Endymion (Hyperion Cantos, #4) and author Dan Simmons, import it yourself manually jelu | 2023-12-02T20:22:38.625+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 36172990, title Viriconium (Viriconium #1-4) and author M. John Harrison, import it yourself manually jelu | 2023-12-02T20:22:38.642+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10294361, title Endymion (Hyperion Cantos, #3) and author Dan Simmons, import it yourself manually jelu | 2023-12-02T20:22:38.642+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10429950, title The Fall of Hyperion (Hyperion Cantos, #2) and author Dan Simmons, import it yourself manually jelu | 2023-12-02T20:22:38.642+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 17980416, title Dust (Silo, #3) and author Hugh Howey, import it yourself manually jelu | 2023-12-02T20:22:38.676+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 11952012, title Otcové a děti and author Ivan Turgenev, import it yourself manually jelu | 2023-12-02T20:22:38.677+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 315939, title Ubik and author Philip K. Dick, import it yourself manually jelu | 2023-12-02T20:22:38.678+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 11297277, title Hard-Boiled Wonderland and the End of the World and author Haruki Murakami, import it yourself manually jelu | 2023-12-02T20:22:38.730+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 13560311, title The Handmaid's Tale and author Margaret Atwood, import it yourself manually jelu | 2023-12-02T20:22:38.801+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 28015955, title Gulliverovy cesty and author Jonathan Swift, import it yourself manually jelu | 2023-12-02T20:22:38.869+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 28947002, title Self Discipline Mindset: Why Self Discipline Is Lacking In Most And How To Unleash It Now (Habits, Willpower, Confidence, Emotional Intelligence Book 1) and author Curtis Leone, import it yourself manually jelu | 2023-12-02T20:22:38.930+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 18626849, title Stories of Your Life and Others and author Ted Chiang, import it yourself manually jelu | 2023-12-02T20:22:38.949+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 60727041, title We Have Always Lived in the Castle and author Shirley Jackson, import it yourself manually jelu | 2023-12-02T20:22:38.950+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 7805, title Pale Fire and author Vladimir Nabokov, import it yourself manually jelu | 2023-12-02T20:22:39.030+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 45064996, title Hyperion (Hyperion Cantos, #1) and author Dan Simmons, import it yourself manually jelu | 2023-12-02T20:22:39.070+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 24546130, title Nepřemožitelný and author Stanisław Lem, import it yourself manually jelu | 2023-12-02T20:22:39.130+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 50728243, title The Martian and author Andy Weir, import it yourself manually jelu | 2023-12-02T20:22:39.149+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 8870798, title Sourcery (Discworld, #5) and author Terry Pratchett, import it yourself manually jelu | 2023-12-02T20:22:39.406+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 33314081, title The Stone Sky (The Broken Earth #3) and author N.K. Jemisin, import it yourself manually jelu | 2023-12-02T20:22:39.481+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 25194939, title The Fifth Season (The Broken Earth, #1) and author N.K. Jemisin, import it yourself manually jelu | 2023-12-02T20:22:39.602+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 485894, title The Metamorphosis and author Franz Kafka, import it yourself manually jelu | 2023-12-02T20:22:39.603+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 20337607, title The Accidental Universe: The World You Thought You Knew and author Alan Lightman, import it yourself manually jelu | 2023-12-02T20:22:39.603+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 36661734, title Rozmarné léto and author Vladislav Vančura, import it yourself manually jelu | 2023-12-02T20:22:39.604+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 25899336, title When Breath Becomes Air and author Paul Kalanithi, import it yourself manually jelu | 2023-12-02T20:22:39.625+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 13484973, title Povětroň and author Karel Čapek, import it yourself manually jelu | 2023-12-02T20:22:39.730+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 53485326, title The Last Wish (The Witcher, #0.5) and author Andrzej Sapkowski, import it yourself manually jelu | 2023-12-02T20:22:39.791+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 32951598, title Aristotle's Poetics and author Aristotle, import it yourself manually jelu | 2023-12-02T20:22:39.811+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 17905709, title The Narrow Road to the Deep North and author Richard Flanagan, import it yourself manually jelu | 2023-12-02T20:22:39.833+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 32758901, title All Systems Red (The Murderbot Diaries, #1) and author Martha Wells, import it yourself manually jelu | 2023-12-02T20:22:40.056+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 2517, title My Name Is Red and author Orhan Pamuk, import it yourself manually jelu | 2023-12-02T20:22:40.057+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 97333, title At Swim-Two-Birds and author Flann O'Brien, import it yourself manually jelu | 2023-12-02T20:22:40.075+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 275, title The Lover and author Marguerite Duras, import it yourself manually jelu | 2023-12-02T20:22:40.130+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 13079982, title Fahrenheit 451 and author Ray Bradbury, import it yourself manually jelu | 2023-12-02T20:22:40.131+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 21064438, title The Murder of Roger Ackroyd (Hercule Poirot, #4) and author Agatha Christie, import it yourself manually jelu | 2023-12-02T20:22:40.131+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10687642, title The Crippled God (Malazan Book of the Fallen, #10) and author Steven Erikson, import it yourself manually jelu | 2023-12-02T20:22:40.151+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 8486685, title Dust of Dreams (Malazan Book of the Fallen, #9) and author Steven Erikson, import it yourself manually jelu | 2023-12-02T20:22:40.299+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 38102522, title dot: Rory Sutherland: Thinking like a marketing genius and author Mike Dariano, import it yourself manually jelu | 2023-12-02T20:22:40.300+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 6560981, title Stalo se zítra and author Ivo Železný, import it yourself manually jelu | 2023-12-02T20:22:40.319+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 44528152, title Super Thinking: The Big Book of Mental Models and author Gabriel Weinberg, import it yourself manually jelu | 2023-12-02T20:22:40.339+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 3, title Harry Potter and the Sorcerer's Stone (Harry Potter, #1) and author J.K. Rowling, import it yourself manually jelu | 2023-12-02T20:22:40.358+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 6334, title Never Let Me Go and author Kazuo Ishiguro, import it yourself manually jelu | 2023-12-02T20:22:40.416+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 34943643, title Agency (Jackpot, #2) and author William Gibson, import it yourself manually jelu | 2023-12-02T20:22:40.502+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 12108632, title Toll the Hounds (Malazan Book of the Fallen, #8) and author Steven Erikson, import it yourself manually jelu | 2023-12-02T20:22:40.535+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10194157, title Shadow and Bone (The Shadow and Bone Trilogy, #1) and author Leigh Bardugo, import it yourself manually jelu | 2023-12-02T20:22:40.598+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 7840870, title Proces and author Franz Kafka, import it yourself manually jelu | 2023-12-02T20:22:40.742+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 6037544, title The Fourth Profession and author Larry Niven, import it yourself manually jelu | 2023-12-02T20:22:40.880+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 55713987, title Foundation (Foundation, #1) and author Isaac Asimov, import it yourself manually jelu | 2023-12-02T20:22:40.895+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 6088007, title Neuromancer (Sprawl, #1) and author William Gibson, import it yourself manually jelu | 2023-12-02T20:22:40.896+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 53313009, title Blood Meridian: Or the Evening Redness in the West and author Cormac McCarthy, import it yourself manually jelu | 2023-12-02T20:22:40.958+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 1852, title The Call of the Wild and author Jack London, import it yourself manually jelu | 2023-12-02T20:22:40.974+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 1885, title Pride and Prejudice and author Jane Austen, import it yourself manually jelu | 2023-12-02T20:22:40.975+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 13574302, title Pět japonských novel and author Ichiyō Higuchi, import it yourself manually jelu | 2023-12-02T20:22:41.071+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 11532022, title Reaper's Gale (Malazan Book of the Fallen, #7) and author Steven Erikson, import it yourself manually jelu | 2023-12-02T20:22:41.089+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 11859032, title The Bonehunters (Malazan Book of the Fallen, #6) and author Steven Erikson, import it yourself manually jelu | 2023-12-02T20:22:41.105+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 34507927, title How to Take Smart Notes: One Simple Technique to Boost Writing, Learning and Thinking – for Students, Academics and Nonfiction Book Writers and author Sönke Ahrens, import it yourself manually jelu | 2023-12-02T20:22:41.106+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 17929200, title Tanečnice z Izu a jiné prózy and author Yasunari Kawabata, import it yourself manually jelu | 2023-12-02T20:22:41.202+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 823, title Quicksilver (The Baroque Cycle, #1) and author Neal Stephenson, import it yourself manually jelu | 2023-12-02T20:22:41.234+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 51960607, title Slaughterhouse-Five and author Kurt Vonnegut Jr., import it yourself manually jelu | 2023-12-02T20:22:41.326+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 20157104, title Poor Charlie's Almanack: The Wit and Wisdom of Charles T. Munger and author Charles T. Munger, import it yourself manually jelu | 2023-12-02T20:22:41.327+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 16902, title Walden and author Henry David Thoreau, import it yourself manually jelu | 2023-12-02T20:22:41.423+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 43081599, title Povídky z jedné kapsy druhé kapsy and author Karel Čapek, import it yourself manually jelu | 2023-12-02T20:22:41.532+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 38619, title Magic Bites (Kate Daniels, #1) and author Ilona Andrews, import it yourself manually jelu | 2023-12-02T20:22:42.109+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 34466963, title Why We Sleep: Unlocking the Power of Sleep and Dreams and author Matthew Walker, import it yourself manually jelu | 2023-12-02T20:22:42.110+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 16981, title Invisible Man and author Ralph Ellison, import it yourself manually jelu | 2023-12-02T20:22:42.193+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 42378674, title Trapné povídky and author Karel Čapek, import it yourself manually jelu | 2023-12-02T20:22:42.228+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 17666815, title Hovory s T. G. Masarykem and author Karel Čapek, import it yourself manually jelu | 2023-12-02T20:22:42.348+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 3184732, title Settlers: The Mythology of the White Proletariat and author J. Sakai, import it yourself manually jelu | 2023-12-02T20:22:42.597+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 8176064, title Breakthrough Rapid Reading and author Peter Kump, import it yourself manually jelu | 2023-12-02T20:22:42.634+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 2189576, title Vějíř a meč and author Vlasta Winkelhöferová, import it yourself manually jelu | 2023-12-02T20:22:42.650+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 46170, title For Whom the Bell Tolls and author Ernest Hemingway, import it yourself manually jelu | 2023-12-02T20:22:42.698+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 414999, title Childhood's End and author Arthur C. Clarke, import it yourself manually jelu | 2023-12-02T20:22:42.734+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 18869177, title Lying and author Sam Harris, import it yourself manually jelu | 2023-12-02T20:22:42.871+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 23275060, title 15 Commitments of Conscious Leadership and author Jim Dethmer, import it yourself manually jelu | 2023-12-02T20:22:43.088+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 23346740, title Sapiens: A Brief History of Humankind and author Yuval Noah Harari, import it yourself manually jelu | 2023-12-02T20:22:43.241+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 1383701, title Death of a Salesman and author Arthur Miller, import it yourself manually jelu | 2023-12-02T20:22:43.258+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10632190, title The Emperor of All Maladies: A Biography of Cancer and author Siddhartha Mukherjee, import it yourself manually jelu | 2023-12-02T20:22:43.931+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 242472, title The Black Swan: The Impact of the Highly Improbable and author Nassim Nicholas Taleb, import it yourself manually jelu | 2023-12-02T20:22:44.109+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 27108913, title The Little Book of Stupidity: How We Lie to Ourselves and Don't Believe Others and author Sia Mohajer, import it yourself manually jelu | 2023-12-02T20:22:44.957+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 34624575, title Čas žít, čas umírat and author Erich Maria Remarque, import it yourself manually jelu | 2023-12-02T20:22:44.957+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 40961427, title 1984 and author George Orwell, import it yourself manually jelu | 2023-12-02T20:22:44.999+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 24761454, title The Psychology of Human Misjudgement and author Charles T. Munger, import it yourself manually jelu | 2023-12-02T20:22:45.112+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 12625589, title Turing's Cathedral: The Origins of the Digital Universe and author George Dyson, import it yourself manually jelu | 2023-12-02T20:22:45.244+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10260, title The Pleasure of Finding Things Out: The Best Short Works of Richard P. Feynman and author Richard P. Feynman, import it yourself manually jelu | 2023-12-02T20:22:45.263+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 38117105, title The Monster Baru Cormorant (The Masquerade, #2) and author Seth Dickinson, import it yourself manually jelu | 2023-12-02T20:22:45.263+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 35755710, title The market for lemons : quality uncertainty and the market mechanism de George A. Akerlof: Les Fiches de Lecture d'Universalis (French Edition) and author Encyclopædia Universalis, import it yourself manually jelu | 2023-12-02T20:22:45.366+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 44245196, title The Great Mental Models: General Thinking Concepts and author Shane Parrish, import it yourself manually jelu | 2023-12-02T20:22:45.501+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 17333324, title Ancillary Justice (Imperial Radch, #1) and author Ann Leckie, import it yourself manually jelu | 2023-12-02T20:22:45.740+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 567189, title Language in Thought and Action and author S.I. Hayakawa, import it yourself manually jelu | 2023-12-02T20:22:45.771+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10016013, title Harry Potter and the Methods of Rationality and author Eliezer Yudkowsky, import it yourself manually jelu | 2023-12-02T20:22:45.904+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 24601008, title Gore Vidal: History of The National Security State and author Paul Jay, import it yourself manually jelu | 2023-12-02T20:22:46.376+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 40857434, title ComSec: Off-The-Grid Communications Strategies for Privacy Enthusiasts, Journalists, Politicians, Crooks, and the Average Joe and author Justin Carroll, import it yourself manually jelu | 2023-12-02T20:22:52.152+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 15857513, title Rory Sutherland: The Wiki Man and author Rory Sutherland, import it yourself manually jelu | 2023-12-02T20:22:52.186+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 162898, title A Connecticut Yankee in King Arthur's Court and author Mark Twain, import it yourself manually jelu | 2023-12-02T20:22:52.223+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 1257586, title Zahrah the Windseeker and author Nnedi Okorafor, import it yourself manually jelu | 2023-12-02T20:22:52.224+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 492773, title Nine Hundred Grandmothers and author R.A. Lafferty, import it yourself manually jelu | 2023-12-02T20:22:52.224+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 8466224, title Jonathan Strange & Mr Norrell and author Susanna Clarke, import it yourself manually jelu | 2023-12-02T20:22:52.260+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10257528, title To Kill a Mockingbird and author Harper Lee, import it yourself manually jelu | 2023-12-02T20:22:52.427+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 525304, title Star Maker and author Olaf Stapledon, import it yourself manually jelu | 2023-12-02T20:22:52.427+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 49838, title Titan (Gaea, #1) and author John Varley, import it yourself manually jelu | 2023-12-02T20:22:52.520+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 9460487, title Miss Peregrine's Home for Peculiar Children (Miss Peregrine's Peculiar Children, #1) and author Ransom Riggs, import it yourself manually jelu | 2023-12-02T20:22:52.580+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 1, title Harry Potter and the Half-Blood Prince (Harry Potter #6) and author J.K. Rowling, import it yourself manually jelu | 2023-12-02T20:22:52.714+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 12113357, title Mile 81 and author Stephen King, import it yourself manually jelu | 2023-12-02T20:22:52.733+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 17284577, title Guns and author Stephen King, import it yourself manually jelu | 2023-12-02T20:22:52.753+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 1713426, title Predictably Irrational: The Hidden Forces That Shape Our Decisions and author Dan Ariely, import it yourself manually jelu | 2023-12-02T20:22:52.842+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 11299, title After the Quake and author Haruki Murakami, import it yourself manually jelu | 2023-12-02T20:22:52.879+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 63697, title The Man Who Mistook His Wife for a Hat and Other Clinical Tales and author Oliver Sacks, import it yourself manually jelu | 2023-12-02T20:22:52.898+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 6289283, title Born to Run: A Hidden Tribe, Superathletes, and the Greatest Race the World Has Never Seen and author Christopher McDougall, import it yourself manually jelu | 2023-12-02T20:22:53.030+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 140671, title The Black Company (The Chronicles of the Black Company, #1) and author Glen Cook, import it yourself manually jelu | 2023-12-02T20:22:53.165+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10619, title Rose Madder and author Stephen King, import it yourself manually jelu | 2023-12-02T20:22:53.186+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 133266, title Four Past Midnight and author Stephen King, import it yourself manually jelu | 2023-12-02T20:22:53.186+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 10611, title The Eyes of the Dragon and author Stephen King, import it yourself manually jelu | 2023-12-02T20:22:53.205+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 233667, title Firestarter and author Stephen King, import it yourself manually jelu | 2023-12-02T20:22:53.205+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 107291, title Needful Things and author Stephen King, import it yourself manually jelu | 2023-12-02T20:22:53.255+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 5091, title The Dark Tower (The Dark Tower, #7) and author Stephen King, import it yourself manually jelu | 2023-12-02T20:22:53.345+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 43615, title The Gunslinger (The Dark Tower, #1) and author Stephen King, import it yourself manually jelu | 2023-12-02T20:22:53.411+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 13635342, title Snatched (Will Trent, #5.5) and author Karin Slaughter, import it yourself manually jelu | 2023-12-02T20:22:53.522+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 350, title Stranger in a Strange Land and author Robert A. Heinlein, import it yourself manually jelu | 2023-12-02T20:22:53.522+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 49552, title The Stranger and author Albert Camus, import it yourself manually jelu | 2023-12-02T20:22:53.522+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 136251, title Harry Potter and the Deathly Hallows (Harry Potter, #7) and author J.K. Rowling, import it yourself manually jelu | 2023-12-02T20:22:53.523+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 2, title Harry Potter and the Order of the Phoenix (Harry Potter, #5) and author J.K. Rowling, import it yourself manually jelu | 2023-12-02T20:22:53.523+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 6, title Harry Potter and the Goblet of Fire (Harry Potter, #4) and author J.K. Rowling, import it yourself manually jelu | 2023-12-02T20:22:53.542+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 15881, title Harry Potter and the Chamber of Secrets (Harry Potter, #2) and author J.K. Rowling, import it yourself manually jelu | 2023-12-02T20:22:53.542+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 157993, title The Little Prince and author Antoine de Saint-Exupéry, import it yourself manually jelu | 2023-12-02T20:22:53.613+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 4407, title American Gods (American Gods, #1) and author Neil Gaiman, import it yourself manually jelu | 2023-12-02T20:22:53.632+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : Missing isbn for line with goodreadsId 2122, title The Fountainhead and author Ayn Rand, import it yourself manually jelu | 2023-12-02T20:22:53.919+01:00 DEBUG 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : parsing finished, 473 entries recorded jelu | 2023-12-02T20:22:53.922+01:00 INFO 1 --- [ task-1] i.g.b.j.s.imports.CsvImportService : csv parsing of /files/imports/goodreads_library_export.csv ended after : 15 seconds jelu | 2023-12-02T20:28:12.770+01:00 WARN 1 --- [nio-1111-exec-7] o.s.web.servlet.PageNotFound : No mapping for GET /books ```
bayang commented 7 months ago

That's weird. Your config seem fine and the logs are the same as those I have after importing your file, except that on your side the processing seem to stop at one moment, after the csv parsing.

If you asked the import to fetch the metadata you can try to set the metadata logs to TRACE level to see if there is any activity : 

logging:
  level:
    io.github.bayang.jelu: DEBUG
    io.github.bayang.jelu.service.metadata: TRACE

This is what I have in my logs, it is the same as you : 

2023-12-02T18:44:51.824+01:00 DEBUG 21538 --- [task-1] i.g.b.j.s.imports.CsvImportService       : Missing isbn for line with goodreadsId 6, title Harry Potter and the Goblet of Fire (Harry Potter, #4) and author J.K. Rowling, import it yourself manually
2023-12-02T18:44:51.857+01:00 DEBUG 21538 --- [task-1] i.g.b.j.s.imports.CsvImportService       : Missing isbn for line with goodreadsId 15881, title Harry Potter and the Chamber of Secrets (Harry Potter, #2) and author J.K. Rowling, import it yourself manually
2023-12-02T18:44:51.859+01:00 DEBUG 21538 --- [task-1] i.g.b.j.s.imports.CsvImportService       : Missing isbn for line with goodreadsId 157993, title The Little Prince and author Antoine de Saint-Exupéry, import it yourself manually
2023-12-02T18:44:51.985+01:00 DEBUG 21538 --- [task-1] i.g.b.j.s.imports.CsvImportService       : Missing isbn for line with goodreadsId 4407, title American Gods (American Gods, #1) and author Neil Gaiman, import it yourself manually
2023-12-02T18:44:52.018+01:00 DEBUG 21538 --- [task-1] i.g.b.j.s.imports.CsvImportService       : Missing isbn for line with goodreadsId 2122, title The Fountainhead and author Ayn Rand, import it yourself manually
2023-12-02T18:44:52.605+01:00 DEBUG 21538 --- [task-1] i.g.b.j.s.imports.CsvImportService       : parsing finished, 473 entries recorded
2023-12-02T18:44:58.816+01:00  INFO 21538 --- [task-1] i.g.b.j.s.imports.CsvImportService       : csv parsing of /home/me/perso/jelu-data/imports/goodreads_library_export2.csv ended after : 50 seconds
2023-12-02T18:45:37.578+01:00 DEBUG 21538 --- [task-1] i.g.b.jelu.service.metadata.OpfParser    : parsed dto MetadataDto(title=Problém tří těles, isbn10=null, isbn13=9788075770615, summary=null, image=null, publisher=Host, pageCount=null, publishedDate=2017-12-15T17:45:23.104509+00:00, authors=[Cixin Liu], tags=[], series=null, numberInSeries=null, language=ces, googleId=X3NCswEACAAJ, amazonId=null, goodreadsId=null)
2023-12-02T18:45:52.819+01:00 DEBUG 21538 --- [task-1] i.g.b.jelu.service.metadata.OpfParser    : parsed dto MetadataDto(title=A Deepness in the Sky, isbn10=null, isbn13=9780812536355, summary=<p>After thousands of years searching, humans stand on the verge of first contact with an alien race. Two human groups: the Qeng Ho, a culture of free traders, and the Emergents, a ruthless society based on the technological enslavement of minds.</p>

etc...

Also, what architecture are you on ? ARM or not ? Can you manually test to import a book to check if the online metadata import is working ? (in add book -> auto fill from internet)

KucharczykL commented 7 months ago

My server is running Debian on amd64 architecture.

I've tried manually importing a book with ISBN 0060512806 - this worked 👍 I've tried importing the Goodreads export again, and this is stuck as before. I've enabled the more detailed logging:

jelu  | 2023-12-03T10:01:54.273+01:00 DEBUG 1 --- [         task-1] i.g.b.j.s.imports.CsvImportService       : parsing finished, 473 entries recorded
jelu  | 2023-12-03T10:01:54.276+01:00  INFO 1 --- [         task-1] i.g.b.j.s.imports.CsvImportService       : csv parsing of /files/imports/goodreads_library_export.csv ended after : 9 seconds
jelu  | 2023-12-03T10:01:55.018+01:00 TRACE 1 --- [         task-1] i.g.b.j.s.metadata.FetchMetadataService  : plugins to use : [PluginInfo(name=calibre, order=1000)]
jelu  | 2023-12-03T10:01:55.018+01:00 TRACE 1 --- [         task-1] i.g.b.j.s.metadata.FetchMetadataService  : fetching provider for plugin calibre with order 1000
jelu  | 2023-12-03T10:01:55.018+01:00 TRACE 1 --- [         task-1] i.g.b.j.s.m.p.CalibreMetadataProvider    : fetch metadata command : [/calibre/fetch-ebook-metadata, -o, -d 90, -i, 9780812536355, -p, Google, -p, Amazon.com, -c, /files/images/meta-import-9780812536355-1701594115018.jpg]

With the manual import, the metadata was brought almost immediately. With this, nothing 🤔 If I do ps aux | grep calibre:

root     1660102  0.5  1.7 723768 140208 ?       S    10:01   0:01 /calibre/bin/fetch-ebook-metadata -o -d 90 -i 9780812536355 -p Google -p Amazon.com -c /files/images/meta-import-9780812536355-1701594115018.jpg

According to the manual, the S means: S interruptible sleep (waiting for an event to complete)

KucharczykL commented 7 months ago

The ISBN 9780812536355 however doesn't work even manually! Maybe the fetch-ebook-metadata executable is choking on it for some reason?

KucharczykL commented 7 months ago

I've tried running the command inside the container: /calibre/fetch-ebook-metadata -i 9780812536355 and this worked.

bayang commented 7 months ago

first of all, thank you for debugging.

It looks like the calibre fetch-ebook-metadata executable is choking on some task and stops there.

What I don't get is that I explicitly set the -d 90 parameter to avoid this kind of problem since this it sets a timeout.

I would expect the command to return after 90 secs no matter what.

Can you try to add the timeout parameter to the command that worked to see if this parameter is broken for some reason :

/calibre/fetch-ebook-metadata -i 9780812536355 -d 90

What you can also try is to remove the line that blocks from your csv and relaunch an import, to see if that is this specific book that makes calibre choke or if we have a broader problem.

The ideal would be to try on a fresh database because the previous lines from your previous attempts are still in db waiting to be fetched.

If you don't want to start from a fresh DB I'll tell you how to change the status of the entries so that they are flagged as processed and ignored.

KucharczykL commented 7 months ago

Can you try to add the timeout parameter to the command that worked to see if this parameter is broken for some reason :

/calibre/fetch-ebook-metadata -i 9780812536355 -d 90

This worked without a problem.

What you can also try is to remove the line that blocks from your csv and relaunch an import, to see if that is this specific book that makes calibre choke or if we have a broader problem. The ideal would be to try on a fresh database because the previous lines from your previous attempts are still in db waiting to be fetched.

I have removed "Deepness in the Sky" (line 3) from the CSV, deleted my database, and started the import. This time it worked!

So that's curious that running the command manually from inside the container works, but as an import trigger by Jelu it gets stuck.

bayang commented 7 months ago

Ok that is a good news, I guess. There is probably something funny happening somewhere but I can't tell what. This is the first time this is mentioned in an issue, although I expect more weird stuff happening as long as we use calibre to fetch metadata (cf pinned issue). This calibre stuff is heavily rate limited on googles side and I guess also on amazon, this is why I'm adding other sources. The initial import is still the harder to get due to the volume of metadata to get.

KucharczykL commented 7 months ago

Would something like https://openlibrary.org/developers/api work?

bayang commented 7 months ago

Yes, it is definitely in the candidates for the future metadata plugins among others. I wanted calibre to start because as a non english speaker, this is the most comprehensive source of metadata. You can try to find the books you read in openlibrary for example, my guess is that you won't find that many references. Google books api is also available in jelu if you have an api key and you can also directly use your epubs as a source now.

Taubin commented 7 months ago

I just wanted to pop in and say this has been happening to me as well. I'm running the docker version on unraid. I was able to import a few days ago, but is broken when I attempt to import today. I exported the csv using calibre from my library and it wouldn't import.

I tried exporting just a list of isbn numbers to a csv file, and it didn't work either.

Before coming here, I even tried the sample that is provided by goodreads, and that fails as well: https://www.goodreads.com/assets/sample_export.csv

Here is the error using the sample file:

12/07/2023
12:38:33 PM
2023-12-06T15:38:33.098-08:00 DEBUG 1 --- [io-11111-exec-6] i.g.b.jelu.controllers.ImportController  : target import file at /files/imports/sample_export.csv
12/07/2023
12:38:33 PM
2023-12-06T15:38:33.138-08:00 DEBUG 1 --- [         task-8] i.g.b.j.s.imports.CsvImportService       : Missing isbn for line with goodreadsId memoirs of a geisha, title arthur golden and author 99498189, import it yourself manually
12/07/2023
12:38:33 PM
2023-12-06T15:38:33.139-08:00 DEBUG 1 --- [         task-8] i.g.b.j.s.imports.CsvImportService       : Missing isbn for line with goodreadsId Blink: The Power of Thinking Without Thinking, title Malcolm Gladwell and author 316172324, import it yourself manually
12/07/2023
12:38:33 PM
2023-12-06T15:38:33.139-08:00 DEBUG 1 --- [         task-8] i.g.b.j.s.imports.CsvImportService       : Missing isbn for line with goodreadsId Power of One, title Bryce Courtenay and author 034541005X, import it yourself manually
12/07/2023
12:38:33 PM
2023-12-06T15:38:33.139-08:00 DEBUG 1 --- [         task-8] i.g.b.j.s.imports.CsvImportService       : Missing isbn for line with goodreadsId Dune (Dune Chronicles Book 1), title Frank Herbert and author 441172717, import it yourself manually
12/07/2023
12:38:33 PM
2023-12-06T15:38:33.140-08:00 DEBUG 1 --- [         task-8] i.g.b.j.s.imports.CsvImportService       : parsing finished, 0 entries recorded
12/07/2023
12:38:33 PM
2023-12-06T15:38:33.140-08:00  INFO 1 --- [         task-8] i.g.b.j.s.imports.CsvImportService       : csv parsing of /files/imports/sample_export.csv ended after : 0 seconds
12/07/2023
12:38:33 PM
2023-12-06T15:38:33.143-08:00 DEBUG 1 --- [         task-8] i.g.b.j.s.imports.CsvImportService       : File /files/imports/sample_export.csv was successfully renamed after processing : true
12/07/2023
12:38:33 PM
2023-12-06T15:38:33.143-08:00  INFO 1 --- [         task-8] i.g.b.j.s.imports.CsvImportService       : Import for /files/imports/sample_export.csv ended after : 0 seconds, with 0 imports and 0

The attempted import takes less than a second so it's not a timeout issue.

Importing manually works fine, however I have a very large library and doing it all manually isn't feasible unfortunately.

bayang commented 7 months ago

@Taubin the goodreads sample csv does not look like the exports I've seen until now, I'll have a look, maybe goodreads changed their export format. That is why you the "missing isbn" logs, the parsing actually fails.

If you want you can send me your real csv and I can check to find what is wrong with it.

We have actually several cases in this issue :

Taubin commented 7 months ago

@bayang I'm so sorry, I screwed up. I was attempting to import my calibre export, not my goodreads export. I had accidentally overwritten my goodreads export with my calibre export. Sorry for adding clutter.

bayang commented 7 months ago

@Taubin don't worry, it happens to everyone.