Crawl Images issue

[altomator]

Hi all,

I'm trying out Vikus after seeing it demonstrated at the IIIF Napoli conference last June.

When I run the "Generate instance" on the default collection: https://iiif.wellcomecollection.org/presentation/collections/genres/Stickers

or any of mine, like this one: https://mirador.bnf.fr/?iiif-content=https://raw.githubusercontent.com/altomator/IIIF/main/collection/collection-bd.json

it failed at the CrawlImages step: [2023-11-18 17:24:00 +0000] [685] [INFO] connection open vikus-iiif-generator-main-vikusdocker-1 | this function is a coroutine: crawlCollection vikus-iiif-generator-main-redis-1 | 1:M 18 Nov 2023 17:24:33.956 1 changes in 3600 seconds. Saving... vikus-iiif-generator-main-redis-1 | 1:M 18 Nov 2023 17:24:33.960 Background saving started by pid 20 vikus-iiif-generator-main-redis-1 | 20:C 18 Nov 2023 17:24:33.980 DB saved on disk vikus-iiif-generator-main-redis-1 | 20:C 18 Nov 2023 17:24:33.981 Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB vikus-iiif-generator-main-redis-1 | 1:M 18 Nov 2023 17:24:34.067 Background saving terminated with success vikus-iiif-generator-main-vikusdocker-1 | crawlCollection took 2.0 seconds vikus-iiif-generator-main-vikusdocker-1 | this function is a coroutine: crawlImages vikus-iiif-generator-main-vikusdocker-1 | crawlImages took 0.0023 seconds vikus-iiif-generator-main-vikusdocker-1 | this function is a coroutine: makeSpritesheets vikus-iiif-generator-main-vikusdocker-1 | ['/modules/sharpsheet/bin/sharpsheet', '/data/juicy-omelette/images/thumbs/.jpg', '--outputPath', '/data/juicy-omelette/images/sprites', '--outputFormat', 'jpg', '--outputQuality', '60', '--sheetDimension', '2048', '--spriteSize', '224'] vikus-iiif-generator-main-vikusdocker-1 | 17:24:36 INFO INFO:rich:stdout: sharpsheet.py:42 vikus-iiif-generator-main-vikusdocker-1 | looking for images at
vikus-iiif-generator-main-vikusdocker-1 | /data/juicy-omelette/images/thumbs/*.jpg
vikus-iiif-generator-main-vikusdocker-1 | done
vikus-iiif-generator-main-vikusdocker-1 |
vikus-iiif-generator-main-vikusdocker-1 | ERROR ERROR:rich:error generating spritesheet: no sharpsheet.py:45 vikus-iiif-generator-main-vikusdocker-1 | images found
vikus-iiif-generator-main-vikusdocker-1 |
vikus-iiif-generator-main-vikusdocker-1 | makeSpritesheets took 0.18 seconds vikus-iiif-generator-main-vikusdocker-1 | this function is a coroutine: makeFeatures vikus-iiif-generator-main-vikusdocker-1 | 0/0 images found in cache vikus-iiif-generator-main-vikusdocker-1 | makeFeatures took 1.2 seconds vikus-iiif-generator-main-vikusdocker-1 | this function is a coroutine: makeUmap vikus-iiif-generator-main-vikusdocker-1 | 17:24:39 INFO INFO:umaper:Fit UMAP dimensionReduction.py:30 vikus-iiif-generator-main-vikusdocker-1 | INFO INFO:umaper:X shape: (0,) dimensionReduction.py:32 vikus-iiif-generator-main-vikusdocker-1 | [2023-11-18 17:24:39 +0000] [572] [ERROR] Exception in ASGI application vikus-iiif-generator-main-vikusdocker-1 | Traceback (most recent call last): vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi vikus-iiif-generator-main-vikusdocker-1 | result = await app( # type: ignore[func-returns-value] vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in call vikus-iiif-generator-main-vikusdocker-1 | return await self.app(scope, receive, send) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 1106, in call vikus-iiif-generator-main-vikusdocker-1 | await super().call(scope, receive, send) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 122, in call vikus-iiif-generator-main-vikusdocker-1 | await self.middleware_stack(scope, receive, send) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in call vikus-iiif-generator-main-vikusdocker-1 | raise exc vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in call vikus-iiif-generator-main-vikusdocker-1 | await self.app(scope, receive, _send) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 91, in call vikus-iiif-generator-main-vikusdocker-1 | await self.simple_response(scope, receive, send, request_headers=headers) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 146, in simple_response vikus-iiif-generator-main-vikusdocker-1 | await self.app(scope, receive, send) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 79, in call vikus-iiif-generator-main-vikusdocker-1 | raise exc vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 68, in call vikus-iiif-generator-main-vikusdocker-1 | await self.app(scope, receive, sender) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in call vikus-iiif-generator-main-vikusdocker-1 | raise e vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in call vikus-iiif-generator-main-vikusdocker-1 | await self.app(scope, receive, send) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 718, in call vikus-iiif-generator-main-vikusdocker-1 | await route.handle(scope, receive, send) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 276, in handle vikus-iiif-generator-main-vikusdocker-1 | await self.app(scope, receive, send) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 66, in app vikus-iiif-generator-main-vikusdocker-1 | response = await func(request) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 274, in app vikus-iiif-generator-main-vikusdocker-1 | raw_response = await run_endpoint_function( vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 191, in run_endpoint_function vikus-iiif-generator-main-vikusdocker-1 | return await dependant.call(values) vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/main.py", line 411, in run vikus-iiif-generator-main-vikusdocker-1 | await make_umap( vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/main.py", line 351, in make_umap vikus-iiif-generator-main-vikusdocker-1 | await makeUmap( vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/helpers.py", line 37, in wrapper vikus-iiif-generator-main-vikusdocker-1 | result = await helper(func, *args, *kwargs) vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/helpers.py", line 29, in helper vikus-iiif-generator-main-vikusdocker-1 | return await func(args, kwargs) vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/vikus.py", line 208, in makeUmap vikus-iiif-generator-main-vikusdocker-1 | embedding = umaper.fit_transform(features) vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/dimensionReduction.py", line 33, in fit_transform vikus-iiif-generator-main-vikusdocker-1 | embeddings = self.umap.fittransform(X) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/umap/umap.py", line 2887, in fit_transform vikus-iiif-generator-main-vikusdocker-1 | self.fit(X, y, force_allfinite) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/umap/umap.py", line 2354, in fit vikus-iiif-generator-main-vikusdocker-1 | X = check_array(X, dtype=np.float32, accept_sparse="csr", order="C", force_all_finite=force_all_finite) vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/sklearn/utils/validation.py", line 938, in check_array vikus-iiif-generator-main-vikusdocker-1 | raise ValueError( vikus-iiif-generator-main-vikusdocker-1 | ValueError: Expected 2D array, got 1D array instead: vikus-iiif-generator-main-vikusdocker-1 | array=[]. vikus-iiif-generator-main-vikusdocker-1 | Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.

[cpietsch]

Hi @altomator, I will have a look at this issue. It looks like the api at wellcomecollection differs from the time the code was created.

[altomator]

Thanks a lot!But I failed too to use it on my IIIF collections.All the best,JpEnvoyé de mon iPhoneLe 24 nov. 2023 à 11:55, Christopher Pietsch @.***> a écrit : Hi @altomator, I will have a look at this issue. It looks like the api at wellcomecollection differs from the time the code was created.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>