Incorporating Blob with ClientStorage

[CtheCondor]

Is it possible to incorporate blob media in a ClientStorage using OOBTrees? Trying to store list of blobs representing reports that are mapped to a given string for storage and retrieval.

[d-maurer]

CtheCondor wrote at 2022-8-8 16:53 -0700:

Is it possible to incorporate blob media in a ClientStorage using OOBTrees?

It is. You can use a blob (-> ZODB.blob.Blob) whereever you can use a persistent object.

[CtheCondor]

What I tried(that failed) was this.

with db.transaction() as conn2:
    root=conn2.root()
    root['user']['report']=Btree()
    report=blob.Blob()
    with report.open('w') as f:
        f.write(b'This is a test')
    root['user']['report'].update({'report1":report})
    transaction.commit()

What I get is a "AttributeError: 'NoneType' Object has no attribute 'getPathForOID'" did I make a mistake since I am trying to place a blob inside a persistent object?

[d-maurer]

CtheCondor wrote at 2022-8-9 06:41 -0700:

What I tried(that failed) was this.

What happened (precisely)? What exception/traceback did you get? The exceptions/error messages are good (most of the time). It is worth to read them carefully.

root['user']['report']=Btree() report=blob.Blob() with report.open('w') as f: f.write(b'This is a test') root['user']['report'].update({'report1":report}) transaction.commit()

did I make a mistake since I am trying to place a blob inside a persistent object?

I tried to reproduce a variant of your code locally and got:

     ...
     File "/home/dieter/tmp/ZODB/src/ZODB/Connection.py", line 572, in _store_objects
    repr(self._storage))
ZODB.POSException.Unsupported: Storing Blobs in <ZODB.mvccadapter.MVCCAdapterInstance object at 0x7f55b01999b0> is not supported.

Is this the exception you have seen?

The code behind this exception is:

            if isinstance(obj, Blob):
                if not IBlobStorage.providedBy(self._storage):
                    raise Unsupported(
                        "Storing Blobs in %s is not supported." %
                        repr(self._storage))

This means: you can store a blob only in a storage which supports blobs. A FileStorage supports blobs only if it is configured to do so (not the case for my test).

If you see the same exception, verify the your storage is configured to support blobs.

[CtheCondor]

Configuration:

% server.config
<zeo>
   address :8090
</zeo>
<filestorage>
   path C:\ZODB\test.fs
   blob-dir C:\ZODB\blob\
</filestorage>

% client.config
<zodb>
   %import ZEO
    <ClientStorage>
       server localhost:8090
       wait-timeout 4
     </ClientStorage>
</zodb>

Above code revised to include error: "AttributeError: 'NoneType' Object has no attribute 'getPathForOID'"

[d-maurer]

CtheCondor wrote at 2022-8-9 07:42 -0700:

... Above code revised to include error: "AttributeError: 'NoneType' Object has no attribute 'getPathForOID'"

When you report exceptions, you should always include the associated traceback.

I tried to reproduce your problem -- and failed:

>>> c=app._p_jar
>>> r=c.root()
>>> from BTrees.OOBTree import OOBTree
>>> r["reports"] = rps = OOBTree()
>>> from ZODB.blob import Blob
>>> b = Blob()
>>> with b.open("w") as f: f.write(b"Hello")
...
5
>>> 5
5
>>> rps["r"] = b
>>> from transaction import commit; commit()
>>> del b
>>> b = rps["r"]
>>> with b.open("r") as f: print(f.read())
...
b'Hello'

[CtheCondor]

I am unable to copy the full traceback information as I am working on a locked intranet only computer and I am retyping everything here for questions.

I copied your effort with the exception of "c=app._p_jar" which I did not understand nor do I have an equivalent setup in my program and NOW received the "ZODB.POSException.Unsupported: Storing Blobs in ZODB.mvccadapter.MVCCAdapterInstance object at ###> is not supported." that you expected me to have in your original solution.

Am I missing configuration options in my config's above to enable Blob storage? I have the "blob-dir" option within the filesystem tags for the server config and I though that was all that was necessary?

[d-maurer]

CtheCondor wrote at 2022-8-9 09:45 -0700:

... I copied your effort with the exception of "c=app._p_jar" which I did not understand nor do I have an equivalent setup in my program

I have a ready to use ZODB in the context of a Zope application. The app is Zope's application object; app._p_jar is the corresponding ZODB connection.

You get your ZODB connection in a different way, but this should not matter.

and NOW received the "ZODB.POSException.Unsupported: Storing Blobs in ZODB.mvccadapter.MVCCAdapterInstance object at ###> is not supported." that you expected me to have in your original solution.

Am I missing configuration options in my config's above to enable Blob storage? I have the "blob-dir" option within the filesystem tags for the server config and I though that was all that was necessary?

With a zeoclient, the client, too, would like to have a blob-dir (used either as primary blob storage (with shared-blob-dir on) or as blob cache (with shared-blob-dir off)).

[CtheCondor]

Apologize for leaving this open for so long. What finally worked for me was the following: server.config

<zeo>
    address :8090
</zeo>
<filestorage>
   path U:\sharedfiles\server.fs
   blob-dir U:\sharedfiles\blobs
<\filestorage>

client.config

<zodb>
   %import ZEO
   <ClientStorage>
      server localhost:8090
      wait-timeout 4
      blob-dir $UserPath\blobs
      shared-blob-dir off
   <ClientStorage>
</zodb>

These configurations were able to allow for the server to connect and for blobs to be added to Persistent objects; namely OOBTree but also the Persist.List.PersistLists.

Thank you for your help and hope this is useful for others if needed.