BCDB import/export [development_importexport]

waleedhammam commented 4 years ago

Testing on one bcdb instance

create some data


s= """@url = world.ship  
n = 0 (I)  
sub = (O) !world.ship.window

@url = world.ship.window sn = 1 (I)"""

h = j.data.bcdb.get("hellobcdb") m = h.model_get(schema=s) o = h.models.world__ship.new()
o.n = 1 o.sub.sn = 4 o.save()

- check the data
`j.data.bcdb.get("hellobcdb").get_all() # will be there`

- Export the data
`j.data.bcdb.hellobcdb.export(path="/test_export/") `

- now try to import
`j.data.bcdb.get("hellobcdb").import_("/test_export")`
This will return empty result cause in the code it looks for another files extensions in the encrypted state. it will work if we pass  encrypt=False in exporting.

**Testing on multiple bcdb instances**

`j.data.bcdb.export(path="/test_export")`  This will work OK and data is exported but when importing using:
`j.data.bcdb.import_()`

will give us this error:
```bash
JSX> j.data.bcdb.import_()                                                                               
no server running need to start
++ '[' start == kill ']'
++ tmux -f /sandbox/cfg/.tmux.conf has-session -t main
no server running on /tmp/tmux-0/default
++ '[' 1 -eq 1 ']'
++ echo 'no server running need to start'
++ tmux -f /sandbox/cfg/.tmux.conf new -s main -d 'bash --rcfile /sandbox/bin/env_tmux_detach.sh'
++ '[' start '!=' start ']'
Thu 09 16:56:11 /bcdb/BCDBFactory.py - 474 - get_for_threebot                   : EXCEPTION: 
    ttype can only be zdb or sqlite
--TRACEBACK------------------
<stdin> in <module>
    1    
/sandbox/lib/jumpscale/Jumpscale/data/bcdb/BCDBFactory.py in import_
    363    self.import_(name, path=path)
/sandbox/lib/jumpscale/Jumpscale/data/bcdb/BCDBFactory.py in import_
    378    bcdb = self.get_for_threebot(name, namespace=config["namespace"], ttype=config["type"])
/sandbox/lib/jumpscale/Jumpscale/data/bcdb/BCDBFactory.py in get_for_threebot
    474    raise j.exceptions.Input("ttype can only be zdb or sqlite")
-----------------------------

None

serboctor commented 4 years ago

@despiegk I checked the history, and it seems you did some changes to export/import today. I can see that when the data is encrypted the extension you're exporting to is different from the extension you check on while importing.

waleedhammam commented 4 years ago

Encountered bug:

1- Do j.data.bcdb.export() then j.data.bcdb.import_()

gives namespace error key not found when namespace = sdb

in this example was myjobs db

waleedhammam commented 4 years ago

Encountered bug 2:

If setting unknown namespace with none gives this error

waleedhammam commented 4 years ago

Encountered bug 3:

on another machine I found this error

Screenshot from 2020-01-13 16-52-02

despiegk commented 4 years ago

is the next_internal_id always available?

waleedhammam commented 4 years ago

Yes, it was added in zdb 2 weeks ago or so that zdb knows the next_internal_id

serboctor commented 4 years ago

@waleedhammam for bug 3 this simply means that this machine doesnt have the latest zdb. A reinstall would fix it.

waleedhammam commented 4 years ago

@waleedhammam for bug 3 this simply means that this machine doesnt have the latest zdb. A reinstall would fix it.

Yes, Thanks

waleedhammam commented 4 years ago

Updates

Currently if we are using zdb as backend namespace not found error is encountered, this happens because only admin client can create namespace, in the import script it's normal client Screenshot from 2020-01-16 15-28-06