Replication of multiple (nested) dataset

[Peter2121]

I need to replicate some nested datasets. There is a 'root' dataset zroot/jails and multiple child datasets like zroot/jais/mail, zroot/jais/web etc. Following #165 I abandoned the idea of nested replication, so I try to configure separately the replication of every dataset. The problem is that I need to replicate several datasets inside zroot/jails AND zroot/jails itself (there are some configuration data and system roots there). And when I configure the replication of zroot/jails - the ZFS parameters of this dataset, added by zrep, are propagated to all child datasets, so when I try to configure a child dataset - I get an error from zrep:

zroot/jais/mail is at least partially configured by zrep
Error: zrep:master detected!!

Evidently, zrep clear zroot/jais/mail does not help here as the ZFS parameters, set by zrep, are inherited from zroot/jails. A workaround - configure firstly all child datasets and then the root dataset - works correctly, but when I need to add a new child dataset or reconfigure an old one - I hit the same problem. How can I deal with this problem?

[ppbrown]

Maybe use tags. use ZREP_TAG=zreproot (or something) for the top level. Then all the rest will be "normal" and wont see the zrep use of the top level.

On Fri, Feb 5, 2021 at 1:55 AM Пётр notifications@github.com wrote:

I need to replicate some nested datasets. There is a 'root' dataset zroot/jails and multiple child datasets like zroot/jais/mail, zroot/jais/web etc. Following #165 https://github.com/bolthole/zrep/issues/165 I abandoned the idea of nested replication, so I try to configure separately the replication of every dataset. The problem is that I need to replicate several datasets inside zroot/jails AND zroot/jails itself (there are some configuration data and system roots there). And when I configure the replication of zroot/jails - the ZFS parameters of this dataset, added by zrep, are propagated to all child datasets, so when I try to configure a child dataset - I get an error from zrep:

zroot/jais/mail is at least partially configured by zrep Error: zrep:master detected!!

Evidently, zrep clear zroot/jais/mail does not help here as the ZFS parameters, set by zrep, are inherited from zroot/jails. A workaround - configure firstly all child datasets and then the root dataset - works correctly, but when I need to add a new child dataset or reconfigure an old one - I hit the same problem. How can I deal with this problem?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bolthole/zrep/issues/166, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANEV6PU7CYAVTCGJFUWHSDS5O6C3ANCNFSM4XEO4PCA .

[Peter2121]

If I understand correctly, exporting ZREP_TAG=zreproot should change the prefix of ZFS parameters of the dataset? It is not the case for my current version, the root dataset is reinitialized with zrep:... parameters despite ZREP_TAG=zreproot set in the environment.

[ppbrown]

I dont think you have thought through the implementation details.

if you want to migrate your existing setup, then you have to do some manual things, like either converting an existing top-level-only snapshot to a ZREP starting point, or creating a new one The trouble is I'm not sure what happens if you try to do a "zfs send and receive" (notice: zfs, not zrep) on a top level ZFS filesystem, without affecting pre-existing child filesystems.

It may be that it is just not possible with ZFS. in which case there is nothing zrep can do on top of that.

However, if you were to start over, I am confident that the TAG method would work for you.

On Tue, Feb 9, 2021 at 12:04 PM Пётр notifications@github.com wrote:

If I understand correctly, exporting ZREP_TAG=zreproot should change the prefix of ZFS parameters of the dataset? It is not the case for my current version, the root dataset is reinitialized with zrep:... parameters despite ZREP_TAG=zreproot set in the environment.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bolthole/zrep/issues/166#issuecomment-776205892, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANEV6LUEN2W23GIESRNFVDS6GINLANCNFSM4XEO4PCA .

[Peter2121]

@ppbrown I don't try to use or migrate the existing setup. I removed the existing configuration for the 'root' dataset with zrep clear, then I added a new jail dataset with zrep -i (successfully), then I configured ZREP_TAG with the command export ZREP_TAG=zreproot (I'm using zsh). The environment variable is visible in env. Then I tried to configure the replication of 'root' dataset with zrep -i. The replication was successfully finished, but if I do zfs get all on the 'root' dataset - I see zrep:master yes local etc. What did I miss here?

[ppbrown]

I apologise: i pulled that variable name from memory. It appears I have been inconsistent in naming. It really should be "ZREP_TAG", but the actual variable is apparently "ZREPTAG". Please try again using that.

I shall have to update the code to be consistent and support both spellings

On Wed, Feb 10, 2021 at 12:59 AM Пётр notifications@github.com wrote:

@ppbrown https://github.com/ppbrown I don't try to use or migrate the existing setup. I removed the existing configuration for the 'root' dataset with zrep clear, then I added a new jail dataset with zrep -i (successfully), then I configured ZREP_TAG with the command export ZREP_TAG=zreproot (I'm using zsh). The environment variable is visible in env. Then I tried to configure the replication of 'root' dataset with zrep -i. The replication was successfully finished, but if I do zfs get all on the 'root' dataset

• I see zrep:master yes local etc. What did I miss here?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bolthole/zrep/issues/166#issuecomment-776550915, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANEV6I6YYF7FPXCDTSA6Z3S6JDI3ANCNFSM4XEO4PCA .

[Peter2121]

With ZREPTAG it almost works, but, as the 'child' datasets inherit the zfs parameters from the root one, during snapshots cleanup their snapshots are accidentally deleted, so the replication of child snapshots is broken following the cleanup of snapshots of the 'root' dataset. As I cannot block the inheritance, the system remains really fragile and it is impossible to configure the different number of snapshots to save for different datasets.

[ppbrown]

hmm. that’s surprising. could you help me out by doing your own simplified test case, and uploading the detailed results please? very important to include would be:

1. exact arguments used for init and then sync runs

2. showing me the state on things with zfs list -t all

3. making sure you do NOT have the -R option set. no -R in flags and no ZREP_R

[Peter2121]

It seems that the problem is more complicated than I thought. I created a simple 'lab' setup, and I cannot reproduce the problem here. I definitely need to digg more...