"kernel modules must be upgraded to receive this stream." on 0.7.6

rincebrain commented 6 years ago

System information

Type	Version/Name
Distribution Name	Debian
Distribution Version	9.4
Linux Kernel	4.9.0-6-amd64
Architecture	x86_64
ZFS Version	0.7.6
SPL Version	0.7.6

(Using 0.7.6 with #7170 and #7048 cherrypicked; will try with 0.7.7 soon, but I didn't see any bugs that looked even vaguely similar in the changelog...)

Describe the problem you're observing

~5T dataset on a pool, tried a zfs send -Recv | zfs recv -ds, it mostly finished when it eventually reported "cannot receive incremental stream: kernel modules must be upgraded to receive this stream."

Using the resume token immediately reports the same.

Describe how to reproduce the problem

Evidently, just zfs send -Recv 5.5T of this 5.54T referenced dataset.

Include any warning/errors/backtraces from the system logs

dbgmsg.log

dmesg and kern.log don't contain anything interesting.

Oh, yes, feature flags. source pool:

monolith  feature@async_destroy          enabled                        local
monolith  feature@empty_bpobj            active                         local
monolith  feature@lz4_compress           active                         local
monolith  feature@multi_vdev_crash_dump  disabled                       local
monolith  feature@spacemap_histogram     active                         local
monolith  feature@enabled_txg            active                         local
monolith  feature@hole_birth             active                         local
monolith  feature@extensible_dataset     active                         local
monolith  feature@embedded_data          active                         local
monolith  feature@bookmarks              active                         local
monolith  feature@filesystem_limits      enabled                        local
monolith  feature@large_blocks           active                         local
monolith  feature@large_dnode            disabled                       local
monolith  feature@sha512                 disabled                       local
monolith  feature@skein                  disabled                       local
monolith  feature@edonr                  disabled                       local
monolith  feature@userobj_accounting     disabled                       local

destination pool:

workspace  feature@async_destroy          enabled                        local
workspace  feature@empty_bpobj            active                         local
workspace  feature@lz4_compress           active                         local
workspace  feature@multi_vdev_crash_dump  disabled                       local
workspace  feature@spacemap_histogram     active                         local
workspace  feature@enabled_txg            active                         local
workspace  feature@hole_birth             active                         local
workspace  feature@extensible_dataset     active                         local
workspace  feature@embedded_data          active                         local
workspace  feature@bookmarks              active                         local
workspace  feature@filesystem_limits      enabled                        local
workspace  feature@large_blocks           enabled                        local
workspace  feature@large_dnode            disabled                       local
workspace  feature@sha512                 disabled                       local
workspace  feature@skein                  disabled                       local
workspace  feature@edonr                  disabled                       local
workspace  feature@userobj_accounting     disabled                       local

rincebrain commented 6 years ago

Unsurprisingly, 0.7.7 also reproduces this.

Here's the dbgmsg from that (I built 0.7.7 with --enable-debug) with zfs_flags=255.

dbgmsg_dprintf2.log

rincebrain commented 6 years ago

Clearly, my memory is garbage.

Closing; duplicate of #7024

openzfs / zfs