search

Seagate / cortx-motr

CORTX Motr is a distributed object and key-value storage system targeting mass capacity storage configurations. It's the core component of CORTX storage system.
https://github.com/Seagate/cortx
Apache License 2.0
59 stars 142 forks source link

m0cp write need to create .img file first? #10

Closed yanqingfu closed 3 years ago

yanqingfu commented 4 years ago

followed https://github.com/Seagate/cortx/blob/main/doc/Cluster_Setup.md#16-perform-the-io

do we need to create .img file first?

[root@ssc-vm-1091 cortx-hare]# hctl status Profile: 0x7000000000000001:0x22 Data pools: 0x6f00000000000001:0x23 Services: localhost (RC) [started] hax 0x7200000000000001:0x6 10.230.247.138@tcp:12345:1:1 [started] confd 0x7200000000000001:0x9 10.230.247.138@tcp:12345:2:1 [started] ioservice 0x7200000000000001:0xc 10.230.247.138@tcp:12345:2:2 [unknown] m0_client 0x7200000000000001:0x1c 10.230.247.138@tcp:12345:4:1 [unknown] m0_client 0x7200000000000001:0x1f 10.230.247.138@tcp:12345:4:2

[root@ssc-vm-1091 cortx-hare]# cortx/cortx-motr/motr/st/utils/m0cp -l 10.230.247.138@tcp:12345:4:1 -H 10.230.2 47.138@tcp:12345:1:1 \

 -p 0x7000000000000001:0x22 -P 0x7200000000000001:0x1c -o 12:10 \
 -s 1m -c 128 /var/misc/random.img -L 9

-bash: cortx/cortx-motr/motr/st/utils/m0cp: No such file or directory [root@ssc-vm-1091 cortx-hare]# /var/cortx/cortx-motr/motr/st/utils/m0cp -l 10.230.247.138@tcp:12345:4:1 -H 10. 230.247.138@tcp:12345:1:1 -p 0x7000000000000001:0x22 -P 0x7200000000000001:0x1c -o 12:10 -s 1m -c 12 8 /var/misc/random.img -L 9 m0_write failed! Object id 12:10rc = -1 [root@ssc-vm-1091 cortx-hare]# /var/cortx/cortx-motr/motr/st/utils/m0cp -l 10.230.247.138@tcp:12345:4:1 -H 10. 230.247.138@tcp:12345:1:1 -p 0x7000000000000001:0x22 -P 0x7200000000000001:0x1c -o 14:10 -s 1m -c 12 8 /var/misc/random.img -L 9 m0_write failed! Object id 14:10rc = -1

madhavemuri commented 4 years ago

Yes, random.img is for reference purpose only, here any source file which is to be stored needs to be used.

yanqingfu commented 4 years ago

@madhavemuri write error

[root@ssc-vm-1091 var]# cd motr [root@ssc-vm-1091 motr]# ls disk0.img disk2.img disk4.img disk6.img disk8.img hax m0d-0x7200000000000001:0xc m0ut disk1.img disk3.img disk5.img disk7.img disk9.img m0d-0x7200000000000001:0x9 m0spiel.pkg.files root [root@ssc-vm-1091 motr]# cd .. [root@ssc-vm-1091 var]# cortx/cortx-motr/motr/st/utils/m0cp -l 10.230.247.138@tcp:12345:4:1 -H 10.230.247.138@tcp:12345:1:1 -p 0x7000000000000001:0x22 -P 0x7200000000000001:0x1c -o 14:10 -s 1m -c 128 motr/disk6.img -L 9 motr[00515]: ea80 ERROR [io_req_fop.c:208:io_bottom_half] ioo=0x1393810 from=10.230.247.138@tcp:12345:2:2 rc=-7 ti_rc=-7 @<4300000300000014:10> motr[00515]: eab0 ERROR [io_req.c:538:ioreq_iosm_handle_executed] iro_dgmode_write() failed, rc=-7

yanqingfu commented 4 years ago

Yes, random.img is for reference purpose only, here any source file which is to be stored needs to be used.

text file .txt also could be used?

yanqingfu commented 4 years ago

[root@ssc-vm-1091 var]# cortx/cortx-motr/motr/st/utils/m0cp -l 10.230.247.138@tcp:12345:4:1 -H 10.230.247.138@tcp:12345:1:1 -p 0x7000000000000001:0x22 -P 0x7200000000000001:0x1c -o 17:10 -s 1m -c 128 motr/random.txt -L 9 motr[09471]: 72e0 FATAL [lib/assert.c:50:m0_panic] panic: (rc == bcount) at m0_write() (motr/st/utils/helper.c:431) [git: cortx1.0-rc1-235-g4d78776] /var/m0trace.9471 Motr panic: (rc == bcount) at m0_write() motr/st/utils/helper.c:431 (errno: 0) (last failed: none) [git: cortx1.0-rc1-235-g4d78776] pid: 9471 /var/m0trace.9471 /var/cortx/cortx-motr/motr/.libs/libmotr.so.1(m0_arch_backtrace+0x20)[0x7f16fdd972c0] /var/cortx/cortx-motr/motr/.libs/libmotr.so.1(m0_arch_panic+0xe6)[0x7f16fdd97476] /var/cortx/cortx-motr/motr/.libs/libmotr.so.1(+0x34d6d4)[0x7f16fdd866d4] /var/cortx/cortx-motr/motr/st/utils/.libs/lt-m0cp[0x4026fb] /var/cortx/cortx-motr/motr/st/utils/.libs/lt-m0cp(main+0xad)[0x401acd] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f16fc537555] /var/cortx/cortx-motr/motr/st/utils/.libs/lt-m0cp[0x401b92] Aborted (core dumped)

madhavemuri commented 4 years ago

-s 1m -c 128 Make sure that size of the source file ("motr/random.txt") is greater than equal to (size count) i.e 128M (1m 128) here

madhavemuri commented 4 years ago

Also -7 errors means E2BIG error, this means special handling is needed for them. As you have few devices in the setup, increasing them will resolve the issue. Or I would recommend to use lower than 100 value like "-b 30 "to the command., -b, --blocks-per-io (default is 100)

huanghua78 commented 4 years ago

@madhavemuri write error

[root@ssc-vm-1091 var]# cd motr [root@ssc-vm-1091 motr]# ls disk0.img disk2.img disk4.img disk6.img disk8.img hax m0d-0x7200000000000001:0xc m0ut disk1.img disk3.img disk5.img disk7.img disk9.img m0d-0x7200000000000001:0x9 m0spiel.pkg.files root [root@ssc-vm-1091 motr]# cd .. [root@ssc-vm-1091 var]# cortx/cortx-motr/motr/st/utils/m0cp -l 10.230.247.138@tcp:12345:4:1 -H 10.230.247.138@tcp:12345:1:1 -p 0x7000000000000001:0x22 -P 0x7200000000000001:0x1c -o 14:10 -s 1m -c 128 motr/disk6.img -L 9 motr[00515]: ea80 ERROR [io_req_fop.c:208:io_bottom_half] ioo=0x1393810 from=10.230.247.138@tcp:12345:2:2 rc=-7 ti_rc=-7 @<4300000300000014:10> motr[00515]: eab0 ERROR [io_req.c:538:ioreq_iosm_handle_executed] iro_dgmode_write() failed, rc=-7

These "disk0.img disk2.img disk4.img disk6.img disk8.img" are files to store data for the Motr services!!! These are "devices" managed by Motr and clovis app is writing data to them. You are reading these disks and Motr is writing to themselves!!

THIS IS ABSOLUTELY WRONG.

madhavemuri commented 4 years ago

@huanghua78 : Good obeservation, i missed it. But i think in this case @yanqingfu may be using actual devices in cluster.yaml file.

yanqingfu commented 4 years ago

@madhavemuri @huanghua78 as you can see from singlenode.yaml I am using actual devices

I used command

 for i in {0..9}; do
    sudo dd if=/dev/zero of=/var/motr/disk$i.img bs=1M seek=9999 count=1
    sudo losetup /dev/loop$i /var/motr/disk$i.img
done

to create the disk*.img file

nodes: hostname: localhost data_iface: eth0 m0_servers: runs_confd: true io_disks: data: [] io_disks:

data:

  • /dev/vde
  • /dev/vdf
  • /dev/vdg
  • /dev/vdh m0_clients: s3: 0 other: 2 pools: name: the pool data_units: 2 parity_units: 1
justinzw commented 4 years ago

Transferring this to the Motr repo. Let me know if there are any issues. Thank you.

stale[bot] commented 3 years ago

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @huanghua78 @mukundkanekar for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

huanghua78 commented 3 years ago

This issue is not seen any more.