gboudreau / Greyhole

Greyhole uses Samba to create a storage pool of all your available hard drives, and allows you to create redundant copies of the files you store.
http://www.greyhole.net
GNU General Public License v3.0
263 stars 34 forks source link

Can't add drive to drive selection group! #318

Open vonj opened 1 year ago

vonj commented 1 year ago

I have read up and down and can't understand how to add a third drive.

I had two disks and I want to add a third, since the two are running low on space.

greyhole -s shows

/data/WXX4T: 3666G - 3599G = 67G + 0G = 67G /data/WYY4T: 3666G - 3599G = 67G + 0G = 68G /data/USB5T: Offline

I have tried to tell greyhole -r /data/USB5T but nothing changes.

When I copy a new file to the share and look in the log, I see

Drives with available space: /data/WXX4T (57.4GB avail) - /data/WYY4T (57.3GB avail) [2867144]

but not the third, new drive!

In the config file I have:

drive_selection_groups = OK: /data/USB5T, /data/WXX4T, /data/WYY4T drive_selection_algorithm = most_available_space drive_selection_algorithm = forced (3xOK) most_available_space

The new drive is always offline.

gboudreau commented 1 year ago

Hi,

Please try this and report back: https://github.com/gboudreau/Greyhole/wiki/GetHelp#gather-system-information This will provide helpful debugging details.

I have a suspicion that on daemon start, it should log something relevant about this problem in the greyhole (error) log. You can also just look there manually, if you want.

Cheers.

vonj commented 1 year ago

I will try that. Another things - is it normal to:

select count(*) from settings ;

return 250 rows? They all look the same to me.

Btw, https://bit.ly/gh-infos is HTTP 403 Forbidden

gboudreau commented 1 year ago

I have 16 rows in the settings table here. Are you using sticky folders? I think that's the only thing that would add more rows in settings. If you're unsure, you can export that data, and email it to me at guillaume@greyhole.net

gboudreau commented 1 year ago

Btw, https://bit.ly/gh-infos is HTTP 403 Forbidden

Not here. I just tried in a CLI, and it load correctly :

$ curl -L 'https://bit.ly/gh-infos'
#!/bin/sh

{
echo "## System information"
...
gboudreau commented 1 year ago

Btw, https://bit.ly/gh-infos is HTTP 403 Forbidden

You can try the URL it redirects to: https://gist.githubusercontent.com/gboudreau/1caa58d93b1f99ca8e57/raw/

vonj commented 1 year ago

Btw if I do this:

touch /data/USB5T/.greyhole_uses_this

then greyhole can "see" the share for a little while, but then the file is deleted and USB5T is back to "offline" status.

gboudreau commented 1 year ago

but then the file is deleted and USB5T is back to "offline" status

That's normal. This file is only used for drives that are remote, and thus have no UUID Greyhole can use to track if the drive is mounted or not. When Greyhole sees this file on a local mount, it removes it.

Btw if I do this: touch /data/USB5T/.greyhole_uses_this then greyhole can "see" the share for a little while

That would point out to Greyhole thinking this drive's UUID changed, since it first saw it.

gboudreau commented 1 year ago

From the log you sent me, it seems to work correctly:

Oct 30 01:57:18 DEBUG balance: ││├┘ Balancing storage pool drive: /data/WDC4T (63.0GB available, target: 1.46TB) [3328072]
Oct 30 01:57:18 DEBUG balance: ││├┐ Working on file: storage_pool_drive/media/Movies/[...].mkv (17.9GB) [3324920]
Oct 30 01:57:18 DEBUG balance:   Drives with available space: /data/USB5T (4.27TB avail) - /data/WDC4T (60.1GB avail) - /data/SEA4T (56.9GB avail)  [3325984]
Oct 30 01:57:18 DEBUG balance: ││││ Target drive: /data/USB5T (4.27TB available) [3325136]
Oct 30 01:57:18 DEBUG balance: ││││ Moving file copy... [3325904]
vonj commented 1 year ago

I ran balance just to see what happened. When I cancelled the balance before it was complete, the drive went back to "offline" again.

For instance if I run balance now again, this is in the log:


Oct 30 02:27:26 DEBUG balance: │├ Drives to balance: /data/WDC4T, /data/SEA4T, /data/USB5T [2858680]
Oct 30 02:27:26 DEBUG balance: │├ Will work on the storage pool drives in the following order: /data/SEA4T, /data/WDC4T [2862208]
Oct 30 02:27:26 DEBUG balance: │├┐ Balancing storage pool drive: /data/SEA4T (56.9GB available, target: 59.9GB) [2862192]
gboudreau commented 1 year ago

From the MySQL data you sent me, it looks like your settings table is broken. The name column should be the primary key, and thus should never contain more than one rows with the same name, but I see multiple entries for name = sp_drives_definitions (and many other settings).

Can you check the structure of your settings table, using the MySQL prompt?

desc settings;
vonj commented 1 year ago
mysql> desc settings;
+-------+------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+------+------+-----+---------+-------+
| name  | text | NO   |     | NULL    |       |
| value | text | NO   |     | NULL    |       |
+-------+------+------+-----+---------+-------+
2 rows in set (0.05 sec)
gboudreau commented 1 year ago

Not sure how you ended up with this broken table... Pretty sure Greyhole could not cause this.

Stop the greyhole daemon, then you can rename your old settings table, and re-create a new one with:

RENAME TABLE settings TO settings_broken;

CREATE TABLE `settings` (
`name` VARCHAR(255) NOT NULL,
`value` TEXT NOT NULL,
PRIMARY KEY (`name`)
) ENGINE = MYISAM DEFAULT CHARSET=utf8;

INSERT INTO `settings` (`name`, `value`) SELECT * FROM settings_broken WHERE name IN ('last_read_log_smbd_line', 'last_OOS_notification', 'db_version') ON DUPLICATE KEY UPDATE `value` = VALUES(`value`);

Then re-start the daemon.

See if it works better now.

You can look il you settings_broken table, see if you have any other settings you'd want to migrate, apart from the 3 I copied above.

vonj commented 1 year ago

Trying now.

I don't know what the settings mean :-D The drive paths are in there, but they are also in the config file, so not sure if that's important.

gboudreau commented 1 year ago

You don't need to copy the sp_drives_definitions setting ; it's re-created by greyhole the first time it sees each drive.

vonj commented 1 year ago

Wow, it really works now! Thanks soooo much. Amazing!

One more question - when I copy more stuff and the older drive starts filling, will greyhole auto-balance to fit new files while keeping 2 copies?

Or will I have to schedule a balance command?

gboudreau commented 1 year ago

A balance is only needed if you want to move stuff currently on the old drives to the new one. If you're ok with the data on the old drives staying there, no need to balance. But Greyhole doesn't auto-balance, unless files change, in which case (depending in your config) it can move a modified file copy to a new storage pool drive with more empty space. It's probably better and easier to run balance for a few hours, to have enough free space available on your old drives for a while.