dealing with low trip temp drives

[ggruber]

another point: in my JBOD there are disks which report a trip temperature of 40°C. So when nwipe starts there are some disks at 35 or 36°C, which are red flagged. how could we ever wipe these? make the "safety temperature gap" an option? for risky natures negative gaps could be allowed in combination with the --force option. what do you think?

[ggruber]

I think I'd be looking at why a trip temperature of 40 Deg.C is being reported. Is that trip temperature a valid trip temperature as specified in the manufacturer's drive specs?

Found this: https://forums.servethehome.com/index.php?threads/exos-40c-drive-trip-temperature.40682/ So it seems, this can occur every now and then

If it's actual trip temperature is 45 Deg.C as reported by the drive and verified by the drive specification and only a single temperature is provided by the drive then that's why we shouldn't artificially substitute values, i.e adding the -5 degree value I think it was in the code. It's unnecessary as the GUI code that handles the colour of the temperature will quite happily work with a single limit.

I'd respect the values from the drive, in every case. But those 40°C seem to give room for interpretation. Yes, I agree, it can't be ment seriously, literally. Until a technical person from one or another vendor/firmware maker explains the backgrounds

If that's the case the arbitrary -5 that gets substituted in the code should just be removed and the variable left in its original initialised state of 10000000 or whatever the number was.

nack

I don't think you need a force option, it's not like nwipe is restricting or slowing I/O to the drive, the temperature colour is just a benign warning that can be ignored.

Well, as max temp will remain reported in the drive, as temp history can (in some drives), I think it's not that easy. But to cool down the whole box (how?) seems also no valid option. There is a German saying "Augen zu und durch" (is there an English equivalent?). With a "You have been warned!" in case of possible warranty or reliability issue. In this sense I think the requested --force Option is appropriate.

I would remove that arbitrary -5 though if those drives genuinely have a 45 Deg.C temperature operating limit according to the drive specs. What make model are those drives?

again: nack Genuine Seagates, as far as I can see.

[PartialVolume]

Interesting, I think I would agree with one of the comments, that whoever wrote the firmware for that drive screwed up and that 40 Deg.C should be 60 deg.c

Some drive specs report an ambient max room temperature of 40 Deg.C and whoever wrote the code used the wrong value.

[PartialVolume]

Just out of interest, what is the ambient room temperature where you are running those drives. Most of the rooms I work in are air conditioned at 20 deg.c.

[ggruber]

the server is in our serverroom @work. No remote room temperature measurement, and this server also gives no inlet temperature. Usually we have ~22°C (energy saving reasons)

[ggruber]

I think I would agree with one of the comments, that whoever wrote the firmware for that drive screwed up and that 40 Deg.C should be 60 deg.c

I'd like to agree, but this error must have happend several times, to different engineers (Seagate firmware, Dell firmware, ...) From my POV not explained completely

[ggruber]

btw, the wiping of 49 disks (some are taken out), started from a drastically enhanced eraseHDD (to deal with dual ported disks, even without multipath setup) (the fix for the crash of nwipe being called from eraseHDD lead me into the development of nwipe ;-) works obviously) runs very smooth in the GUI. No hangings seen so far. :-) Can't see the blinking lights from here ;-) The sort modification PR has it's cause also in this box, of course

[PartialVolume]

how could we ever wipe these? make the "safety temperature gap" an option? for risky natures negative gaps could be allowed in combination with the --force option. what do you think?

I don't really understand what the purpose of --force is.

I still think the -5 should be removed as the ATA spec provides for a warning temperature in the form of max continuous operating temperature. If the drive manufacturer chooses not to utilize it then I don't see why nwipe should substitute its own value.

If 40 deg.c is the max continuous operating temperature and it's not a mistake in the firmware then I guess Seagate and Dell are expecting the drives to be operated in a server with an ambient room temperature < 20 Deg.C.

Seems very odd that the published spec says 60 Deg.C

Anyway, the drive reports 40 deg.C so that's when the colour should turn to red. Not at 35 Deg.C as it does at the moment.

[PartialVolume]

another point: in my JBOD there are disks which report a trip temperature of 40°C. So when nwipe starts there are some disks at 35 or 36°C, which are red flagged. how could we ever wipe these? make the "safety temperature gap" an option? for risky natures negative gaps could be allowed in combination with the --force option. what do you think?

Looking at those individual transfer speeds it looks like the controller is acting as a bottleneck as the spec for each of those drives is sustained transfer rate of 175MB/s. Assuming Seagate are exaggerating a bit and the real world speed averaged across the whole disc is 150MB/s then maybe a second controller might improve throughput or is the bottleneck the bus?

[PartialVolume]

Also the load averages being up at 50 ( how many cores do you have?) would maybe mean there are quite a few threads waiting around for I/0 which is very likely as the throughput is so low. I think it's more likely that the high load averages are caused by the controller not being able to go faster than 2GB/s rather than the CPU cores working particularly hard. Top should show nwipes CPU load, but I'm guessing the CPU isn't working too hard just hanging around waiting for I/O.

[PartialVolume]

One other thing, I'm guessing you are wiping more than the sixteen drives shown in the snapshot, as the sum of 16 x 44MB/s = 704MB/s, so doesn't add up to the overall throughput of 2,061MB/s. So I'm guessing drives sde-sdi are also being wiped? which would account for the discrepancy.

[PartialVolume]

Just out of interest, does anybody ever come across 520 or 528 byte per sector drives rather than the normal 512.

I was just wondering how nwipe dealt with non standard sector sizes?

Or does such a drive have to be reformatted to 512 before nwipe will work with it?

[ggruber]

another point: in my JBOD there are disks which report a trip temperature of 40°C. So when nwipe starts there are some disks at 35 or 36°C, which are red flagged. how could we ever wipe these? make the "safety temperature gap" an option? for risky natures negative gaps could be allowed in combination with the --force option. what do you think?

Looking at those individual transfer speeds it looks like the controller is acting as a bottleneck as the spec for each of those drives is sustained transfer rate of 175MB/s. Assuming Seagate are exaggerating a bit and the real world speed averaged across the whole disc is 150MB/s then maybe a second controller might improve throughput or is the bottleneck the bus?

for sure there is a bottleneck, if I connect a box with 24+20 Disks per backplane with a host controller. The backplanes have two 6Gbps Expander Chips. But every of the 4 rear side SAS Ports can reach all disks. As the disks are dual ported the shelf is well suited for a shared storage with redundant head notes that offer storage to other systems, e.g. by means of NFS. A cluster mechanism was used to decide which one is the active headnode.

I think, this box is well suited as archive storage. (It is approx. 10 y old)

I could widen a bit the I/O performance by using both channels, that are physically connected now. As I used the other headnode for other purposes I could indeed insert a second HBA into the headnode and connect to all 4 SAS Ports of the JBOD. I could carefully select then which of the 4 times visible sdxy disks I should talk to. And spend lots of effort into a clean multipathing configuration for redundancy purposes. What is, what the box deserves.

I see 1800MB/s now. For the wiping I can wait, there is no deadline, I'm just recycling old hardware.

[ggruber]

Also the load averages being up at 50 ( how many cores do you have?) would maybe mean there are quite a few threads waiting around for I/0 which is very likely as the throughput is so low. I think it's more likely that the high load averages are caused by the controller not being able to go faster than 2GB/s rather than the CPU cores working particularly hard. Top should show nwipes CPU load, but I'm guessing the CPU isn't working too hard just hanging around waiting for I/O.

40 CPU cores, ~73% I/O Wait

[ggruber]

One other thing, I'm guessing you are wiping more than the sixteen drives shown in the snapshot, as the sum of 16 x 44MB/s = 704MB/s, so doesn't add up to the overall throughput of 2,061MB/s. So I'm guessing drives sde-sdi are also being wiped? which would account for the discrepancy.

the wipe was started with some 40 disks. Six 600GB SAS on another controller (onboard in the headnode, so are the disks) are done, so are the two 1TB SATA disks, which are single ported in the JBOD. Had to outcomment another two disks that are about to fail as the eraseHDD preflight check revealed. Those could be good test objects for your wipe backwards approach.

So there should be 8 (in headnode) + 24 + 20 (2 backplanes in JBOD) - 4 (2 OS, 2 defective) Disks being wiped. Actively wiping should be 42 disks. Which gives a good match with a load of 42.89 now.

[ggruber]

Just out of interest, does anybody ever come across 520 or 528 byte per sector drives rather than the normal 512.

I was just wondering how nwipe dealt with non standard sector sizes?

Or does such a drive have to be reformatted to 512 before nwipe will work with it?

One of the STEC SAS SSD came with 520 or 528 bytes/sector. I was glad that I could change this to 512. We could change this again, if you would like to see it.

[ggruber]

how could we ever wipe these? make the "safety temperature gap" an option? for risky natures negative gaps could be allowed in combination with the --force option. what do you think?

I don't really understand what the purpose of --force is.

I still think the -5 should be removed as the ATA spec provides for a warning temperature in the form of max continuous operating temperature. If the drive manufacturer chooses not to utilize it then I don't see why nwipe should substitute its own value.

If 40 deg.c is the max continuous operating temperature and it's not a mistake in the firmware then I guess Seagate and Dell are expecting the drives to be operated in a server with an ambient room temperature < 20 Deg.C.

Seems very odd that the published spec says 60 Deg.C

Anyway, the drive reports 40 deg.C so that's when the colour should turn to red. Not at 35 Deg.C as it does at the moment.

I understand the trip temperature that is reported by SAS(!) drives as the temperature that must not be exceeded.

So it makes completely sense for me to warn before this limit is reached. Ok, 5 degrees are disputable, we could have it in the config file.

Those 40°C are probably meant in a way we have not real glue at the moment: possible.

I'd not like to be responsible if a drive gets damaged when operated outside it's defined environmental conditions. If someone does this he/she should ack this by using --force.

But that's just my opinion.

[PartialVolume]

I'd not like to be responsible if a drive gets damaged when operated outside it's defined environmental conditions. If someone does this he/she should ack this by using --force.

Ah, ok I get it now. The default action of nwipe should be to pause the wipe if the temperature hit's critical or whatever the max temperature limit is, if the drive provides just the one upper limit, then recommence the wipe once the temperature has dropped a couple of degrees below the limit. --force overrides this and allows nwipe to continue wiping irrespective of temperature. I'd probably use something like --no-pause-on-critical-temperature.

There may need to be changes to the code that works out drive throughput that accounts for these pauses otherwise the throughput of the drive could be much lower than the real drive throughput the drive was capable of. I'm not sure if that really matters or not?

[PartialVolume]

One of the STEC SAS SSD came with 520 or 528 bytes/sector. I was glad that I could change this to 512. We could change this again, if you would like to see it.

Yes, that would be useful to know how nwipe copes with such a drive. Does it wipe the full sector or only 512 bytes of it?

Or is it incapable of wiping such a drive and errors out?

[ggruber]

use something like --no-pause-on-critical-temperature.

sounds clearer than --force

I'm not sure if that really matters or not? well, I think it will depend. If you take the throughput number as indicator for the health or performance of the drive, you'd like to see the "net" throughput. Otherwise the overall throughput would be ok, possibly with a P mark (like Pauses occured).

@PartialVolume: sector size change: will have to get the tool again. Do you want the check soon? Otherwise I will provide you some doc to perform the change yourself.

I'm off for today

[PartialVolume]

use something like --no-pause-on-critical-temperature.

sounds clearer than --force

I'm not sure if that really matters or not? well, I think it will depend. If you take the throughput number as indicator for the health or performance of the drive, you'd like to see the "net" throughput. Otherwise the overall throughput would be ok, possibly with a P mark (like Pauses occured).

@PartialVolume: sector size change: will have to get the tool again. Do you want the check soon? Otherwise I will provide you some doc to perform the change yourself.

No hurry, docs are ok, although I'm unlikely to come across a drive that supports this. I'm assuming most drives don't support the ability to change sector size?

[ggruber]

@PartialVolume : uploaded new videos, from the 44 disks box

[PartialVolume]

@PartialVolume : uploaded new videos, from the 44 disks box

@ggruber sorry, I can't seem to find the link for that amongst all the stuff that's been posted.

[ggruber]

@PartialVolume : uploaded new videos, from the 44 disks box

@PartialVolume : uploaded new videos, from the 44 disks box

@ggruber sorry, I can't seem to find the link for that amongst all the stuff that's been posted.

PM probably