pbatard / rufus

The Reliable USB Formatting Utility
https://rufus.ie
GNU General Public License v3.0
29.18k stars 2.59k forks source link

Rufus 2.13 can't access my drive, but Rufus 2.12 could! #924

Closed pbatard closed 7 years ago

pbatard commented 7 years ago

(Logging this as a new issue, as expect a handful of people to run into it)

When using Rufus 2.13, you might find that you are no longer able to create a drive, whereas version 2.12 seems to be okay. Especially, you may see something like this in the 2.13 log:

Format operation started
Requesting disk access...
Opened drive \\.\PHYSICALDRIVE2 for write access
Will use 'G:' as volume mountpoint
Waiting for access...
Could not open drive \\?\Volume{b0324320-003f-11e4-9a86-f46d0466079c}: [0x00000020] The process cannot access the file because it is being used by another process.
Could not lock volume
Re-mounted volume as 'G:' after error

The reason for this is that, in Rufus 2.13, write sharing to the USB device is now restricted (for the record, this is mentioned in the list of changes for 2.13) which means that the most recent version of Rufus will bail out, if it detects that you are running another application that is keeping write access to your USB.

As you can imagine, if 2 applications try to share write access to the same device/volume, and especially if one is trying to repartition or format it, very bad things can (and do) happen. Therefore, the latest version of Rufus is designed to work only if it can gain exclusive write access to the device. This way, a whole stream of issues, that have been reported in the past, can be avoided, where Rufus was unable to successfully complete its operations.

Now, for the vast majority of Windows users, this is not a problem. For instance, Rufus 2.13 was extensively tested with regular installations of Windows (including Windows 10 1703), and there were no issues. Also, even though version 2.13 has been released for a few days, I am getting very few reports from people running into the problem above.

However, if you are unlucky enough to be running a problematic applications (i.e. one that seems to want to keep write access to your USB drive, when it really shouldn't), you may be wondering what you can do.

For now, I have no plans to revert Rufus to being permissive gain, unless I can get a good idea of what other applications are keeping a write lock on USB devices. Thus, if you are running into this issue, I would greatly appreciate your help.

Unfortunately however, Windows also makes it exceedingly difficult to tell what other application is the one that is causing the issue (otherwise, Rufus would be providing this information) so to try to determine it, you have to go through a tedious process that requires you to:

  1. Download and install Process Explorer
  2. Download Rufus 2.13 TEST2 from here
  3. Launch rufus-2.13_TEST2.exe and try your operation again
  4. On error, check the Rufus log and make a note of the [\Device\HarddiskVolume##] or [\Device\Harddisk5\DR#] that appears between square brackets ([ and ]) as reported in the error message. For instance, if you see: Could not open \\?\Volume{b0324320-003f-11e4-9a86-f46d0466079c} [\Device\HarddiskVolume12]: [0x00000020] The process cannot access the file because it is being used by another process. then \Device\HarddiskVolume12 is what you you will need to be looking for.
  5. Run Process Explorer as administrator
  6. In Process Explorer type Ctrl-L until you see the lower pane, and also type Ctrl-H, so that the lower pane will display Handles.
  7. Go through each process one by one and look at the handles, until you find the one that is keeping a handle to the same \Device\HarddiskVolume## or [\Device\Harddisk5\DR#]

Once you have done that, you should have found the application that is causing the write lock out, and you can report it below.

RJARPCGP commented 7 years ago

Looks a lot like I'm required to go back to 2.12 to write Windows ISOs to USB flash drives...

rufus 2 13

handles

JonnyTech commented 7 years ago

@RJARPCGP please follow step #9:

Still in Porcess Explorer, go through each process one by one and look at the handles, until you find the one that is keeping a handle to the same \Device\HarddiskVolume##

In your case for that particular instance search for \Device\HarddiskVolume2

pbatard commented 7 years ago

Indeed, now you need to go through each individual process, and find which one has \Device\HarddiskVolume2 open. Again, I very much need your help in identifying the ill-behaved applications that keep a write access handle on a flash drive, when they have no business doing so. Once these applications have been identified, I will be able to determine the best course of action. But I can't do that without the help of the users running into this issue...

Hlsgs commented 7 years ago

Simply having a Windows Explorer window opened causes this consistently for me. That is regardless of where said window is exploring, even if it has nothing to do with the USB drive. Using alt + , to disable exclusive locking does not help, but closing all WE windows does.

pbatard commented 7 years ago

@Hlsgs, what version of Windows are you using?

I've been testing with the File Explorer open to the USB drive in Windows 10, and I'm not seeing any issue.

Hlsgs commented 7 years ago

@pbatard W10 x64 1607 14939.969 I will do some further testing(safe mode, the above mentioned method of checking which process hangs the drive) and get back to you. But shouldn't using alt + , bypass this issue altogether?

pbatard commented 7 years ago

Thanks. I did test W10 x64 1607 with File Explorer open (into the drive's content or outside of it), and couldn't replicate your findings. When File Explorer was opened to the drive, it got closed automatically by Windows as the drive was being formatted. For good measure I also tested Windows 7, and things also worked as expected.

The part that I find strange is that Rufus should only have an issue if the drive is opened for write access (because Rufus always enables file sharing for read access), and I don't really understand why File Explorer would open any drive or volume for write access, unless it is actively writing to them.

The only standard Windows behaviour that I am aware of, that results in write access outside of an action that was initiated by the user, is the creation of the System Volume Information\ directory on a newly formatted drive. However, I tried to plan for that, by having Rufus try to get write access for 15 seconds, before giving up, which, even on a slow drive, should give Windows enough time to complete the writing of the system files.

All in all, because I've never ever seen a problem with 2.13 on any of the Windows machines I tested (10, 8.1, 7, including multiple versions of each one), and also because very few people seem to be reporting the issue so far (which is not to say I am not going to try to help those who are experiencing it - I think I may create a TEST version of Rufus to help with the troubleshooting if I can), I am relatively confident that this behaviour is triggered by some non Windows native application being installed on the system, and I would greatly appreciate if the persons experiencing it could help figure out which one(s) it might be.

prat-man commented 7 years ago

I am facing the exact same issue. Rufus is reporting that some other program is trying to access the media. I think it may be caused by antivirus programs that are trying to scan the media while Rufus is partitioning the media. I have reverted to version 2.12 for now. However, it would be very helpful if you fix this issue as Rufus 2.13 is completely unusable.

pbatard commented 7 years ago

I think it may be caused by antivirus programs that are trying to scan the media while Rufus is partitioning the media.

An antivirus should not require write access when scanning a drive, and the change between 2.12 and 2.13 only has to do with Rufus now disabling the FILE_SHARE_WRITE to the drives and volumes it is trying to format. So, if an antivirus is not able to perform a scan, then I would say that it was poorly designed, because it should not require write access to do so.

Still, have you tested what happens when you disable your antivirus?

Once again, I need your help to identify what applications have the USB drive opened for write access.

RJARPCGP commented 7 years ago

I went through all the processes with the USB drive plugged in and I don't see "\Device\HarddiskVolume2" anywhere! Facepalming...

pbatard commented 7 years ago

Damn! Thanks for trying though. I'll see if I can produce a version of Rufus that lets you know the exact handle that is causing the block, as one of the issues from the method I highlighted, is that it can be either the volume or the disk handle for which exclusive write access is denied, and these are separate handles... Ideally, you should try to open the disk device as well as the volume(s - there may also bee more than one) in HxD, to find all the possible handles, but I appreciate that this can get confusing if you're not familiar with Windows. So I'll try to have Rufus give the actual handle to look for, so that the HxD step can be skipped altogether.

csdev commented 7 years ago

I encountered this problem while attempting to create Windows 10 installation media from a Windows 7 host. I noticed that after Rufus finished formatting the USB drive, but before it began copying files, a Windows "autorun" dialog popped up on screen. Then, I got the Rufus error. This makes me think that Windows Explorer is the culprit.

pbatard commented 7 years ago

I noticed that after Rufus finished formatting the USB drive, but before it began copying files, a Windows "autorun" dialog popped up on screen.

Nah, that can happen and is not something that I expect to produce a write lock.

However, according to your report, and as opposed to others experiencing the original issue described, Rufus does manage to access the drive and format it, before you encounter an error.

Do you have the full log from your attempt? I'd like to be able to examine this output. Also, did Rufus seem to wait 15 seconds after formatting, before producing the error, or was it faster than that?

csdev commented 7 years ago

I did not notice a delay after formatting.

Rufus version: 2.13.1081 (Portable)
Windows version: Windows 7 SP1 64-bit
Syslinux versions: 4.07/2013-07-25, 6.03/2014-10-06
Grub versions: 0.4.6a, 2.02~rc2
System locale ID: 0x0409
Will use default UI locale 0x0409
Found USB 2.0 device 'hp v125w USB Device' (03F0:3307)
1 device found
Disk type: Removable, Sector Size: 512 bytes
Cylinders: 3946, TracksPerCylinder: 255, SectorsPerTrack: 63
Partition type: MBR, NB Partitions: 1
Disk ID: 0x00A5760C
Drive has a Rufus Master Boot Record
Partition 1:
  Type: NTFS (0x07)
  Size: 30.2 GB (32462864384 bytes)
  Start Sector: 2048, Boot: Yes
Scanning image...
ISO analysis:
  Image is an UDF image
Disk image analysis:
  Image does not have an x86 Master Boot Record
ISO label: 'ESD-ISO'
  Size: 4.4 GB
  Uses: EFI
  Uses: Bootmgr
Using image: Windows10_x64.iso
Scanning image...
ISO analysis:
  Image is an UDF image
Disk image analysis:
  Image does not have an x86 Master Boot Record
ISO label: 'ESD-ISO'
  Size: 4.4 GB
  Uses: EFI
  Uses: Bootmgr
Using image: Windows10_x64.iso

Format operation started
Requesting disk access...
Opened drive \\.\PHYSICALDRIVE2 for write access
Will use 'F:' as volume mountpoint
I/O boundary checks disabled
Analyzing existing boot records...
Drive has a Rufus Master Boot Record
Volume has an unknown Partition Boot Record
Clearing MBR/PBR/GPT structures...
Erasing 128 sectors
Deleting partitions...
Partitioning (MBR)...
Closing existing volume...
Waiting for logical drive to reappear...
Formatting (NTFS)...
Using cluster size: 4096 bytes
Quick format was selected
Creating file system...
Format completed.
Writing master boot record...
Drive has a Zeroed Master Boot Record
Set bootable USB partition as 0x80
Using Rufus MBR
Found volume GUID \\?\Volume{97fa8c4e-e9a2-11e3-a794-005056c00008}\
Waiting for access...
Could not open drive \\?\Volume{97fa8c4e-e9a2-11e3-a794-005056c00008}: [0x00000020] The process cannot access the file because it is being used by another process.
Could not re-mount volume for partition boot record access
\\?\Volume{97fa8c4e-e9a2-11e3-a794-005056c00008}\ was already mounted as F:\
Re-mounted volume as 'F:' after error

Found USB 2.0 device 'hp v125w USB Device' (03F0:3307)
1 device found
Disk type: Removable, Sector Size: 512 bytes
Cylinders: 3946, TracksPerCylinder: 255, SectorsPerTrack: 63
Partition type: MBR, NB Partitions: 1
Disk ID: 0x00D1D194
Drive has a Rufus Master Boot Record
Partition 1:
  Type: NTFS (0x07)
  Size: 30.2 GB (32462864384 bytes)
Start Sector: 2048, Boot: Yes
pbatard commented 7 years ago

Thanks for the log.

For the record, I have now uploaded a TEST version of Rufus 2.13 (TEST2) that will display the exact handle that is causing the issue, and that one should look for in Process Explorer. I have also updated the steps above to look for the handle accordingly. Hopefully, this will finally enable one of you to identify the other application that might be keeping the handle open for write access.

jorymorrison commented 7 years ago

I am experiencing the same issue with 2.13. I tried 2.11 and it worked fine.

Here's a log from 2.13:

Rufus version: 2.13.1081
Windows version: Windows 7 SP1 64-bit
Syslinux versions: 4.07/2013-07-25, 6.03/2014-10-06
Grub versions: 0.4.6a, 2.02~rc2
System locale ID: 0x0409
Will use default UI locale 0x0409
Found USB 3.0 device 'Kingston DataTraveler 3.0 USB Device' (0951:1666)
1 device found
Disk type: Removable, Sector Size: 512 bytes
Cylinders: 3769, TracksPerCylinder: 255, SectorsPerTrack: 63
Partition type: MBR, NB Partitions: 1
Disk ID: 0x001245D6
Drive has a Rufus Master Boot Record
Partition 1:
  Type: NTFS (0x07)
  Size: 28.9 GB (31003246592 bytes)
  Start Sector: 2048, Boot: Yes
0 devices found
0 devices found
Found USB 3.0 device 'Kingston DataTraveler 3.0 USB Device' (0951:1666)
1 device found
Disk type: Removable, Sector Size: 512 bytes
Cylinders: 3769, TracksPerCylinder: 255, SectorsPerTrack: 63
Partition type: MBR, NB Partitions: 1
Disk ID: 0x001245D6
Drive has a Rufus Master Boot Record
Partition 1:
  Type: NTFS (0x07)
  Size: 28.9 GB (31003246592 bytes)
  Start Sector: 2048, Boot: Yes
Scanning image...
ISO analysis:
  Image is an UDF image
Disk image analysis:
  Image does not have an x86 Master Boot Record
ISO label: 'CSLA_X64FREO_EN-US_DV5'
  Size: 4.0 GB
  Uses: EFI
  Uses: Bootmgr
  Uses: Install.wim (version 0.13.1)
Using image: Win10_1703_SingleLang_English_x64.iso

Format operation started
Requesting disk access...
Opened drive \\.\PHYSICALDRIVE1 for write access
Will use 'F:' as volume mountpoint
Waiting for access...
Could not open drive \\?\Volume{352097e0-12d6-11e6-8137-34e6d7057f85}: [0x00000020] The process cannot access the file because it is being used by another process.
Could not lock volume
Re-mounted volume as 'F:' after error

Found USB 3.0 device 'Kingston DataTraveler 3.0 USB Device' (0951:1666)
1 device found
Disk type: Removable, Sector Size: 512 bytes
Cylinders: 3769, TracksPerCylinder: 255, SectorsPerTrack: 63
Partition type: MBR, NB Partitions: 1
Disk ID: 0x001245D6
Drive has a Rufus Master Boot Record
Partition 1:
  Type: NTFS (0x07)
  Size: 28.9 GB (31003246592 bytes)
  Start Sector: 2048, Boot: Yes
pbatard commented 7 years ago

Here's a log from 2.13

Great. Now can you please try to follow the steps from the first post (using the TEST2 build), and report if you can find the process that is holding the handle that is causing the error?

Hlsgs commented 7 years ago

@pbatard Sorry for the delay. I am unable to find the respective handle in teh process list. However, i have to do a severe mea culpa for not specifying the settings I was using. I can only replicate the issue while using dual UEFI/BIOS mode, MBR for BIOS or UEFI AND formatting as FAT32(by far my preferred method, please don't disable this). After a few failed attempts, it seems to work though. While not in dual boot mode or by using the default NTFS, I am seeing no issues.

EDIT: I am unable to replicate this under Safe Mode. However, upon further digging, it still fails when using FAT32 and the handle is nowhere to be found in process explorer, even when spamming the search button in process explorer search. Said hadle only appears under the Rufus task while performing the "creating file system task".

renevane commented 7 years ago

Hi Pete, I have a similar issue. In my case it happens when Rufus is trying to write the MBR to the USB drive. Using your TEST2 build (also running as Administrator) I looked for the handle in Process Explorer, but it wasn't there. I have also tried in Windows' Safe Mode with the same result. In version 2.12 it works without this problem. Including log file; hope it helps!

Rufus version: 2.13.1083 (Test2)
Windows version: Windows 7 SP1 64-bit
Syslinux versions: 4.07/2013-07-25, 6.03/2014-10-06
Grub versions: 0.4.6a, 2.02~rc2
System locale ID: 0x0409
Will use default UI locale 0x0409
Found USB 2.0 device 'Lexar USB Flash Drive USB Device' (05DC:A838)
1 device found
Disk type: Removable, Sector Size: 512 bytes
Cylinders: 3891, TracksPerCylinder: 255, SectorsPerTrack: 63
Partition type: MBR, NB Partitions: 1
Disk ID: 0x00120F4B
Drive has a Rufus Master Boot Record
Partition 1:
  Type: NTFS (0x07)
  Size: 29.8 GB (32007782400 bytes)
  Start Sector: 2048, Boot: Yes
Scanning image...
ISO analysis:
  Image is an UDF image
Disk image analysis:
  Image does not have an x86 Master Boot Record
ISO label: 'CCSA_X64FRE_EN-GB_DV5'
  Size: 4 GB
  Uses: EFI
  Uses: Bootmgr
  Uses: Install.wim (version 0.13.1)
Using image: en-gb_windows_10_multiple_editions_version_1703_updated_march_2017_x64_dvd_10194881.iso

Format operation started
Requesting disk access...
Opened \\.\PHYSICALDRIVE4 [\Device\Harddisk4\DR4] for write access
Will use 'H:' as volume mountpoint
I/O boundary checks disabled
Analyzing existing boot records...
Drive has a Rufus Master Boot Record
Volume has an unknown Partition Boot Record
Clearing MBR/PBR/GPT structures...
Erasing 128 sectors
Deleting partitions...
Partitioning (MBR)...
Closing existing volume...
Waiting for logical drive to reappear...
Formatting (NTFS)...
Using cluster size: 4096 bytes
Quick format was selected
Creating file system...
Format completed.
Writing master boot record...
Drive has a Zeroed Master Boot Record
Set bootable USB partition as 0x80
Using Rufus MBR
Found volume GUID \\?\Volume{8570d789-81f4-11e5-82e9-005056c00008}\
Waiting for access...
Could not open \\?\Volume{8570d789-81f4-11e5-82e9-005056c00008} [\Device\HarddiskVolume6]: [0x00000020] The process cannot access the file because it is being used by another process.
Could not re-mount volume for partition boot record access
\\?\Volume{8570d789-81f4-11e5-82e9-005056c00008}\ was already mounted as H:\
Re-mounted volume as 'H:' after error

Found USB 2.0 device 'Lexar USB Flash Drive USB Device' (05DC:A838)
1 device found
Disk type: Removable, Sector Size: 512 bytes
Cylinders: 3891, TracksPerCylinder: 255, SectorsPerTrack: 63
Partition type: MBR, NB Partitions: 1
Disk ID: 0x0013B07A
Drive has a Rufus Master Boot Record
Partition 1:
  Type: NTFS (0x07)
  Size: 29.8 GB (32007782400 bytes)
  Start Sector: 2048, Boot: Yes
pbatard commented 7 years ago

@rvane,

Your issue intervenes after the formatting operation, so I'm not entirely surprised you can't find the handle, as,in this case, it should be a more transient behaviour from whatever application is keeping a write handle to the volume.

For reference, there are two distinct cases for this issue:

Unfortunately, I only expect the Process Explorer thing to be useful in the first case (because then I expect an application to keep a permanent write handle to the volume) but not in the second (since the volume could be accessed for formatting).

@Hlsgs,

I'm not planning to disable the dual BIOS/UEFI cheat mode. And thanks for the clarification, as I feel like we might be going somewhere with this. However, I still can't reproduce the issue even when making sure I'm using FAT32 (I also tested with Large FAT32 just in case). I'll try to test this combination further, as this is the closest thing we have to the beginning of a clue.

Overall, it still looks to me like you guys must have an application somewhere that is causing this conflict (which also must be running in safe mode). Can you take a closer look at the services you may have running in the background (security solutions, virtual drive mounters, etc.), and see if disabling some of those help?

renevane commented 7 years ago

@pbatard Thanks for the quick reply Pete. To rule out any programs, services etc. which might be accessing my USB, I ran two additional tests on both another physical Windows 7 x64 machine (without any additional software installed) and from within a cleanly installed Windows 7 x64 VMware virtual machine. Both tests had the same result: Rufus 2.13 can't write the MBR. Then I installed a Windows 10 x64 virtual machine and repeated the test from within that machine. This time, it worked!

So it seems that from within Windows 7, I can't get Rufus 2.13 to create a Windows 10 installation USB, but from within Windows 10, I can.

I really do not know what (if any) conclusion can be drawn from that. I hope it helps you. Please let me know if I can help by performing additional tests.

Hlsgs commented 7 years ago

@pbatard i fall within the second case aswell. I am also unable to consistently replicate this. Sometimes it works, sometimes not. I caught a glimpse of a process beside Rufus handling the drive by spamming teh search in Process Explorer and it was svchost(figures). The issue continues to manifest only while using FAT32 and dual boot mode.

Here's my log: ``` Rufus version: 2.13.1083 (Test2) Windows version: Windows 10 64-bit (Build 14393) Syslinux versions: 4.07/2013-07-25, 6.03/2014-10-06 Grub versions: 0.4.6a, 2.02~rc2 System locale ID: 0x0409 Will use default UI locale 0x0409 Found USB 2.0 device 'USB DISK Pro USB Device' (13FE:3623) 1 device found Disk type: Removable, Sector Size: 512 bytes Cylinders: 3909, TracksPerCylinder: 255, SectorsPerTrack: 63 Partition type: MBR, NB Partitions: 1 Disk ID: 0x0148B00B Drive has a Windows 7 Master Boot Record Partition 1: Type: FAT32 LBA (0x0c) Size: 29.9 GB (32155631616 bytes) Start Sector: 2048, Boot: Yes Scanning image... ISO analysis: Image is an UDF image Disk image analysis: Image does not have an x86 Master Boot Record ISO label: 'CCSA_X64FRE_EN-US_DV5' Size: 4 GB Uses: EFI Uses: Bootmgr Uses: Install.wim (version 0.13.1) Using image: en_windows_10_multiple_editions_version_1703_updated_march_2017_x64_dvd_10189288.iso Format operation started Requesting disk access... Opened \\.\PHYSICALDRIVE3 [\Device\Harddisk3\DR6] for write access Will use 'H:' as volume mountpoint I/O boundary checks disabled Analyzing existing boot records... Drive has a Windows 7 Master Boot Record Volume has an unknown FAT16 or FAT32 Partition Boot Record Clearing MBR/PBR/GPT structures... Erasing 128 sectors Deleting partitions... Partitioning (MBR)... Closing existing volume... Waiting for logical drive to reappear... Formatting (FAT32)... Using cluster size: 16384 bytes Quick format was selected Creating file system... Format completed. Writing master boot record... Drive has a Zeroed Master Boot Record Partition is already FAT32 LBA... Set bootable USB partition as 0x80 Using Windows 7 MBR Found volume GUID \\?\Volume{62a18908-1548-11e7-a8f0-902b34df67fd}\ Waiting for access... Could not open \\?\Volume{62a18908-1548-11e7-a8f0-902b34df67fd} [\Device\HarddiskVolume12]: [0x00000020] The process cannot access the file because it is being used by another process. Could not re-mount volume for partition boot record access \\?\Volume{62a18908-1548-11e7-a8f0-902b34df67fd}\ was already mounted as H:\ Re-mounted volume as 'H:' after error Found USB 2.0 device 'USB DISK Pro USB Device' (13FE:3623) 1 device found Disk type: Removable, Sector Size: 512 bytes Cylinders: 3909, TracksPerCylinder: 255, SectorsPerTrack: 63 Partition type: MBR, NB Partitions: 1 Disk ID: 0x014962C1 Drive has a Windows 7 Master Boot Record Partition 1: Type: FAT32 LBA (0x0c) Size: 29.9 GB (32155631616 bytes) Start Sector: 2048, Boot: Yes ```
pbatard commented 7 years ago

Thanks for the updates.

At this stage, it looks like I'm going to have to produce a bugfix release, that reverts part of the changes from 2.13, because I have to admit that this issue appears to affect more users that I'm comfortable with, and it doesn't look like there's an easy way to pinpoint what other application (or possibly Windows service) is the one that is causing us trouble...

Hlsgs commented 7 years ago

I'll be watching this thread in case you want some further testing done.

pbatard commented 7 years ago

Thanks, I appreciate that. I'll probably make a TEST3 available for testing before I release 2.14, which I'll ask you and the other people facing the issue to test.

By the way, nice one with the <details> / <summary> HTML5 combo. I wasn't aware of those, but this is awesome to hide lengthy logs. According to this the only gotcha to use standard github markdown formatting inside (such as ```), is that there needs to be blank line between the </summary> tag and the following content (which I confirmed by editing your comment).

Hlsgs commented 7 years ago

Wasn't aware of that thought markdown didn't work, hence the <pre>. Makdown's definitely better, so thanks, cool.

pbatard commented 7 years ago

Alright, I have just uploaded rufus-2.14_TEST1.exe, which I expect to solve the issue.

The way it works is that, Rufus still tries to obtain exclusive write access, but if it can't get it after 5 seconds, it enables write sharing (just like with Rufus 2.12) and tries for 10 more seconds. You may see a warning such as the following in the log (and you'll get an extra 5 secs wait), but I expect the issue to go away:

Warning: Could not obtain exclusive rights. Retrying with write sharing enabled...

I'll greatly appreciate if the people who ran into the issue can let me know if the 2.14 TEST version works for them. If everything looks good, then I'll probably release the actual 2.14 tomorrow.

Hlsgs commented 7 years ago

I was able to replicate the issue several times and it all works fine, but with that warning like you explained. Again, this is only when using FAT32, With NTFS exclusive access seems to work like a charm. Thank you for your continued effort!

pbatard commented 7 years ago

Great! Thanks for the update.

renevane commented 7 years ago

Using the 2.14 TEST1 version I can also confirm it's working now (using write sharing). Thanks!

Found volume GUID \\?\Volume{8570d789-81f4-11e5-82e9-005056c00008}\
Waiting for access...
Warning: Could not obtain exclusive rights. Retrying with write sharing enabled...
Opened \\?\Volume{8570d789-81f4-11e5-82e9-005056c00008} [\Device\HarddiskVolume15] for write access
Writing partition boot record...
Using Standard NTFS partition boot record
Confirmed new volume has an NTFS boot sector
Successfully remounted Volume{8570d789-81f4-11e5-82e9-005056c00008}\ on I:\
Copying ISO files...
Extracting files...
jwwq commented 7 years ago

2.13/2.13test2, win7-64, win7 installation iso:

so it looks like it's not about just some other app blocking and even not about filesystem on the flash drive.

fails with "writing mbr... ... Could not open drive \?\Volume{8d5... used by another process" could not find blocking volume/device-id in process explorer, two different flash drives were used.

pbatard commented 7 years ago

FYI, Rufus 2.14 (which includes the change above) has now been released.

fails every time when MBR-BIOS/NTFS is selected

Yeah, it looks like this only applies when writing boot records... which only occurs when you don't create a pure UEFI bootable drive (such as MBR-UEFI/whatever or GPT-UEFI).

so it looks like it's not about just some other app blocking

Not really, if you include Windows and its services in the list of apps that can be blocking.

Windows tends to be a real drama queen about writing boot records (you need to have a handle for both the disk device AND any overlapping volume) and other low level stuff, which is precisely why I was trying to disable FILE_WRITE_SHARE in the first place.

If you look at the code form FormatThread() (which is the core of Rufus), you'll see that's it's closer to some kind of incantation, aimed at not to displeasing the irrational god that is Windows, than anything logical. Heck, literally half of the comments from that code are complaints about the various ways, which I all experienced, in which Windows will screw you up when trying to recreate a drive. So, I'm pretty confident that the issue has to do with something external to Rufus. Probably something like a service that detects boot records (because Windows is known to modify things like MBR's, even when it has no business doing so), and doesn't relinquish access.

The only thing we have established so far is that the file system doesn't matter (NTFS or FAT32 are affected) and that the version of Windows doesn't matter (Windows 10 and Windows 7 appear to be affected, with maybe a bit more people on the Windows 7 side), but that this is somehow related to writing boot records...

Krammig commented 7 years ago

Rubbish ! There is absolutely nothing accessing the USB device. Rebooted and ran, still get the error.

I've used another app that actually works.

JonnyTech commented 7 years ago

@Krammig which app are you referring to? Is it also open source? Maybe it can be of help here. Please bear in mind that this issue only seems to affect a small amount of users and will probably not manifest on your system. Rufus is attempting the correct and safe process of locking the device for exclusive write access which the other app may not.

pbatard commented 7 years ago

As far as I am concerned, unless someone is still seeing an issue in Rufus 2.14, which was released today, I couldn't care less about how much they want to complain. I believe I have taken the appropriate course of action with the new policy, which tries to obtain exclusive write access, but falls back to shared write access if that is not possible.

And for those who tend to only see issues in black and white, I also tried to provide some insights which I hope can help people forge a more nuanced view.

If that still leaves some Rufus users unsatisfied, so be it.

mkavidas commented 7 years ago

had the same problem as @jwwq last night. Works fine as long as file system is not NTFS AND all file explorers are closed (fails when any file explorer window is open). I am running windows 7. I will test 2.14/ look more into this. If not I can always use 2.12 for now.

I can see that you are handling it as you should (obtaining exclusive write access). Now I'm just wondering what process is interfering with that..

As this is a widespread issue, maybe having an alert that says "A process is blocking write access to the drive. Please end this process or enable the advanced setting ____. Warning: bla bla unstable". That way you can have a setting for people who have this issue, and can't for some reason find or kill the guilty process, the ability to continue.

Edit: I see you already did that.

prat-man commented 7 years ago

I can now confirm, having tested using the same ISO image and same flash drive, that Rufus 2.14 works perfectly where Rufus 2.13 failed. I think the policy you have adopted of "fall back strategy" is the best resolution. Thank you for fixing the issue.

byroniac commented 7 years ago

I can confirm that Paragon ExtFS for Windows (for viewing Linux ext4 partitions) causes this problem with Rufus (was wondering what was going on and about to pull my hair out), and the culprit is the service "DokanMounter" which can be stopped. For now I've just uninstalled the whole thing. But FWIW, in case anyone else uses that program, hope it helps.

byroniac commented 7 years ago

Arrrgghhhh!!! Something else hosed Rufus, too. Turns out it was the Front Panel monitoring software recommend by Airtop-PC for the Airtop-W that I purchased. Uninstalled. Problem solved. Sighhhh.... But fixed now!!! (EDIT: it is software called Open Hardware Monitor, and is very intrusive... even sets up a scheduled task to run this, had to disable that using Autoruns, but finally decided to just uninstall it and try to live without it).

pbatard commented 7 years ago

@byroniac, thank you very much for the testing!

The incompatibility between Paragon and Rufus is known (it's an issue that existed before the 2.13 release), and is listed at the end of the FAQ.

I wasn't aware about Open Hardware Monitor, so I have now added it to the list.

I'll just remind everyone to also have a look at the list of software that Rufus is incompatible with, in case they experience an issue.

byroniac commented 7 years ago

Insert embarrassed realization that I need to pay more attention to FAQs and make fewer attempts to "wing it" here. OK, thank you.

tcmaps commented 7 years ago

2.14 fixed the issues for me also (Win7x64 doing W10 Creators Update ISO)

i don'even know what process was blocking for me, as it's a rather fresh install the only thing running in background i have are Defender, Everything and RMClock

pbatard commented 7 years ago

Okay. As I am not hearing any more reports of the initial problem, now that 2.14 is out, I will close this issue.

JRexA commented 7 years ago

I know that this case is closed, but I have almost exactly the same problem, with Rufus 2.14 (Build 1086) I also had that problem, with 2.13

When I use a Fresh USB, it works, perfectly. When I try to reuse an USB, previously created with Rufus, it fails.

I tried the 2.15BETA, and that solved it, for me.

Agalyn77 commented 7 years ago

Just for information, had the same issue with 2.15. Had to close all the explorer windows opened and it worked. On windows 7 Enterprise

pbatard commented 7 years ago

What does the 2.15 log say? Rufus 2.15 attempts to tell you what process is causing the issue, which is then reported in the log. Have you looked at the log?

Agalyn77 commented 7 years ago

Sorry. But did not see any process in the log. I think I did not clear the log. Will try to see tomorrow. here is the message WARNING: The following process(es) or service(s) are accessing \Device\HarddiskVolume6: o 'Unknown_Process_11440' (pid: 11440, access: r)

Now I have this error, when I try to make my USB key: F:\Deploy\Operating Systems\Standard Build 3.1e\WIN7X64SYS.wim (3.7 GB) libcdio: File offset out of bounds Error reading UDF file /Deploy/Operating Systems/Standard Build 3.1e/WIN7X64SYS.wim

Is the wim too large ?

chsboot commented 7 years ago

Trying to write Windows10 ISO. All windows closed, no Explorer instance open.

It cannot be done with 2.15.1117 (Portable) Warning: Could not obtain exclusive rights. Retrying with write sharing enabled... but it works just fine with Rufus version: 2.11.995 (Portable)

I attached logs for 2.15 and 2.11

Rufus version: 2.15.1117 (Portable)
Windows version: Windows 7 SP1 64-bit
Syslinux versions: 4.07/2013-07-25, 6.03/2014-10-06
Grub versions: 0.4.6a, 2.02
System locale ID: 0x0111
Will use default UI locale 0x0111
SetLGP: Successfully set NoDriveTypeAutorun policy to 0x0000009E
Localization set to 'en-US'
Found USB device 'Kingston DataTraveler 108 USB Device' (0930:6545) [ID]
1 device found
Disk type: Removable, Sector Size: 512 bytes
Cylinders: 948, TracksPerCylinder: 255, SectorsPerTrack: 63
Partition type: MBR, NB Partitions: 1
Disk ID: 0x7486675F
Drive has a Rufus Master Boot Record
Partition 1:
  Type: NTFS (0x07)
  Size: 7.3 GB (7802126336 bytes)
  Start Sector: 2048, Boot: Yes
Scanning image...
ISO analysis:
  Image is an UDF image
Disk image analysis:
  Image does not have an x86 Master Boot Record
ISO label: 'ESD-ISO'
  Size: 3.4 GB (Projected)
  Uses: EFI
  Uses: Bootmgr
Using image: Windows10.iso

Format operation started
Requesting disk access...
Opened \\.\PHYSICALDRIVE3 for exclusive write access
Requesting lock...
Will use 'Z' as volume mountpoint
I/O boundary checks disabled
Requesting lock...
Analyzing existing boot records...
Drive has a Rufus Master Boot Record
Volume has an unknown Partition Boot Record
Clearing MBR/PBR/GPT structures...
Erasing 128 sectors
Deleting partitions...
Partitioning (MBR)...
Closing existing volume...
Waiting for logical drive to reappear...
Formatting (NTFS)...
Using cluster size: 4096 bytes
Quick format was selected
Creating file system...
Format completed.
Writing master boot record...
Drive has a Zeroed Master Boot Record
Set bootable USB partition as 0x80
Using Rufus MBR
Found volume GUID \\?\Volume{861233f2-2f7a-11e2-b644-004056c00008}\
Waiting for access on \\?\Volume{861233f2-2f7a-11e2-b644-004056c00008} [\Device\HarddiskVolume30]...
Warning: Could not obtain exclusive rights. Retrying with write sharing enabled...
Rufus version: 2.11.995
Windows version: Windows 7 SP1 64-bit
Syslinux versions: 4.07/2013-07-25, 6.03/2014-10-06
Grub versions: 0.4.6a, 2.02~beta3
System locale ID: 0x0111
Will use default UI locale 0x0111
Found USB 2.0 device 'Kingston DataTraveler 108 USB Device' (0930:6545)
1 device found
Disk type: Removable, Sector Size: 512 bytes
Cylinders: 948, TracksPerCylinder: 255, SectorsPerTrack: 63
Partition type: MBR, NB Partitions: 1
Disk ID: 0x74880706
Drive has a Rufus Master Boot Record
Partition 1:
  Type: NTFS (0x07)
  Size: 7.3 GB (7802126336 bytes)
  Start Sector: 2048, Boot: Yes
Scanning image...
ISO analysis:
  Image is an UDF image
Disk image analysis:
  Image does not have an x86 Master Boot Record
ISO label: 'ESD-ISO'
  Size: 3638820864 bytes
  Uses: EFI
  Uses: Bootmgr
Using image: Windows10.iso

Format operation started
Requesting disk access...
Opened drive \\.\PHYSICALDRIVE3 for write access
Will use 'G:' as volume mountpoint
I/O boundary checks disabled
Analyzing existing boot records...
Drive has a Rufus Master Boot Record
Volume has an unknown Partition Boot Record
Clearing MBR/PBR/GPT structures...
Erasing 128 sectors
Deleting partitions...
Partitioning (MBR)...
Closing existing volume...
Waiting for logical drive to reappear...
Formatting (NTFS)...
Using cluster size: 4096 bytes
Quick format was selected
Creating file system...
Format completed.
Writing master boot record...
Drive has a Zeroed Master Boot Record
Set bootable USB partition as 0x80
Using Rufus MBR
Found volume GUID \\?\Volume{861733f2-2f7a-11e2-b644-004056c00008}\
Opened drive \\?\Volume{861733f2-2f7a-11e2-b644-004056c00008} for write access
Writing partition boot record...
Using Standard NTFS partition boot record
Confirmed new volume has an NTFS boot sector
Successfully remounted Volume{861233f2-2f7a-11e2-b644-005056c00008}\ on G:\
Copying ISO files...
Extracting files...
Image is an UDF image
Extracting: Z:\boot\en-us\bootsect.exe.mui (16 KB)
Extracting: Z:\boot\fonts\chs_boot.ttf (3.5 MB)
..etc
byroniac commented 7 years ago

Could possibly be either the antivirus or the file indexer grabbing that volume, I think?

pbatard commented 7 years ago

Unless Rufus crashed, there should definitely after your last line from the 2.15 log.

Can you please retry and make sure you paste the complete log?