users.csv does not flash correctly on Windows

[davetelling]

I have an MD-380 UHF radio. I did the firmware and users.csv update, following the instructions for the Windows O/S. I tried on a W10 and a W7 system, with the same result. The make processes ran without errors, and showed the SPI flash being erased and written with the new database. However, when I get a call, I get a green pop-up window that says that the user ID can't be found in the database. I have no idea what to do at this point. As far as I know, the radio has a 16mbit SPI flash. Is there any diagnostic or something else I can do to figure out what is going on (or not going on...)? I enabled the UserCSV option in the menu.

[aeickho]

you have flashed the UserCSV in normal operating mode? after fw flash put the radio off an on.... make flashdb --> when flashing is finished the radio is rebooting automatically.

-alex

[davetelling]

Alex, Yes, I have done this multiple times - everything goes according to the instructions, but no info shows up in the green window.

On 7/19/2016 5:38 PM, alex, df8av wrote:

you have flashed the UserCSV in normal operating mode? after fw flash put the radio off an on.... make flashdb --> when flash is finished the radio is rebooting automatically.

-alex

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/travisgoodspeed/md380tools/issues/186#issuecomment-233807458, or mute the thread https://github.com/notifications/unsubscribe-auth/ATexYMvA2yIaXKJiLkHP3TlUArkGtNC_ks5qXW4jgaJpZM4JQRFq.

[aeickho]

please check on folder db users.csv and data.csv

[davetelling]

In the db folder, I do have a file users.csv, I DO NOT have a file data.csv there. However, I have a data.csv in the md380tools folder. Should it be in the db folder?

On 7/19/2016 5:59 PM, alex, df8av wrote:

please check on folder db users.csv and data.csv

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/travisgoodspeed/md380tools/issues/186#issuecomment-233810229, or mute the thread https://github.com/notifications/unsubscribe-auth/ATexYJVdIPsAdVb--YPbdKxgJSMJVgxTks5qXXLagaJpZM4JQRFq.

[davetelling]

OK, just for fun, I moved the data.csv into the db folder. I ran make flashdb and same result.

[iball]

OK, sometimes the user calling you is new and unless you delete the users.csv in the db directory, it will NOT fetch a fresh copy of the users.csv file from the DMR-MARC ID database dump. If it's a new ID that's not in the users.csv you're using then it's going to give you that "error". I usually refresh them on my MD-380s/390 once a week. More often if we get a lot of new users in my area.

[davetelling]

I checked the users.csv file with ID's shown in the green popup window, and all were already in the database, so that doesn't appear to be the problem, either. Several people have suggested that the database install can only be done under Linux - is there any truth to that?

On 7/19/2016 10:12 PM, Nonya wrote:

OK, sometimes the user calling you is new and unless you delete the users.csv in the db directory, it will NOT fetch a fresh copy of the users.csv file from the DMR-MARC ID database dump. I usually refresh them on my MD-380s/390 once a week. More often if we get a lot of new users in my area.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/travisgoodspeed/md380tools/issues/186#issuecomment-233839872, or mute the thread https://github.com/notifications/unsubscribe-auth/ATexYBQUKwVSDvt-OAtLFtHnBUA3g2Ruks5qXa4xgaJpZM4JQRFq.

[iball]

Might be. I use a Raspberry Pi 3 for all this stuff myself.

[davetelling]

Well... after beating my head against a wall for days, I installed Mint Linux in a dual-boot configuration on my W10 machine, beat my head against a wall for a few more hours trying to get the various apps & files set up on Linux (whoever said Linux is easy was... misinformed!), I finally tried the make flashdb operation - and it worked! Users name and call show up in the little green window. So, the moral of this story is that, apparently, you can't flash the users.csv database in Windows, even though the computer SAYS that it is writing the database. Thanks for all the suggestions from others, now I have additional Linux intricacies to learn...

[travisgoodspeed]

Perhaps Dave's troubles came from CRLF line endings?

[d235j]

This sounds like a bug though; I'll reopen it and name it appropriately, and then look into the problem. @travisgoodspeed hmm, that's certainly possible.

[m0ivq]

Is there a way to delete the database from the radio? I had already run flashdb on linux prior to windows so can't confirm if this is an issue or not.

[travisgoodspeed]

"make flashdb" erases the old database as it writes the new one.

[m0ivq]

Yes but if under windows the database isn't flashing then it may not be erasing either.

[m0ivq]

Ive managed to erase the flash by modifying md380-tool. Just need to confirm has erased before I try flashing on windows but nobody seems to be on air tonight!

[iball]

In Linux, make flashdb does NOT erase the old database - it uses any pre-existing users.csv file it finds in the db directory.
I have to manually delete the users.csv file in the db directory and then run make flashdb in order to get it to pull down a fresh copy from DMR-MARC.

[m0ivq]

'make flashdb' works correctly on Windows 10 32-bit if I change all instances of '$(MAKE)' to 'make' in relevant Makefiles, otherwise fails due to the space in 'C:\Program Files' (default installation folder for make for windows). I have verified this by erasing the flash on a Linux machine (added an input() line between erasing the flash and writing the new in md380-tool, then ctrl-c to prevent writing of new flashdb), then ran make flashdb on windows. Contact info displays correctly on receive.

[travisgoodspeed]

$(MAKE) is needed for BSD, where 'make' and 'gmake' are different programs.

I'm nervous that if we work around too many Windows bugs, we'll still be stuck with random breaks. Might it be easier to recommend Cygwin or Windows Services for Linux, which won't be so clunky?

[d235j]

The issue here with Windows is that the Make program is in a path containing a space, which causes various issues for software intended for POSIX. I suggest using MSYS2 as described in the instructions.

[m0ivq]

I've amended the Windows Instructions so the PATH variable now uses 'C:\Progra~1\' instead of Program Files

[m0ivq]

Is it possible to read the spiflash so we can compare the difference between flashing on Windows and Linux. I'm unable to reproduce the issue on 32-bit Windows 10. I will try 64-bit once I have a new screen for my laptop.

[aeickho]

what is the status of this issue?

[travisgoodspeed]

@M0IVQ, You can use md380-tool spiflashdump image.bin to dump the complete 16MB SPI Flash image. Users.csv is at 0x100000.

@aeickho Perhaps we should probably choose one Windows environment--either GNUWin32, Windows Services for Linux, or Cygwin--and test it regularly while abandoning the other two?

[m0ivq]

@aeickho I will compare flash dumps at the weekend to see what the differences are between flashing on Windows and Linux. I saw the same issue myself yesterday so it does in fact happen with 32-bit windows 10. I'm not getting it with every contact but the ones that do show the error are in the csv file.

[iball]

Curious. Maybe put an exception in your PC antivirus to not scan the md380tools folder on your PC? Perhaps the AV could be corrupting the file as it tries to read it in real-time when it's being accessed during the flash process?

[m0ivq]

I performed a make flashdb on Windows 10 32-bit (using GnuWin32) and then used my Ubuntu machine to perform an spiflashdump - this shows that there is nothing in Flash from offset 0x100000 (all FF). Repeating after running make flashdb on linux performed as expected.

[m0ivq]

@aeickho @travisgoodspeed - It would appear that if using GnuWin32 to flashdb on Windows, the flash doesn't erase either - giving the illusion that the user database has flashed correctly if you have already used Linux to flash the database.

However, Cygwin appears to work correctly (apart from shasum commands don't work on mine unless I change them to sha256sum - this hopefully is just a case of installing shasum when I can find it). Maybe we should drop GnuWin32 unless someone can work out why this doesn't work?

[yoony64]

My radio was originally a V13.20 firmware, before modifying. I mistakenly used a V2.32 based patch firmware and everything seems to work(flashdb ran without errors), but enabling users.csv, doesn't actually find any records..

With this wrong firmware, the make flashdb thinks the SPI is 10 dc 1.

Once I corrected the firmware to the patched 13.20 version, now the SPI ID says 70 f1 1 and make flashdb fails with unknown ID.

I am running windows 7 64 bit, and was originally using git, but now I'm using Cygwin and having the same issues.

[travisgoodspeed]

Does anyone know why his flash ID would change?

[m0ivq]

I have confirmed these findings:

With D13.20: On Linux make flashdb works fine, with SPI Flash ID of ef 40 18 Under Cygwin, make flashdb now fails with SPI Flash ID of 70 f1 1

With D02.032, On Linux, works with SPI Flash ID of ef 40 18 Under Cygwin, works but with SPI Flash ID of 10 dc 1

My gut feeling is there is an issue with libusb-win32. When I get some time I'll sniff the USB data to see what's going on.

Flashing of firmware works fine on both linux and windows.

[m0ivq]

Correction - with D02.032, Cygwin goes through the motions but doesn't write anything to the radio. I obviously didn't erase the flash prior to trying when I originally did the test a month ago. In which case, neither GnuWin32 or Cygwin work with make flashdb. This almost certainly points to some issue with libusb-win32.

@travisgoodspeed @aeickho - what is done differently between flashing firmware to the radio and the flashdb on the PC side other than the radio being in firmware upload mode?

[yoony64]

While bouncing back and forth from LibusK and libusb0, using Zadig, I was able to flash the users.csv, using GitBash, under windows. When it worked, the flash ID shows EF 40 18.

I subsequently ran Inf Wizard and re-assigned libusb-win32, I was no longer able to flashdb. It always came up with 70 f1 1 for SPI and I was not able to flash. Further swapping between libusK and libusb0 using Zadig had no affect.

My radio is able to pull up the users and properly display their callsign, due to the one time that it flashed correctly.

[m0ivq]

@yoony64 are you able to go into device manager, right-click on the libusb-32 device for the radio, select uninstall and tick the box to delete the driver file. Then try Zadig again and see what happens?

[iball]

For what it's worth, in my experience, I have had no issues flashing radios using any of the following methods:

1) Raspberry Pi in a PiTop laptop case running latest version of Raspbian OS. Both firmware update mode and flashdb/codeplug regular radio mode. This is my main mobile "let's reflash other users radios on-the-road" device since I can also use it to load pre-built codeplugs into radios as well.

2) Windows 10 desktop and Windows 10 laptop running VirtualBox with latest version of Ubuntu OS - I did do a static mapping of the USB device for the radios so that Virtualbox always grabs the radio when it's plugged into the desktop or laptop. Both firmware update mode and flashdb/codeplug regular radio mode.

[tdicola]

For what it's worth I couldn't get the make flashdb command to work on Windows with the latest firmware either. I kept getting the SPI flash error mentioned above. I switched to an Ubuntu machine, compiled the project, and the flashdb command worked great (even with firmware built and flashed previously on Windows). For anyone running into the same issue you probably want to setup an Ubuntu VM with USB passthrough (if using virtualbox you need the virtualbox extension pack and then add the radio VID & PID as a persistent USB filter: https://help.ubuntu.com/community/VirtualBox/USB ) and do all the programming, flashing, etc. there.

[m0ivq]

I've been looking into this issue and have made the following observations (most of which is a summary of above to save reading through it all):

• The correct SPI Flash ID is: ef 40 18
• On Windows, using Git Bash with GnuWin32, the SPI Flash ID is reported as:
  • 10 dc 01 on D002.032
  • 70 f1 01 on D013.020
• Occasionally, under Linux, the SPI Flash ID is reported as 10 dc 01 on D002.032
• Firmware flashes fine under Windows.
• md380-dfu read & write both work fine under Windows but from what I can make out, these use existing commands in the radio, where as flashing the userdb and reading the SPI Flash ID use custom commands.

I am wondering if there is a timing issue of some sort which affects LibUSB-win32, especially as some Linux machines report the wrong SPI Flash ID (10 dc 10). When I have some hardware, I will attempt to compare the raw USB data.

[roelandjansen]

I always see SPI Flash ID: ef 40 18 even if the hypervisor/host is busy. This is with vmware workstation.

[m0ivq]

Ed N6YN posted the following fix in the google group:

1. With either patched firmware v2.032 or D13.020 installed in your radio...
2. run Zadig, and install libusb-win32 (v1.2.6.0) or (libusb-win32 from: https://sourceforge.net/projects/libusb-win32/ as described in MD380tools / Windows-instructions.txt) Once the driver is installed you will see that if you issue a ./md380-tool spiflashid command, you will received either a # 1 or #2 from the above responses.
3. Now, execute the "testlibusb-win.exe" file, then click on the Refresh button at the bottom of the popup window. (Note this file is located in the libusb-win32-bin-1.2.6.0\bin\x86 folder). Once you click on the Refresh button you will find that if you again issue a ./md380-tool spiflashid command in the MINGW32 window, you will receive the proper response from the radio: SPI Flash ID: EF 40 18 W25Q128FV 16MByte

Now you can issue a make flashdb and your CSV file will be properly installed into the SPI Flash. (remember to perform a "make clean all" in your md380tools/db folder, to update the "users.csv" file)

Close this issue once above fix has been confirmed and GNUWin32 instruction have been updated accordingly. Self assigning unless someone gets time to do it before me :)

[m0ivq]

Windows instructions have now been updated to include the fix. Closing this issue.

[KG5RKI]

If you do this before any other usb operations it will fix the spi flash ID issue and will not require testlibusb.exe for windows.

UInt32 Result = STDFU.STDFU_SelectCurrentConfiguration(ref hDevice, 0, 0, 0); STDFU.DFU_Status dfuStatus = new STDFU.DFU_Status(); STDFU.STDFU_GetStatus(ref hDevice, ref dfuStatus); STDFU.STDFU_ClrStatus(ref hDevice); IntPtr strinng = Marshal.AllocHGlobal(256); Result = STDFU.STDFU_GetStringDescriptor(ref hDevice, 1, strinng, 256); STDFU.STDFU_GetStatus(ref hDevice, ref dfuStatus); STDFU.STDFU_ClrStatus(ref hDevice); Marshal.FreeHGlobal(strinng);